Model Serving

MLflow training job rejected by Kubernetes admission controller

by
photo of computer cables

Problem Description When launching an MLflow training run that uses the Kubernetes backend, the pod creation fails with an admission‑controller error. The MLflow client reports a kubectl error similar to the following: Error from server (Forbidden): pods “mlflow-train-7f9c8d5b9-xyz” is forbidden: exceeded quota: compute-resources, request: cpu=8, memory=16Gi Other observed messages include: admission webhook “validation.gatekeeper.sh” denied the … Read more

RAG retrieval empty results with vLLM in hybrid cloud

by
A ribbon with gradient colors floats in the air.

Problem Description In a hybrid‑cloud deployment of a Retrieval‑Augmented Generation (RAG) pipeline that uses vLLM for inference, the retrieval step consistently returns an empty list or null despite: Valid user queries arriving at the API endpoint. Document embeddings successfully indexed in the vector store (e.g., Milvus or Pinecone). No errors reported by the vLLM server … Read more

ONNX Runtime inference fails after CI/CD deployment due to incorrect routing

by
img IX mining rig inside white and gray room

Problem – ONNX Runtime Inference Fails After CI/CD Deployment During a routine CI/CD rollout, newly built ONNX model containers are deployed to a Kubernetes cluster. After the deployment completes, API calls that should hit the onnx-runtime service return HTTP 502/504 errors and the ONNX Runtime logs show Failed to load model: file not found. The … Read more

PyTorch CRD validation failure due to tensor dimension mismatch

by
a neon circle with a snake on it

Problem – CRD validation failure due to tensor dimension mismatch During a multi‑node distributed training run, the pipeline aborts with a validation error similar to: ValueError: Expected tensor of shape (N, C, H, W) but got (N, C, H, W, 1) – CRD validation failed Other observed symptoms include: Inconsistent batch shapes reported by the … Read more

vLLM text image modality mismatch after GPU passthrough

by
A ribbon with gradient colors floats in the air.

Problem – Text‑Image Modality Mismatch after GPU Passthrough When running a multimodal model (e.g., llava‑1.5‑7b) inside a Docker container that uses the NVIDIA Container Toolkit, the generated responses contain text that is unrelated to the supplied image. The model appears to process the textual prompt correctly but ignores the visual content, resulting in incoherent or … Read more