indexing

LlamaIndex inference queue backlog during A/B testing

by
photo of computer cables

Problem – Inference Queue Backlog During A/B Testing During a recent high‑traffic A/B test two LlamaIndex variants (A and B) were deployed side‑by‑side in a Kubernetes cluster. Users of the test variant experienced: Latency spikes from ~200 ms to >30 s. HTTP 503 responses with the log line Inference queue full: max_queue_size reached. Increasing numbers of “Worker … Read more

ONNX Runtime index rebuild failure during batch inference

by
img IX mining rig inside white and gray room

Problem Description During high‑throughput batch inference the ONNX Runtime throws an exception similar to: Ort::Exception: Failed to rebuild model index: InvalidArgument – Model index is corrupted. or RuntimeError: Unable to rebuild model index after session reset – possible concurrent modification. The failure occurs after the first batch finishes and subsequent batches either produce incorrect results … Read more

LlamaIndex training loop rejected by admission controller Kubernetes

by
a rack of servers in a server room

Problem Description The LlamaIndex training loop fails to start in a Kubernetes‑based ML platform. The pod creation request is rejected by the llamaindex-validation admission webhook. Typical error output from kubectl describe pod looks like: Error from server (BadRequest): admission webhook “llamaindex-validation” denied the request: unauthorized access to secret “llama-index-config” Other observed rejections include: “pod llamaindex-trainer-7f9c9d … Read more

LlamaIndex token limit exceeded during model evaluation

by
photo of computer cables

Problem – Token Limit Exceeded During Model Evaluation with LlamaIndex When running automated evaluation of a large document corpus through LlamaIndex, the evaluation script aborts with errors such as: openai.error.InvalidRequestError: This model’s maximum context length is 4096 tokens ValueError: Prompt exceeds max_input_size (got 5273 tokens, max allowed 4096) RuntimeError: Token limit exceeded while building prompt … Read more

Haystack query decomposition error with FAISS index on A100 GPU

by
brown round grass on green grass field during daytime

Problem Description In a distributed Retrieval‑Augmented Generation (RAG) pipeline built with Haystack 1.19.x, the query_decomposition step crashes when the FAISS index is placed on an A100/H100 GPU. Typical error messages observed in the logs are: Traceback (most recent call last): File “…/haystack/pipeline/pipeline.py”, line 312, in run results = component.run(**inputs) File “…/haystack/nodes/retriever/faiss.py”, line 421, in run … Read more