Jordan Lee

Hugging Face Transformers evaluation webhook timeout during CI/CD run

by
Robotic figure with blue and red details against sky

Problem – Webhook Timeout During Hugging Face Transformers Evaluation in CI/CD During the Trainer.evaluate step of an automated CI/CD pipeline, the post‑evaluation callback that ships evaluation metrics to an external monitoring service (e.g., Prometheus Pushgateway, Datadog, custom HTTP endpoint) consistently fails with a timeout error. The CI job aborts after the runner‑enforced timeout (typically 30 seconds) and marks … Read more

PyTorch batch size overflow during live video streaming inference

by
a purple and black background with a purple and black logo

Problem – Batch Size Overflow During Live Video Streaming Inference In a production pipeline that consumes a continuous stream of video frames, the inference service throws runtime errors such as: RuntimeError: Expected tensor size [16, 3, 720, 1280] but got [32, 3, 720, 1280] AssertionError: batch size exceeds max_batch_size (configured in TorchServe) These errors appear … Read more

Kafka topic creation fails with TopicExistsException after docker‑compose up

by
white concrete building under blue sky during daytime

Problem: Kafka topic creation fails with TopicExistsException after docker‑compose up During integration testing of an AI model inference pipeline, the test harness attempts to create a Kafka topic named inference-logs via the AdminClient. The call consistently returns an error similar to: org.apache.kafka.common.errors.TopicExistsException: Topic ‘inference-logs’ already exists. Despite the exception, the topic is not visible in … Read more

TensorRT engine collection creation failure during CI benchmark on A100

by
white and black wooden wall mounted signage

Problem Description During an automated CI run that benchmarks TensorRT models on an NVIDIA A100, the trtexec command fails when attempting to create an engine collection. The job aborts with one of the following messages (observed in multiple CI runs): [Error] Engine collection creation failed (code: 1) Engine collection creation failed: Unsupported GPU architecture Engine … Read more

LlamaIndex filter expression parse error during Kubernetes autoscaling

by
a rack of servers in a server room

Problem Description When querying a LlamaIndex microservice deployed in a Kubernetes cluster, the service returns a FilterExpressionParseError for any query that contains complex boolean or range filters. Typical error messages observed in the pod logs are: FilterExpressionParseError: Unexpected token ‘AND’ at position 12 ValueError: Could not parse filter expression: syntax error near ‘>=’, expected ‘:’ … Read more

GPT-3.5 token limit exceeded after adding new user turn

by
a rack of servers in a server room

Problem Description Symptoms and Impact During development of a multi‑turn chatbot that uses the OpenAI GPT‑3.5‑turbo model, the API began returning errors after adding a new user turn. Typical log entries looked like: 2024-09-14 10:22:31,842 ERROR openai.error.InvalidRequestError: This model’s maximum context length is 4097 tokens Traceback (most recent call last): File “app.py”, line 112, in … Read more

ChromaDB RAG query decomposition error during multi‑GPU training

by
a rack of servers in a server room

Problem – RAG query decomposition failures in a multi‑GPU training pipeline When scaling a Retrieval‑Augmented Generation (RAG) workflow to multiple GPUs, engineers observed that chroma_client.query calls intermittently returned malformed or empty sub‑queries. The downstream LLM received no context, leading to generation failures or hallucinations. Typical error messages included: ValueError: Query decomposition returned an empty list … Read more

Token limit exceeded error in Nginx during real-time LLM streaming

by
a rack of servers in a server room

Problem Description A real‑time LLM streaming service proxies requests through Nginx (HTTP/2). When the model generates responses that exceed a few kilobytes of token data, the client receives a truncated stream and the Nginx error log contains entries such as: 2026/09/13 12:45:27 [error] 12#12: *12345 client intended to send too large body, request body size … Read more

RAG answer extraction empty after model evaluation in Kubernetes

by
a group of blue squares

Problem Description During automated model evaluation runs in a CI/CD pipeline on a Kubernetes cluster, the Retrieval‑Augmented Generation (RAG) microservice returns an answer field that is either empty or malformed. The API response typically looks like: { “question”: “What is the capital of France?”, “answer”: “” } Typical log excerpts from the RAG pod include: … Read more

RAG decomposition service throws DecompositionFailedException on complex queries

by
Black cables and wires connected to the back of modular LED display panels

Problem: DecompositionFailedException in RAG Query Decomposition Service The RAG decomposition microservice consumes user queries from the rag.requests exchange, splits them into sub‑queries, and republishes each sub‑query to topic queues such as rag.subqueries.*. When a complex query (e.g., nested JSON, multi‑entity request, or very long prompt) is processed, the service throws DecompositionFailedException. The exception propagates, the … Read more