RAG reranker timeout after rolling update of Anthropic Claude service
Problem – RAG Reranker Timeout After Rolling Update of Anthropic Claude Service The Retrieval‑Augmented Generation (RAG) pipeline uses a reranker microservice that forwards candidate passages to the Anthropic Claude LLM for scoring. After a rolling update of the Claude service across a 6‑node fleet, the reranker began returning HTTP 504 timeout errors for POST /rerank calls. … Read more