RAG Retrieval Empty Results Behind HAProxy In CI/CD Pipeline

Q: Investigation and Debugging Steps 1. Capture HAProxy logs with full HTTP details

# Enable detailed HTTP logging in haproxy.cfg global log /dev/log local0 log /dev/log local1 notice daemon defaults log global mode http option httplog option dontlognull timeout connect 5s timeout client 30s timeout server 30s Then reload HAProxy and tail the log:

Q: 4. Confirm timeout hits

Enable HAProxy’s option httplog and look for “client closed connection while waiting for request body” in the logs (HAProxy Logging and Debugging).

Q: Solution – Harden HAProxy for RAG workloads in CI Before – Problematic configuration snippet

# haproxy.cfg (original) frontend http-in bind *:80 default_backend rag-backend backend rag-backend balance roundrobin server srv1 10.0.0.11:8080 check server srv2 10.0.0.12:8080 check server srv3 10.0.0.13:8080 check After – Robust configuration # haproxy.cfg (fixed) global log /dev/log local0 daemon maxconn 5000 defaults log global mode http option httplog option dontlognull timeout connect 5s timeout client 60s # increased for large payloads timeout server 60s option http-buffer-request # enable request body buffering tune.bufsize 32768 # raise internal buffer size if needed frontend http-in bind *:80 default_backend rag-backend # Preserve request body on retries option http-reuse safe # Enable sticky sessions based on source IP stick-table type ip size 1m expire 30s store http_req_rate(10s) stick on src backend rag-backend balance source # ensures same client hits same replica option httpchk GET /health server srv1 10.0.0.11:8080 check inter 2s rise 2 fall 3 server srv2 10.0.0.12:8080 check inter 2s rise 2 fall 3 server srv3 10.0.0.13:8080 check inter 2s rise 2 fall 3 # Do not retry without body retries 0 Why the fix works:

In this article

Table of Contents

Problem – RAG retrieval returns empty results behind HAProxy in CI/CD pipelines

During automated end‑to‑end tests a Retrieval‑Augmented Generation (RAG) service intermittently returns no documents. The symptom is observed only when the request traverses an HAProxy instance that load‑balances traffic across several identical backend replicas.

Typical error messages:
- HAProxy log entry: <...> 0/0/0/30/30 504 0 - - ---- 0/0/0/0/0 0/0 (gateway timeout)
- Backend log: Received empty query payload – returning no results
- HTTP response from HAProxy: 502 Bad Gateway with an empty body
Impact: Test suites that validate knowledge‑base look‑ups fail, causing false negatives in CI pipelines and blocking merges.

Root Cause Analysis

The empty results stem from three interacting HAProxy behaviours that are easy to overlook in CI environments:

Request body buffering limits – HAProxy buffers HTTP request bodies only when option http-buffer-request is enabled (see HAProxy HTTP Mode and Buffering). In CI runners the JSON payload for a RAG query can exceed the default 1 MiB limit, causing the request to be truncated or dropped. The backend then receives a zero‑length body and logs “empty query”.
Timeout configuration – The default timeout client is 30 s (HAProxy Timeout Settings). Large payloads combined with the extra network hop in CI can push the client‑side timeout, leading HAProxy to abort the connection and emit a 504 status. The backend never sees the query.
Lack of session affinity (sticky sessions) – In a CI run parallel jobs hit the same HAProxy frontend. Without balance source or cookie‑based stickiness, a request may be routed to a newly started replica that has not yet loaded the vector index. The service answers with “0 documents (status: EMPTY)”, as documented in the real incident “missing sticky configuration caused queries to be routed to a warm‑up replica”.

These factors explain why the problem appears only under load‑balanced CI conditions and not in local development where HAProxy is bypassed.

Investigation and Debugging Steps

1. Capture HAProxy logs with full HTTP details

# Enable detailed HTTP logging in haproxy.cfg
global
    log /dev/log    local0
    log /dev/log    local1 notice
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5s
    timeout client  30s
    timeout server  30s

Then reload HAProxy and tail the log:

sudo systemctl reload haproxy
journalctl -u haproxy -f

Look for entries like:

Oct 12 14:22:31 haproxy[1234]: :54321 [12/Oct/2026:14:22:31.123] http-in~ backend/server1 0/0/0/30/30 504 0 - - ---- 0/0/0/0/0 0/0

2. Verify request body size handling

Send a deliberately large RAG query from the CI runner and capture the request at HAProxy with tcpdump:

sudo tcpdump -i any -s 0 -w /tmp/haproxy.pcap port 80
curl -X POST http://haproxy.example.com/rag/query \
     -H "Content-Type: application/json" \
     --data @large_query.json -v

Inspect the capture with tcpdump -r /tmp/haproxy.pcap -A. If the payload stops after ~1 MiB, buffering is the culprit.

3. Check backend replica readiness

From a CI job, query the health endpoint of each backend directly:

for ip in 10.0.0.11 10.0.0.12 10.0.0.13; do
    curl -s http://$ip:8080/health | grep ready && echo "$ip ready"
done

If some instances report “not ready”, they are likely the warm‑up replicas serving the empty results.

4. Confirm timeout hits

Enable HAProxy’s option httplog and look for “client closed connection while waiting for request body” in the logs (HAProxy Logging and Debugging).

Solution – Harden HAProxy for RAG workloads in CI

Before – Problematic configuration snippet

# haproxy.cfg (original)
frontend http-in
    bind *:80
    default_backend rag-backend

backend rag-backend
    balance roundrobin
    server srv1 10.0.0.11:8080 check
    server srv2 10.0.0.12:8080 check
    server srv3 10.0.0.13:8080 check

After – Robust configuration

# haproxy.cfg (fixed)
global
    log /dev/log    local0
    daemon
    maxconn 5000

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5s
    timeout client  60s          # increased for large payloads
    timeout server  60s
    option  http-buffer-request # enable request body buffering
    tune.bufsize 32768           # raise internal buffer size if needed

frontend http-in
    bind *:80
    default_backend rag-backend
    # Preserve request body on retries
    option  http-reuse safe
    # Enable sticky sessions based on source IP
    stick-table type ip size 1m expire 30s store http_req_rate(10s)
    stick on src

backend rag-backend
    balance source               # ensures same client hits same replica
    option  httpchk GET /health
    server srv1 10.0.0.11:8080 check inter 2s rise 2 fall 3
    server srv2 10.0.0.12:8080 check inter 2s rise 2 fall 3
    server srv3 10.0.0.13:8080 check inter 2s rise 2 fall 3
    # Do not retry without body
    retries 0

Why the fix works:

option http-buffer-request forces HAProxy to fully read and buffer the JSON payload before forwarding, preventing truncation (evidence: GitHub issue in LangChain about request body truncation).
Increasing timeout client and timeout server to 60 s avoids premature 504 errors on large queries (evidence: CI pipeline with default 30 s timeout cutting off bodies).
Using balance source together with a stick table creates session affinity, ensuring that a warm‑up replica does not receive a query before its vector index is loaded (evidence: real incident about missing sticky configuration).
Setting retries 0 stops HAProxy from silently re‑issuing the request without the body, which was the cause of empty payloads in GitHub Actions (LlamaIndex issue).

Verification – Confirm the issue is resolved

Log inspection – After redeploy, tail HAProxy logs and verify that no 504 or “client closed connection while waiting for request body” entries appear during a test run.
Backend payload check – Add a temporary log line in the RAG service to dump the incoming payload size:
```
logger.info(f"Received query payload size: {len(request.body)}")
```
Ensure the size matches the original large_query.json (e.g., >1 MiB).

Functional test – Run the CI job that performs a RAG lookup and assert that the response contains at least one document. Example:

response=$(curl -s -X POST http://haproxy.example.com/rag/query \
        -H "Content-Type: application/json" \
        --data @large_query.json)
    echo "$response" | jq '.documents | length'   # should be > 0

Health‑check validation – Verify that all backend replicas report ready before the load balancer starts sending traffic (the option httpchk line does this automatically).

Prevention – Operational guardrails for future pipelines

Monitoring: Create a HAProxy metric alert on http_err_504 and on bytes_in drops. Correlate with CI job latency.
Alerting: Trigger a PagerDuty alert if the backend logs “Received empty query payload” more than twice in a 5‑minute window.
Configuration review: Enforce a CI lint step that validates option http-buffer-request and timeout client values are above the minimum required for your payload size.
Sticky session policy: Adopt balance source or cookie‑based stickiness for any stateful service (vector store, embedding cache) behind HAProxy.
Capacity planning: Periodically benchmark the maximum JSON payload size your RAG endpoint can handle and adjust tune.bufsize accordingly.

FAQ – Common follow‑up questions

Why does the problem appear only in CI and not locally?
CI runners often have higher network latency and stricter resource limits, causing HAProxy timeouts and body‑size throttling that are not hit during local development.
Can I keep HAProxy’s default timeouts and still avoid empty results?
Yes, but you must ensure the request body fits within the default 1 MiB buffer and that the client can complete the upload within 30 s. For large RAG queries, adjusting the timeouts is the more reliable approach.
Do I need sticky sessions for all RAG services?
Sticky sessions are required only when the backend maintains in‑memory state (e.g., loaded vector indexes) that is not instantly available on every replica. If all replicas load the index on start‑up, stickiness is optional.
What HAProxy version introduced option http-buffer-request?
The option has been available since HAProxy 1.5, but the default behaviour changed in 2.0. Using HAProxy 2.8 (current LTS) guarantees full support.
How can I test that request bodies are not being truncated without looking at logs?
Add a checksum (e.g., SHA‑256) of the JSON payload in a custom header before sending the request, and have the backend verify the checksum. A mismatch indicates truncation.

Related Topic Hub: Distributed Systems Troubleshooting Hub