Rows of black server racks with white logos in a data center

LLM API function call format error after HAProxy header rewrite

by

Problem – Function‑Calling Format Error When LLM API Is Behind HAProxy

During model evaluation the inference service returns a 400 Bad Request with a body similar to:

{
  "error": {
    "code": "invalid_request_error",
    "message": "Function call JSON is malformed"
  }
}

The error only appears when the request traverses HAProxy, which is used to load‑balance multiple replica pods of the LLM service. Direct calls to the service (bypassing HAProxy) succeed.

Root Cause – Header Rewrite Introduces Invalid JSON Payload

HAProxy is configured to rewrite the Content-Type header and to inject a custom X-Request-ID. The rewrite rule uses the replace‑value directive on the entire header line:

http-request replace-header Content-Type (.*) application/json; charset=utf-8

This transformation appends a space and the literal string charset=utf-8 to the original header value. The LLM inference service expects the Content-Type to be exactly application/json (no parameters). When the header contains the extra ; charset=utf-8, the service’s request parser treats the body as a multipart form and attempts to parse the JSON as a form field, resulting in a malformed function‑call structure.

Additionally, the http-request set-header X-Request-ID rule mistakenly concatenates the original header value with the new UUID without a separator, producing an invalid header that some middleboxes interpret as part of the request body.

Debug – Step‑by‑Step Investigation

1. Capture the raw request at the HAProxy front‑end

# tcpdump -i eth0 -s 0 -w /tmp/haproxy_req.pcap port 443
# tshark -r /tmp/haproxy_req.pcap -Y http.request -V

Sample output (truncated):

GET /v1/chat/completions HTTP/1.1
Host: api.example.com
Content-Type: application/json; charset=utf-8
Content-Length: 342
X-Request-ID: 123e4567-e89b-12d3-a456-426614174000
...

2. Compare with a direct request (no HAProxy)

curl -X POST https://api.internal/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-4","messages":[...],"functions":[...],"function_call":"auto"}'

Resulting request headers:

Content-Type: application/json
X-Request-ID: 123e4567-e89b-12d3-a456-426614174000

3. Inspect HAProxy logs (enabled with option httplog)

Oct 24 12:45:01 haproxy[1024]: 10.0.2.15:54321 [24/Oct/2026:12:45:01.123] http-in~ api/example.com 0/0/2/5/8 200 1234 - - ---- 2/2/0/0/0 0/0 "GET /v1/chat/completions HTTP/1.1"

The log shows a 200 ms backend latency, but no indication that the request body was altered. HAProxy does not log header rewrites by default, so we added http-request capture hdr(Content-Type) len 64 id 0 to verify the exact value passed downstream.

4. Verify backend parsing error

# journalctl -u llm-inference.service -f
...
2026-10-24T12:45:01.456Z ERROR request_id=123e4567-e89b-12d3-a456-426614174000 \
  msg="Failed to parse function call payload" error="unexpected token at line 1"

Solution – Correct HAProxy Header Handling

1. Remove the unnecessary charset parameter

Replace the replace-header rule with a strict set operation that preserves the exact value expected by the service.

Before:

http-request replace-header Content-Type (.*) application/json; charset=utf-8



After:

http-request set-header Content-Type application/json if { hdr(Content-Type) -i -m found }



This ensures the header is exactly application/json and does not add parameters.


2. Fix the X-Request-ID concatenation


Use add-header instead of set-header to avoid overwriting an existing valid ID, and insert a separator if concatenation is required.


Before:

http-request set-header X-Request-ID %[req.hdr(X-Request-ID)]%[uuid()]



After:

http-request add-header X-Request-ID %[uuid()] unless { req.hdr(X-Request-ID) -m found }



3. Reload HAProxy safely


# systemctl reload haproxy
# haproxy -c -f /etc/haproxy/haproxy.cfg   # verify config syntax



Verify – Confirm the Issue Is Resolved


1. Re‑run the evaluation request through HAProxy


curl -X POST https://api.example.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "X-Request-ID: $(uuidgen)" \
  -d '{"model":"gpt-4","messages":[...],"functions":[...],"function_call":"auto"}'



Expected response:


{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "choices": [{ "message": { "role": "assistant", "content": "...", "function_call": {...} } }]
}



2. Check HAProxy captured header


# haproxyctl stats | grep Content-Type



Output should show application/json without any suffix.


3. Verify backend logs no longer contain parsing errors


# journalctl -u llm-inference.service -u llm-inference.service | grep "Failed to parse"



No matching lines indicates the payload was accepted.


Prevent – Guardrails for Future Deployments


    

  • Header whitelist: Define an explicit http-request deny if !{ hdr(Content-Type) -i application/json } rule to reject any request that deviates from the required header.
    • 

  • Schema validation gateway: Deploy a lightweight sidecar (e.g., Envoy with JSON schema filter) that validates the request body before it reaches the LLM service.
    • 

  • Automated HAProxy config linting: Integrate haproxy -c checks into CI pipelines and add a custom rule that flags any replace-header usage on Content-Type.
    • 

  • Observability: Enable http-request capture for critical headers and expose them as Prometheus metrics (e.g., haproxy_http_req_hdr_content_type) to alert on unexpected values.
    • 

  • Testing: Add an end‑to‑end test case in the evaluation pipeline that sends a minimal function‑call payload through HAProxy and asserts a successful 200 response.
    • 



Related Topic Hub: Distributed Systems Troubleshooting Hub


FAQ


    

  1. Why does the error disappear when I call the service directly?
    
Because the direct request does not undergo HAProxy’s header rewrite, so the Content-Type remains exactly application/json, which the LLM parser expects.
    2. 

  2. Can I keep the ; charset=utf-8 suffix and still have the API work?
    
No. The current inference service’s HTTP parser treats any parameter on Content-Type as a signal to use multipart parsing, which corrupts the JSON payload. The service must be updated to ignore parameters, or the proxy must strip them.
    3. 

  3. Is the X-Request-ID header required for function calling?
    
It is not required for the LLM API, but it is used by our observability pipeline. The header must be a single UUID value; concatenating multiple values leads to malformed request metadata.
    4. 

  4. How can I detect similar header‑related issues early?
    
Add a health‑check endpoint that echoes back received headers, and run a periodic curl test that asserts the exact header values. Alert on any deviation.
    5. 

  5. Would using HTTP/2 change this behavior?
    
The issue is independent of the HTTP version; HAProxy applies the same header rewrite logic for both HTTP/1.1 and HTTP/2. The fix must therefore target the rewrite rule itself.
    6.