Mistral AI Webhook Timeout After 30 Seconds Pending Async Job

Q: Increase connection‑pool size to avoid queuing delays

For NGINX, raise worker_connections and keepalive_requests. For Envoy, adjust max_requests_per_connection and circuit_breakers thresholds.

Q: Implement idempotent acknowledgment handling

Ensure the webhook endpoint returns 200 OK as soon as the request is received, before any downstream processing. Queue the payload internally for asynchronous handling.

In this article

Table of Contents

Problem

When using Mistral AI’s asynchronous completion endpoints, webhook callbacks that deliver the final inference result are intermittently failing. The failure manifests as a 30‑second timeout, after which the payload is dropped and the client receives no status update. The issue becomes pronounced under high traffic (≈500 RPS) where the API gateway’s rate limiting and connection‑pool settings interact with Mistral’s retry policy.

Typical symptoms observed in production logs include:

“Webhook delivery failed: timeout after 30 seconds” logged by the mistral-client library.
HTTP 504 responses from the webhook endpoint.
“Connection reset by peer after 30s” in the gateway’s nginx error log.

Mistral response payload:

{
  "error": "WEBHOOK_TIMEOUT",
  "message": "Callback not acknowledged within 30s"
}

Root Cause

Interaction of Mistral’s 30 s delivery window with upstream timeout settings

Mistral AI defines a strict webhook retry policy:

First attempt must be acknowledged within 30 seconds.
If the callback returns a non‑2xx status or times out, Mistral retries up to three times, each still bound by the original 30 s deadline.

If any component in the delivery path (NGINX, Envoy, AWS ALB, Kubernetes Ingress) imposes a request timeout ≤ 30 s, the connection is closed before Mistral can receive an acknowledgment, causing the webhook to be marked as failed.

High‑traffic environments exacerbate the problem because:

Connection pools become exhausted, forcing new requests to wait in the accept queue.
Rate‑limiting throttles inbound webhook traffic, extending the time the request spends in the proxy before reaching the application.
Idle‑timeout defaults (e.g., proxy_read_timeout 30s in NGINX, idle_timeout 30s in AWS ALB) cut the TCP connection exactly at the moment Mistral expects the acknowledgment.

Thus, the root cause is a mismatch between Mistral’s 30 s acknowledgment requirement and the upstream proxy’s timeout/connection‑pool configuration, especially under load.

Debug

Step‑by‑step investigation

Confirm the error originates from the webhook delivery path.
```
journalctl -u nginx -f | grep "30s"
```
Expected snippet:

2026-08-09T12:34:56.789Z error: *30 “client timed out (30: Operation timed out)” while reading response header from upstream

Check Mistral’s retry logs. The client library writes a retry‑exhausted entry:

2026-08-09 12:35:01,234 - mistral_client.webhook - INFO - Retry-exhausted: Exceeded maximum webhook retries (3) - job_id=abc123 - deadline passed

Capture a packet trace during a failing callback.
```
tcpdump -i eth0 -w webhook.pcap port 443 and host webhook.myservice.com
```
Inspect the capture for a TCP FIN or RST occurring at ~30 s after the SYN.
Validate upstream timeout settings.
- NGINX: grep -R "proxy_read_timeout" /etc/nginx
- Envoy: curl http://localhost:9901/config_dump | jq '.configs.listener_config[0].filter_chains[0].filters[] | select(.name=="envoy.filters.network.http_connection_manager") | .typed_config.common_http_protocol_options.idle_timeout'
- AWS ALB: Verify via console or CLI:
```
aws elbv2 describe-load-balancers --names my-alb --query "LoadBalancers[].IdleTimeout"
```
Simulate load to reproduce the timeout. Use hey or wrk to generate 600 RPS against the webhook endpoint while triggering an async job:
```
hey -c 200 -n 12000 -m POST -H "Content-Type: application/json" \
    -d '{"prompt":"large model request"}' https://api.mistral.ai/v1/async/completions
```
Monitor the gateway’s connection‑pool metrics (nginx_status or Envoy’s cluster_manager stats) for “connection_pool_overflow”.

Solution

Align upstream timeouts with Mistral’s 30 s delivery window

Increase the request/idle timeout on every hop that sits between Mistral and the final webhook handler. The recommended values are:

NGINX proxy_read_timeout ≥ 45 s
Envoy idle_timeout ≥ 45 s
AWS ALB IdleTimeout ≥ 45 s (default 60 s is acceptable)
Kubernetes Ingress proxy-connect-timeout and proxy-read-timeout ≥ 45 s

Before – NGINX default

# /etc/nginx/conf.d/webhook.conf
location /webhook {
    proxy_pass http://backend:8080;
    proxy_read_timeout 30s;   # default
}

After – extended timeout

# /etc/nginx/conf.d/webhook.conf
location /webhook {
    proxy_pass http://backend:8080;
    proxy_read_timeout 45s;
    proxy_connect_timeout 10s;
    proxy_send_timeout 45s;
}

Envoy listener configuration snippet (before)

{
  "name": "listener_0",
  "filter_chains": [
    {
      "filters": [
        {
          "name": "envoy.filters.network.http_connection_manager",
          "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
            "common_http_protocol_options": {}
          }
        }
      ]
    }
  ]
}

After – set idle timeout

{
  "name": "listener_0",
  "filter_chains": [
    {
      "filters": [
        {
          "name": "envoy.filters.network.http_connection_manager",
          "typed_config": {
            "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager",
            "common_http_protocol_options": {
              "idle_timeout": "45s"
            }
          }
        }
      ]
    }
  ]
}

Increase connection‑pool size to avoid queuing delays

For NGINX, raise worker_connections and keepalive_requests. For Envoy, adjust max_requests_per_connection and circuit_breakers thresholds.

# NGINX
worker_connections 8192;
keepalive_requests 10000;

# Envoy
"circuit_breakers": {
  "thresholds": [
    {
      "max_connections": 100000,
      "max_pending_requests": 50000,
      "max_requests": 1000000
    }
  ]
}

Implement idempotent acknowledgment handling

Ensure the webhook endpoint returns 200 OK as soon as the request is received, before any downstream processing. Queue the payload internally for asynchronous handling.

# Flask example
from flask import Flask, request, jsonify
app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def webhook():
    # Immediately acknowledge
    payload = request.get_json()
    enqueue_for_processing(payload)   # async worker
    return jsonify({"status":"accepted"}), 200

Verify

Reload the proxy configuration and confirm the new timeouts are active.

nginx -t && nginx -s reload
curl -I http://gateway/health   # should return 200

Trigger a long‑running async job (e.g., a 35 s inference) and watch the webhook logs.

2026-08-09 12:45:12,001 - webhook - INFO - Received callback for job_id=def456
2026-08-09 12:45:12,003 - webhook - INFO - Payload enqueued for async processing

Check Mistral’s client logs for successful acknowledgment.

2026-08-09 12:45:12,150 - mistral_client.webhook - INFO - Webhook delivered successfully - job_id=def456

Validate that no “WEBHOOK_TIMEOUT” errors appear in the past 24 h metrics.
```
grep WEBHOOK_TIMEOUT /var/log/mistral-client/*.log | wc -l   # should be 0
```

Prevent

Monitoring: Export webhook_delivery_time_seconds and set an alert if the 90th percentile exceeds 25 s.
Alerting: Trigger on Mistral client log pattern “WEBHOOK_TIMEOUT”.
Configuration guardrails: Enforce a CI lint rule that checks all ingress/egress proxies for proxy_read_timeout ≥ 45 s when the service subscribes to Mistral async callbacks.
Capacity planning: Keep worker_connections and Envoy pool sizes sized for peak RPS + 10 % headroom.
Idempotent design: Decouple payload ingestion from processing to guarantee a sub‑second HTTP response.

Related Topic Hub: LLM Systems Troubleshooting Hub

FAQ

Why does the webhook succeed for short jobs but fail for jobs >30 s?
Mistral only requires the HTTP response within 30 seconds. Short jobs complete before the upstream timeout fires, while longer jobs cause the proxy to close the connection at its own 30 s limit, leading to a timeout.
Can I increase Mistral’s timeout window?
No. The 30‑second acknowledgment window is fixed in the API contract (Webhook Integration Guide). Adjusting downstream timeouts is the only viable solution.
Do retries help if the first attempt times out?
Retries are still bound by the original 30‑second deadline. If the first attempt never reaches Mistral’s server because the connection is closed early, subsequent retries are discarded as “deadline passed”.
What metric should I watch to detect upcoming timeouts?
Track proxy_read_timeout_exceeded_total (NGINX) or http.downstream_rq_timeout (Envoy). A sudden rise correlates with webhook failures.
Is it safe to set the timeout to 60 seconds?
Yes, as long as it exceeds 30 seconds. However, setting it too high can mask other latency problems. A 45‑second buffer is generally sufficient and aligns with the guidance in the Rate Limiting and Quotas document.