ONNX Runtime Connection Timeout In Docker Container

Q: Problem Description

When running inference inside an ONNX Runtime Docker container the following error is observed:

Q: 2. Provide explicit DNS servers or enable Docker’s built‑in DNS

Update the daemon configuration (/etc/docker/daemon.json) to include reliable resolvers:

In this article

Table of Contents

Problem Description

When running inference inside an ONNX Runtime Docker container the following error is observed:

ONNXRuntimeException: Unable to download model from URL https://example.com/model.onnx – Connection timed out

or, when using the Remote Execution Provider:

grpc call failed: deadline exceeded – RemoteExecutionProvider initialization failed

Typical impact includes:

Model loading hangs for the configured timeout (default 30 seconds).
Inference requests return 5xx errors or are blocked indefinitely.
Container logs are filled with repeated timeout messages, consuming disk space.

Root Cause Analysis

The timeout is not caused by ONNX Runtime itself but by the container’s inability to reach the remote endpoint. The most common underlying reasons, corroborated by the official ONNX Runtime Docker guide and community incidents, are:

Root Cause	Why it leads to a timeout
Docker default `bridge` network with no outbound routing	Containers use an isolated subnet; if the host firewall (iptables) blocks masquerading, SYN packets never leave the host.
Missing DNS resolution inside the container	ONNX Runtime resolves the model URL before the HTTP request; `getaddrinfo ENOTFOUND` is translated into a generic timeout.
Absent or mis‑scoped cloud credentials (e.g., AWS S3)	Remote fetches fail during the authentication phase; the SDK retries until the request timeout expires (see GitHub issue #14789).
Network policies or security groups denying egress	Kubernetes NetworkPolicy or VPC ACLs drop packets, producing the same “deadline exceeded” error reported in #15234.
Incorrect proxy or MTU settings	Large HTTP payloads are fragmented; mismatched MTU on the host‑to‑container path causes silent packet loss, observed in the Azure VM case on Stack Overflow.

In short, ONNX Runtime times out because the underlying TCP connection never completes, not because of an internal library bug.

Investigation and Debugging

Follow these reproducible steps inside the failing container:

Confirm the container can reach the host network.
```
docker exec -it onnx_container curl -v https://example.com/model.onnx --max-time 5
```
Expected output: HTTP 200 and model binary. If you see Connection timed out, the problem is network‑level.

Check DNS resolution.

docker exec -it onnx_container getent hosts example.com

Failure example:

getent: Name or service not known

Inspect the container’s network mode.
```
docker inspect onnx_container --format '{{ .HostConfig.NetworkMode }}'
```
If it returns bridge, verify host iptables -t nat -L POSTROUTING includes MASQUERADE rules.
Validate cloud credentials (if using S3, GCS, etc.).
```
docker exec -it onnx_container env | grep AWS
```
Missing AWS_ACCESS_KEY_ID or AWS_SECRET_ACCESS_KEY explains the failure reported in issue #14789.

Review ONNX Runtime logs for the exact error path.

journalctl -u onnxruntime.service -n 50

Typical snippet:

2026-06-26 12:04:13.021 [E] ONNXRuntimeException: Unable to download model from URL https://example.com/model.onnx – Connection timed out

Check host‑level firewall or security groups.
```
sudo iptables -L -v -n | grep DROP
```
Look for rules that drop outbound traffic on ports 80/443.

Resolution

Apply the fix that matches the identified root cause. Below are three common scenarios with before/after command comparisons.

1. Use host networking for unrestricted outbound access

Before (default bridge):

docker run -d --name onnx_container onnxruntime:latest

After (host network):

docker run -d --name onnx_container --network=host onnxruntime:latest

Host mode bypasses the bridge NAT and uses the host’s network stack, eliminating firewall/NAT misconfigurations. This mirrors the community workaround in GitHub issue #15234.

2. Provide explicit DNS servers or enable Docker’s built‑in DNS

Update the daemon configuration (/etc/docker/daemon.json) to include reliable resolvers:

{
  "dns": ["8.8.8.8", "1.1.1.1"]
}

Then restart Docker and recreate the container:

sudo systemctl restart docker
docker rm -f onnx_container
docker run -d --name onnx_container onnxruntime:latest

This resolves the ENOTFOUND symptom observed in the “failed to resolve host name” errors.

3. Mount cloud credentials and set timeout overrides

For S3 model stores, mount the AWS credentials file and increase the ONNX Runtime remote timeout (default 30 s) to accommodate slower networks:

docker run -d \
  --name onnx_container \
  -v $HOME/.aws:/root/.aws:ro \
  -e ONNXRUNTIME_REMOTE_TIMEOUT=60000 \
  onnxruntime:latest

Increasing ONNXRUNTIME_REMOTE_TIMEOUT (documented in the Remote Execution Provider docs) prevents premature deadline errors while the underlying network issue is being resolved.

4. Adjust host firewall / security groups

On the host, allow outbound HTTPS traffic for the Docker bridge interface (docker0):

sudo iptables -I FORWARD -i docker0 -p tcp --dport 443 -j ACCEPT
sudo iptables -I FORWARD -i docker0 -p tcp --dport 80 -j ACCEPT

For cloud VPCs, ensure the security group attached to the VM permits egress to the model registry’s IP range.

Validation

After applying the fix, verify connectivity and successful model load:

Run a quick curl from inside the container:

docker exec -it onnx_container curl -I https://example.com/model.onnx

Expected header response (e.g., HTTP/1.1 200 OK).

Execute a minimal ONNX Runtime inference script:

docker exec -it onnx_container python - <<'PY'
import onnxruntime as ort
sess = ort.InferenceSession("https://example.com/model.onnx")
print("Model loaded, inputs:", sess.get_inputs())
PY

Successful output confirms the runtime can fetch and parse the model.

Check logs for the absence of timeout messages:

docker logs onnx_container | grep -i timeout || echo "No timeout logs"

Prevention and Best Practices

Network policy as code: Define explicit egress rules in Kubernetes NetworkPolicy or Docker Compose files to guarantee outbound access.
Health‑check endpoint: Add a lightweight HTTP GET to the model URL in a startup script; abort container launch if the check fails.
Credential management: Use Docker secrets or Kubernetes secrets to inject cloud credentials; avoid hard‑coding in images.
Monitoring: Alert on ONNXRuntimeException: Connection timed out occurrences and on outbound traffic spikes from the container’s IP.
Timeout tuning: Align ONNXRUNTIME_REMOTE_TIMEOUT with expected network latency; keep it lower than the orchestrator’s pod termination grace period.
Documented Docker network mode: Record the chosen network mode (bridge, host, custom) in the deployment README to avoid regressions.

Related Topic Hub: Model Serving Troubleshooting Hub

FAQ

Why does the model load succeed locally but time out in Docker?
Because the host’s network stack may have outbound access, whereas the container’s isolated bridge network is blocked by firewall or lacks DNS resolution.
Can I keep using the bridge network and still reach external services?
Yes. Ensure the host masquerade rule is present, add DNS servers to /etc/docker/daemon.json, and open outbound ports in the host firewall.
Is increasing ONNXRUNTIME_REMOTE_TIMEOUT a proper fix?
It only masks the symptom. Use it temporarily while fixing the underlying connectivity issue (e.g., firewall, DNS).
How do I debug a “grpc call failed: deadline exceeded” error?
Run grpcurl -v -proto your_service.proto yourhost:port ListModels from inside the container to verify the gRPC endpoint is reachable; check that the container can resolve the host name and that the port is not blocked.
What role does MTU play in these timeouts?
If the container’s MTU (often 1500) differs from the host’s underlying network (e.g., 1400 on a VPN), fragmented packets can be dropped, leading to silent timeouts. Align MTU settings on the Docker daemon (--mtu) with the host network.