routing

etcd configuration drift after adding AWS node breaks GPT-3.5 routing

by
Rows of black server racks with white logos in a data center

Problem Description After provisioning a new EC2 instance and adding it as an etcd member to a hybrid on‑premises/AWS cluster, the OpenAI GPT‑3.5 routing layer began serving stale model versions. Requests that should have been directed to gpt-3.5-turbo-0613 were occasionally routed to the older gpt-3.5-turbo-0301 endpoint, causing latency spikes and inconsistent billing. Observed symptoms Intermittent … Read more

Ingress routing misconfiguration in Docker GPU cluster

by
a golden docker logo on a black background

Problem Description In a Docker Swarm GPU cluster (A100/H100), services that request –gpus become unreachable through the Swarm ingress routing mesh. Clients receive HTTP 502/504 errors, and the Docker daemon logs report failures such as: Failed to create endpoint: network ingress: driver failed programming the network: MTU mismatch Ingress routing mesh: connection timed out (504) … Read more

ONNX Runtime inference fails after CI/CD deployment due to incorrect routing

by
img IX mining rig inside white and gray room

Problem – ONNX Runtime Inference Fails After CI/CD Deployment During a routine CI/CD rollout, newly built ONNX model containers are deployed to a Kubernetes cluster. After the deployment completes, API calls that should hit the onnx-runtime service return HTTP 502/504 errors and the ONNX Runtime logs show Failed to load model: file not found. The … Read more