tgi

TGI service unreachable during benchmark due to DNS resolution failure

by
white printer paper on white textile

Problem – TGI Service Unreachable During Benchmark During automated performance benchmarking of a Text Generation Inference (TGI) deployment, the benchmark script repeatedly fails with a DNS resolution error: 2026-10-18 14:32:07,123 ERROR benchmark.py: Failed to resolve host “tgi-model-prod.example.com”: Name or service not known Traceback (most recent call last): File “/opt/benchmark/benchmark.py”, line 87, in run ip = … Read more

TGI node OOM during large batch inference with autoscaling disabled

by
The image shows an open bible book of genesis

Problem – TGI Node Out‑of‑Memory (OOM) During Large Batch Inference with Autoscaling Disabled In a production Kubernetes deployment of Text Generation Inference (TGI), a batch job that aggregates thousands of text‑generation requests into a single inference call caused the pod to be killed with an OOM error. Autoscaling was deliberately disabled, so the node could … Read more

TGI container OOM crash loading 30B model on 16GiB pod

by
A book page with printed text and handwritten notes

Problem Description In a staging environment a tgi:latest container is deployed in a Kubernetes pod with 16 GiB of RAM. During startup the container crashes with an out‑of‑memory (OOM) termination: kubectl describe pod tgi-staging-abc123 … State: Waiting Reason: OOMKilled … Events: Type Reason Age From Message —- —— —- —- ——- Warning OOMKill 2m kubelet, node-01 … Read more

TGI checkpoint storage class not found after A/B deployment

by
A book page with printed text and blank lined paper

TGI checkpoint storage class not found after A/B deployment Problem During an A/B testing rollout of Text Generation Inference (TGI) instances, the newly created replica fails to start and logs an error similar to: Failed to load checkpoint: storage class “fast-ssd” not found Other observed symptoms include: Pod tgi-a-12345 stays in Pending state. kubectl describe … Read more

Kubernetes StatefulSet update fails during rolling upgrade TGI deployment

by
Open bible page with handwritten notes on the side.

Problem Description A high‑traffic Text Generation Inference (TGI) service is deployed as a StatefulSet on a Kubernetes cluster. During a rolling upgrade the controller stalls after the first pod is updated. Subsequent pods never reach the Running/Ready state, causing a partial service outage and a noticeable drop in request capacity. Typical symptoms observed in the … Read more