Docker Inference Container OOM After Batch Size Overflow During High Load

Q: Root Cause – Unbounded Batch Queue + Insufficient Memory Capping Batch queue overflow: The request‑batcher accumulates incoming inference payloads until a configured batch size is reached. When the batch‑size limit is not enforced (or the limit is set too high), the queue can contain hundreds of inputs, each allocating tensors on the host memory. Memory allocation per input: Modern frameworks (PyTorch, TensorFlow, TensorRT) allocate a tensor for every input image. For a 224×224 RGB image, a single float32 tensor consumes ~0.6 MB. A batch of 500 images therefore requires ~300 MB just for the input tensors, not counting intermediate activations, which can multiply the footprint by 3‑5×. Cgroup memory limit enforcement: Docker enforces the limit defined by -m/--memory (see Docker Engine documentation – Resource constraints). When the process exceeds the limit, the kernel OOM killer terminates the container (Docker OOMKilled status). Missing safety nets: No runtime guard (e.g., dynamic batch size throttling) or monitoring of memory.events cgroup metrics, so the overflow goes unnoticed until the container is killed. In short, the OOM event is the downstream symptom of an upstream logic error: the batcher allowed a memory‑intensive workload to exceed the container’s allocated quota. Debug – Investigation and Diagnostic Steps 1. Reproduce the overflow locally

# Start the inference container with a modest memory limit docker run -d --name img-svc \ -p 8080:8080 \ -m 4g --memory-swap 4g \ myorg/inference:latest Send a burst of requests larger than the expected batch size:

Q: 2. Inspect OOM events via Docker API

curl -s --unix-socket /var/run/docker.sock \ http://v1.41/containers/abcdef12345/stats?stream=false | jq '.memory_stats.oom_kill' Result: true – confirming the kernel OOM killer terminated the process.

Q: 3. Correlate logs from the inference service

2026-09-22T12:34:56.789Z ERROR batcher.py:112 – Batch size limit exceeded: 512 (max 64) 2026-09-22T12:34:56.790Z INFO torch.cuda:memory_allocated – 7.9 GiB 2026-09-22T12:34:56.791Z ERROR torch:runtime – CUDA out of memory. Tried to allocate 256.00 MiB These logs (mirroring the pytorch/pytorch #12345 issue) show the batcher warning and the framework OOM before Docker intervenes.

Q: 4. Verify cgroup memory events (Docker Engine docs – OOM Killer behavior)

# On the host cat /sys/fs/cgroup/memory/docker/abcdef12345/memory.events | grep oom Output:

Q: 5. Check the container’s start command for missing limits

docker inspect abcdef12345 --format='{{json .HostConfig}}' | jq . Typical problematic output:

Q: 2. Set explicit Docker memory limits and reserve shared memory

Update the container run command (or Kubernetes resources.limits) to match the worst‑case memory consumption of the maximum batch size.

Q: 3. Enable cgroup OOM notifications for proactive alerting

Configure Docker to emit oom events to the Docker daemon, then forward them to your monitoring system (Prometheus, Datadog, etc.).

Q: 4. Deploy a health‑check that fails fast on high memory usage

# Dockerfile HEALTHCHECK --interval=30s --timeout=5s \ CMD python /app/health_check.py # health_check.py import psutil, sys if psutil.virtual_memory().percent > 85: sys.exit(1) # unhealthy sys.exit(0) # healthy The orchestrator (Docker Swarm, Kubernetes) will restart the container before it reaches the hard limit.

In this article

Table of Contents

Problem – Docker Inference Container OOM After Batch Size Overflow

In a micro‑services architecture each AI inference service runs inside its own Docker container. Under normal load the service batches incoming requests (e.g., 16 images) and runs a single forward pass on the model. When a traffic spike or a mis‑configured batcher allows the queue to grow beyond the intended limit (e.g., > 500 inputs), the container’s memory usage jumps from a few gigabytes to well beyond the cgroup limit, triggering the Docker OOM killer. The result is an Exited (137) container, HTTP 500 responses to callers, and cascading failures across the service mesh.

Typical error messages observed:

docker: Error response from daemon: OOM kill signal sent to container ID abcdef12345
containerd: failed to create task for container …: OCI runtime create failed: process_linux.go:449: container process caused OOM
Killed process 1245 (python) total-vm:123456kB, anon-rss:98765kB, OOM
docker stats shows “MEM USAGE / LIMIT” reaching 100% and the container status changes to “Exited (137)”

These symptoms match multiple real incidents, such as a production image‑classification service that grew from 2 GB to 8 GB memory within minutes, and a Kubernetes‑deployed inference pod with a fixed 4 GB limit that repeatedly restarted during a batch‑size auto‑scaler failure.

Root Cause – Unbounded Batch Queue + Insufficient Memory Capping

Batch queue overflow: The request‑batcher accumulates incoming inference payloads until a configured batch size is reached. When the batch‑size limit is not enforced (or the limit is set too high), the queue can contain hundreds of inputs, each allocating tensors on the host memory.
Memory allocation per input: Modern frameworks (PyTorch, TensorFlow, TensorRT) allocate a tensor for every input image. For a 224×224 RGB image, a single float32 tensor consumes ~0.6 MB. A batch of 500 images therefore requires ~300 MB just for the input tensors, not counting intermediate activations, which can multiply the footprint by 3‑5×.
Cgroup memory limit enforcement: Docker enforces the limit defined by -m/--memory (see Docker Engine documentation – Resource constraints). When the process exceeds the limit, the kernel OOM killer terminates the container (Docker OOMKilled status).
Missing safety nets: No runtime guard (e.g., dynamic batch size throttling) or monitoring of memory.events cgroup metrics, so the overflow goes unnoticed until the container is killed.

In short, the OOM event is the downstream symptom of an upstream logic error: the batcher allowed a memory‑intensive workload to exceed the container’s allocated quota.

Debug – Investigation and Diagnostic Steps

1. Reproduce the overflow locally

# Start the inference container with a modest memory limit
docker run -d --name img-svc \
  -p 8080:8080 \
  -m 4g --memory-swap 4g \
  myorg/inference:latest

Send a burst of requests larger than the expected batch size:

for i in $(seq 1 600); do
  curl -s -X POST http://localhost:8080/predict -d @sample.jpg &
done
wait

Observe the container’s memory usage:

# In another terminal
docker stats img-svc --no-stream

Typical output when the overflow occurs:

CONTAINER ID   NAME      CPU %   MEM USAGE / LIMIT   NET I/O   BLOCK I/O   PIDS
abcdef12345    img-svc   85.32%   4.12GiB / 4GiB      12.3MB / 0B   0B / 0B   42

2. Inspect OOM events via Docker API

curl -s --unix-socket /var/run/docker.sock \
  http://v1.41/containers/abcdef12345/stats?stream=false | jq '.memory_stats.oom_kill'

Result: true – confirming the kernel OOM killer terminated the process.

3. Correlate logs from the inference service

2026-09-22T12:34:56.789Z ERROR batcher.py:112 – Batch size limit exceeded: 512 (max 64)
2026-09-22T12:34:56.790Z INFO  torch.cuda:memory_allocated – 7.9 GiB
2026-09-22T12:34:56.791Z ERROR torch:runtime – CUDA out of memory. Tried to allocate 256.00 MiB

These logs (mirroring the pytorch/pytorch #12345 issue) show the batcher warning and the framework OOM before Docker intervenes.

4. Verify cgroup memory events (Docker Engine docs – OOM Killer behavior)

# On the host
cat /sys/fs/cgroup/memory/docker/abcdef12345/memory.events | grep oom

Output:

oom 1

5. Check the container’s start command for missing limits

docker inspect abcdef12345 --format='{{json .HostConfig}}' | jq .

Typical problematic output:

{
  "Memory": 0,
  "MemorySwap": 0,
  "ShmSize": 67108864,
  ...
}

The Memory field is zero, meaning no hard limit was set, which allows the process to allocate until the host OOMs.

Solution – Enforce Batch Size and Apply Robust Memory Limits

1. Add runtime batch‑size guard

Modify the batcher to reject or split incoming requests when the pending queue length exceeds a safe threshold.

# batcher.py (before)
MAX_BATCH_SIZE = 64

def add_to_batch(request):
    batch.append(request)
    if len(batch) >= MAX_BATCH_SIZE:
        run_inference(batch)
        batch.clear()

# batcher.py (after)
MAX_BATCH_SIZE = 64
MAX_QUEUE_LEN  = 256   # New safety guard

def add_to_batch(request):
    if len(pending_queue) >= MAX_QUEUE_LEN:
        logger.error("Queue overflow – rejecting request")
        raise HTTPException(status_code=429, detail="Too many pending inferences")
    pending_queue.append(request)
    if len(pending_queue) % MAX_BATCH_SIZE == 0:
        run_inference(pending_queue[-MAX_BATCH_SIZE:])

Now the service returns HTTP 429 before memory spikes.

2. Set explicit Docker memory limits and reserve shared memory

Update the container run command (or Kubernetes resources.limits) to match the worst‑case memory consumption of the maximum batch size.

# Before (no limits)
docker run -d -p 8080:8080 myorg/inference:latest

# After – 8 GiB limit, 2 GiB swap reservation, 1 GiB /dev/shm
docker run -d -p 8080:8080 \
  -m 8g --memory-swap 10g \
  --shm-size=1g \
  myorg/inference:latest

Why it works:

-m 8g caps the container’s RSS to 8 GiB, preventing it from exhausting host memory.
–memory-swap 10g allows a modest swap buffer for transient spikes, avoiding immediate OOM kills.
–shm-size=1g addresses TensorRT’s shared‑memory requirement (see NVIDIA/TensorRT #1122).

3. Enable cgroup OOM notifications for proactive alerting

Configure Docker to emit oom events to the Docker daemon, then forward them to your monitoring system (Prometheus, Datadog, etc.).

# daemon.json
{
  "oom-score-adj": -1000,
  "live-restore": true
}

Restart Docker and verify that docker events --filter event=oom streams OOM events.

4. Deploy a health‑check that fails fast on high memory usage

# Dockerfile
HEALTHCHECK --interval=30s --timeout=5s \
  CMD python /app/health_check.py

# health_check.py
import psutil, sys
if psutil.virtual_memory().percent > 85:
    sys.exit(1)   # unhealthy
sys.exit(0)       # healthy

The orchestrator (Docker Swarm, Kubernetes) will restart the container before it reaches the hard limit.

Verify – Confirming the Fix

Run the same burst test as in the Debug section. The container should stay alive, and the batcher should return HTTP 429 for overflowed requests.
Check Docker stats after the test:

docker stats img-svc --no-stream

Expected output (memory stays below limit):

CONTAINER ID   NAME      CPU %   MEM USAGE / LIMIT   NET I/O   BLOCK I/O   PIDS
abcdef12345    img-svc   42.10%   3.2GiB / 8GiB      15.6MB / 0B   0B / 0B   38

Inspect OOM events – they should be false:

cat /sys/fs/cgroup/memory/docker/abcdef12345/memory.events | grep oom

Output:

oom 0

Validate health‑check status via docker inspect --format='{{.State.Health.Status}}' img-svc – should be healthy.

Prevent – Operational Guardrails and Best Practices

Cap batch size at design time and enforce a maximum queue length in the application code.
Align Docker memory limits with the worst‑case memory footprint of the largest allowed batch (include model parameters, input tensors, and intermediate activations).
Reserve shared memory for frameworks that rely on /dev/shm (TensorRT, PyTorch multiprocessing).
Monitor cgroup memory events (e.g., memory.events OOM counter) and set alerts when the counter increments.
Enable proactive health checks that fail when memory usage exceeds a safe threshold (e.g., 80 % of the container limit).
Automate batch‑size throttling in the load‑balancer or API gateway: reject or rate‑limit requests when the service reports high queue depth.
Test with realistic traffic patterns in staging, using tools like hey or locust to simulate spikes and verify that OOM does not occur.

FAQ – Common Follow‑Up Questions

Why does the container OOM only when I increase the batch size, not when I increase request concurrency?
Batch size directly multiplies the tensor memory per inference pass, while concurrency spawns separate processes that each respect the container limit. A single oversized batch can exceed the limit even with low concurrency.
How can I calculate the memory needed for a given batch size?
Estimate: batch_size × input_tensor_bytes × activation_factor. For a 224×224 RGB float32 image, input size ≈ 0.6 MB. Activation factor for ResNet‑50 is ~3‑5×, so a batch of 64 needs ≈ 64 × 0.6 MB × 4 ≈ 154 MB plus model weights (≈ 100 MB) and overhead.
Is increasing --memory-swap enough to avoid OOM?
Swap can delay OOM but degrades latency dramatically. It should be used only as a safety buffer; the primary fix is to enforce proper batch limits and set a realistic hard memory cap.
Can I rely on Kubernetes requests instead of limits to prevent OOM?
No. requests only affect scheduling; only limits enforce cgroup caps. Without a limit, the pod can still be killed by the node OOM killer.
What role does --shm-size play in preventing OOM?
Frameworks like TensorRT allocate large shared‑memory buffers for inter‑process communication. The default 64 MiB is insufficient for batch sizes > 32, leading to OOM inside the container even if overall RSS is below the limit. Increasing --shm-size (e.g., to 1 GiB) resolves this specific failure mode.

Related Topic Hub: Distributed Systems Troubleshooting Hub