Container crash loop with segfault loading large ONNX model in sandbox

Problem

In a development sandbox the container that runs an inference service based on ONNX Runtime repeatedly enters a crash‑loop. The container exits with a SIGSEGV during model initialization:

2026-10-22 14:03:12.345 INFO  [onnxruntime] Loading model from /models/large_model.onnx
2026-10-22 14:03:12.347 FATAL [onnxruntime] Segmentation fault (core dumped)
2026-10-22 14:03:12.348 ERROR container runtime: container terminated with exit code 139

The model is > 2 GB on disk, uses dynamic input shapes, and the sandbox imposes a 4 GiB memory cgroup limit. The issue does not appear when a smaller, static‑shape model is used.

Root Cause

The crash is caused by an unbounded memory allocation performed by ONNX Runtime when it attempts to allocate tensor buffers for the dynamic shape dimensions. The runtime computes the maximum possible size based on the model’s dim_param values, which for this model are left unspecified (e.g., batch_size: -1, seq_len: -1). In the sandbox the process is confined to a 4 GiB cgroup; the allocation request exceeds that limit, the kernel delivers a SIGSEGV instead of a clean OOMKilled because the allocation occurs inside a native thread without a proper guard page.

Additional contributing factors:

• ONNX Runtime’s default memory arena disables aggressive memory trimming for dynamic shapes.
• The sandbox’s /dev/shm size is limited to 64 MiB, causing failures when the runtime tries to map large temporary buffers.
• Model is stored as a single monolithic .onnx file; the runtime loads the entire graph into memory before it can apply shape‑inference optimizations.

Debug

Step‑by‑step investigation that reproduced the failure and isolated the cause.

1. Inspect container logs for the exact point of failure.

docker logs inference-sandbox-1
# ... logs shown above ...

2. Enable core dump generation inside the container to capture the faulting address.

# In Dockerfile
RUN echo "/tmp/core.%e.%p" > /proc/sys/kernel/core_pattern
RUN ulimit -c unlimited

3. Run gdb on the core file to see the allocation stack.

gdb /usr/local/bin/onnxruntime_core /tmp/core.ort.12345
(gdb) bt
#0  malloc + 0x...
#1  Ort::Session::Init + 0x...
#2  ModelLoader::LoadModel + 0x...
#3  main + 0x...

4. Check cgroup memory limits that the sandbox enforces.

# From host
cat /sys/fs/cgroup/memory/docker/$(docker inspect -f '{{.Id}}' inference-sandbox-1)/memory.limit_in_bytes
4194304000   # 4 GiB

5. Enable ONNX Runtime diagnostics to log memory allocation attempts.

export ORT_LOG_SEVERITY=VERBOSE
export ORT_ENABLE_CPU_MEM_LIMIT=1
export ORT_CPU_MEM_LIMIT=3500   # MiB

Running the container with these variables produced the following log entry before the crash:

2026-10-22 14:03:12.340 VERBOSE [memory] Requested allocation of 7.8 GiB for tensor 'input_0' (shape: [?, ?, 1024])

Solution

The fix consists of two complementary actions: limit the runtime’s memory demand and adjust the sandbox resources.

1. Constrain dynamic shape dimensions in the model

Replace unconstrained dimensions with realistic upper bounds using onnx.shape_inference or by editing the model with onnx-simplifier.

# Before (model definition)
input_0: float32[?, ?, 1024]

# After (bounded)
input_0: float32[1..8, 1..256, 1024]

2. Enable ONNX Runtime memory‑limit mode

Set environment variables so the runtime aborts allocation before it triggers a segfault.

# Dockerfile snippet
ENV ORT_ENABLE_CPU_MEM_LIMIT=1
ENV ORT_CPU_MEM_LIMIT=3000   # MiB, below cgroup limit
ENV ORT_DISABLE_MEM_PATTERN=0

3. Increase sandbox /dev/shm and memory cgroup (development only)

If bounding the shape is not possible, raise the sandbox limits to accommodate the model.

# docker run
docker run -d \
  --shm-size=1g \
  --memory=8g \
  --memory-swap=8g \
  -e ORT_ENABLE_CPU_MEM_LIMIT=0 \
  my-onnx-inference:latest

4. Externalize large weight tensors

Convert the model to use external_data for weights > 2 GB, reducing the in‑memory footprint during graph loading.

python - <<'PY'
import onnx
model = onnx.load('large_model.onnx')
onnx.save_model(model, 'large_model_external.onnx', save_as_external_data=True, location='weights.bin')
PY

Before / After comparison of Dockerfile

FROM python:3.11-slim
COPY . /app
WORKDIR /app
RUN pip install onnxruntime
CMD ["python", "serve.py"]

FROM python:3.11-slim
COPY . /app
WORKDIR /app
RUN pip install onnxruntime
ENV ORT_ENABLE_CPU_MEM_LIMIT=1
ENV ORT_CPU_MEM_LIMIT=3000
ENV ORT_DISABLE_MEM_PATTERN=0
CMD ["python", "serve.py"]

Verify

After applying the changes, the container should start and remain healthy.

# Deploy
docker compose up -d inference
# Check status
docker ps -f name=inference
# Expected: Up (healthy)

# Log verification
docker logs inference | grep "Segmentation fault"
# No output

# Health endpoint
curl -s http://localhost:8080/healthz
# {"status":"ok"}

Additionally, confirm that the runtime reports the bounded allocation:

2026-10-22 14:05:01.112 VERBOSE [memory] Requested allocation of 1.2 GiB for tensor 'input_0' (shape: [8, 256, 1024])

Prevent

• Always bound dynamic dimensions in production models; leave -1 only for truly variable axes with known limits.
• Enable ORT_ENABLE_CPU_MEM_LIMIT in non‑production environments to catch runaway allocations early.
• Store models larger than 2 GB with external data files to avoid monolithic memory mapping.
• Configure sandbox /dev/shm to at least 256 MiB for ONNX Runtime, which uses shared memory for intermediate tensors.
• Integrate a CI step that runs onnxruntime_test with a memory‑stress flag to detect potential segfaults before deployment.

Related Topic Hub: Model Serving Troubleshooting Hub

FAQ

1. Why does the container crash with SIGSEGV instead of being OOM‑killed?
   ONNX Runtime performs a large malloc inside a native thread. When the allocation exceeds the cgroup limit, the kernel returns an invalid address, leading to a segmentation fault before the OOM killer can intervene.
2. Can I keep the model fully dynamic and still avoid crashes?
   Yes, by setting realistic upper bounds on dynamic dimensions (e.g., batch_size: 1..16) and enabling ORT_ENABLE_CPU_MEM_LIMIT so the runtime enforces those bounds at load time.
3. Is increasing the sandbox memory limit a safe long‑term solution?
   It mitigates the symptom but does not address the underlying unbounded allocation. In production you should bound shapes or externalize weights to keep memory usage predictable.
4. How do I know which tensor caused the oversized allocation?
   Set ORT_LOG_SEVERITY=VERBOSE. The runtime logs each allocation with the tensor name and requested size, as shown in the verification logs.
5. Does GPU execution avoid this issue?
   GPU memory allocation follows similar rules; an unbounded shape will still request the maximum possible GPU buffer and can cause a driver‑level crash. Bounding dimensions is required for both CPU and GPU execution.