PostgreSQL Pod Fails To Start After ConfigMap Mount Error In Dev Sandbox

Q: Debug – Step‑by‑step investigation 1. Verify volume definitions

$ kubectl get deployment postgres -n dev-sandbox -o yaml | grep -A5 volume volumes: - name: pg-config configMap: name: postgres-config - name: pg-secret secret: secretName: postgres-secret 2. Verify volume mounts in the container spec $ kubectl get pod postgres-7f9c9d8b5b-kt9xv -n dev-sandbox -o yaml | grep -A6 volumeMounts volumeMounts: - name: postgres-config #

Q: 3. Inspect the init container behavior

$ kubectl get pod postgres-7f9c9d8b5b-kt9xv -n dev-sandbox -o yaml | grep -A8 initContainers initContainers: - name: copy-default-conf image: busybox command: ["/bin/sh","-c","cp /default/postgresql.conf /etc/postgresql/conf.d/"] volumeMounts: - name: pg-config mountPath: /etc/postgresql/conf.d The init container mounts pg-config (the ConfigMap) at the same path that the Secret later mounts read‑only, causing the Secret to be masked.

Q: 4. Check actual filesystem inside the pod

$ kubectl exec -it postgres-7f9c9d8b5b-kt9xv -n dev-sandbox -- ls -l /etc/postgresql/conf.d total 0 The directory is empty, confirming that neither ConfigMap nor Secret data are present.

Q: Solution – Align volume names and separate mount points 1. Correct the volume name in the container spec

Update the deployment manifest so the volumeMount references the exact name declared in volumes.

Q: 2. Decouple init container from the Secret mount

Mount the ConfigMap at a dedicated path (/etc/postgresql/conf.d) for the init container, and mount the Secret at a separate path (/etc/postgresql/secret) for the main container. Ensure the init container does not overwrite the Secret.

Q: 3. Apply the corrected manifest

kubectl apply -f postgres-deployment.yaml -n dev-sandbox Verify – Confirm the pod starts and loads configuration After applying the changes, watch the pod lifecycle:

In this article

Table of Contents

Problem – PostgreSQL pod fails to start in the development sandbox

In a Kubernetes‑based development sandbox the PostgreSQL deployment never reaches the Running state. The pod repeatedly crashes with the following pattern:


$ kubectl get pod -n dev-sandbox -l app=postgresql
NAME                         READY   STATUS    RESTARTS   AGE
postgresql-7f9c9d8b5b-kt9xv 0/1     CrashLoopBackOff   5          3m12s

Inspecting the container logs shows that PostgreSQL cannot find its configuration files and authentication credentials:


2026-10-16 12:34:56.789 UTC [1] LOG:  could not open configuration file "/etc/postgresql/conf.d/postgresql.conf": No such file or directory
2026-10-16 12:34:56.789 UTC [1] LOG:  could not open password file "/etc/postgresql/secret/pgpass": No such file or directory
2026-10-16 12:34:56.789 UTC [1] FATAL:  could not access configuration file

Both the ConfigMap that provides postgresql.conf and the Secret that provides pgpass are declared in the manifest, but the pod cannot mount them.

Root Cause – Misaligned volume definitions and mount paths

The failure originates from two independent mistakes:

ConfigMap volume name mismatch: The volume is declared as pg-config but the volumeMount references postgres-config. Kubernetes therefore creates an empty directory at the mount point, hiding the ConfigMap data.
Secret mount path conflict with init container: An init container copies a default postgresql.conf into /etc/postgresql/conf.d before the main container starts. Because the Secret is mounted read‑only at the same path, the init container overwrites the Secret volume, leaving the Secret empty.

Both issues cause the main PostgreSQL process to see a missing configuration file and missing password file, leading to the crash loop.

Debug – Step‑by‑step investigation

1. Verify volume definitions


$ kubectl get deployment postgres -n dev-sandbox -o yaml | grep -A5 volume
      volumes:
      - name: pg-config
        configMap:
          name: postgres-config
      - name: pg-secret
        secret:
          secretName: postgres-secret

2. Verify volume mounts in the container spec


$ kubectl get pod postgres-7f9c9d8b5b-kt9xv -n dev-sandbox -o yaml | grep -A6 volumeMounts
        volumeMounts:
        - name: postgres-config   # <-- mismatch, should be pg-config
          mountPath: /etc/postgresql/conf.d
        - name: pg-secret
          mountPath: /etc/postgresql/secret
          readOnly: true

The mismatch is evident: name: postgres-config does not exist in the volumes list.

3. Inspect the init container behavior


$ kubectl get pod postgres-7f9c9d8b5b-kt9xv -n dev-sandbox -o yaml | grep -A8 initContainers
      initContainers:
      - name: copy-default-conf
        image: busybox
        command: ["/bin/sh","-c","cp /default/postgresql.conf /etc/postgresql/conf.d/"]
        volumeMounts:
        - name: pg-config
          mountPath: /etc/postgresql/conf.d

The init container mounts pg-config (the ConfigMap) at the same path that the Secret later mounts read‑only, causing the Secret to be masked.

4. Check actual filesystem inside the pod


$ kubectl exec -it postgres-7f9c9d8b5b-kt9xv -n dev-sandbox -- ls -l /etc/postgresql/conf.d
total 0

The directory is empty, confirming that neither ConfigMap nor Secret data are present.

Solution – Align volume names and separate mount points

1. Correct the volume name in the container spec

Update the deployment manifest so the volumeMount references the exact name declared in volumes.


# Before
volumeMounts:
- name: postgres-config
  mountPath: /etc/postgresql/conf.d

# After
volumeMounts:
- name: pg-config
  mountPath: /etc/postgresql/conf.d

2. Decouple init container from the Secret mount

Mount the ConfigMap at a dedicated path (/etc/postgresql/conf.d) for the init container, and mount the Secret at a separate path (/etc/postgresql/secret) for the main container. Ensure the init container does not overwrite the Secret.


# Updated deployment snippet
spec:
  volumes:
  - name: pg-config
    configMap:
      name: postgres-config
  - name: pg-secret
    secret:
      secretName: postgres-secret

  initContainers:
  - name: copy-default-conf
    image: busybox
    command: ["/bin/sh","-c","cp /default/postgresql.conf /etc/postgresql/conf.d/"]
    volumeMounts:
    - name: pg-config
      mountPath: /etc/postgresql/conf.d

  containers:
  - name: postgres
    image: postgres:15-alpine
    envFrom:
    - secretRef:
        name: postgres-secret
    volumeMounts:
    - name: pg-config
      mountPath: /etc/postgresql/conf.d
      readOnly: true
    - name: pg-secret
      mountPath: /etc/postgresql/secret
      readOnly: true

3. Apply the corrected manifest


kubectl apply -f postgres-deployment.yaml -n dev-sandbox

Verify – Confirm the pod starts and loads configuration

After applying the changes, watch the pod lifecycle:


$ kubectl rollout status deployment/postgres -n dev-sandbox
deployment "postgres" successfully rolled out

Check that the pod reaches Running and that PostgreSQL reports successful configuration loading:


$ kubectl logs -f $(kubectl get pod -l app=postgresql -n dev-sandbox -o name) -n dev-sandbox
2026-10-16 12:38:02.123 UTC [1] LOG:  starting PostgreSQL 15.3 (Debian 15.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2026-10-16 12:38:02.124 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2026-10-16 12:38:02.124 UTC [1] LOG:  listening on IPv6 address "::", port 5432
2026-10-16 12:38:02.125 UTC [1] LOG:  database system was shut down at 2026-10-16 12:35:45 UTC
2026-10-16 12:38:02.126 UTC [1] LOG:  database system is ready to accept connections

Validate that the configuration files are present inside the container:


$ kubectl exec -it $(kubectl get pod -l app=postgresql -n dev-sandbox -o name) -n dev-sandbox -- ls -l /etc/postgresql/conf.d
total 4
-rw-r--r-- 1 postgres postgres 1234 Oct 16 12:38 postgresql.conf
$ kubectl exec -it $(kubectl get pod -l app=postgresql -n dev-sandbox -o name) -n dev-sandbox -- cat /etc/postgresql/secret/pgpass
myuser:mydb:5432:myuser:mypassword

Prevent – Guardrails to avoid similar mount failures

Schema validation: Use kubectl apply --dry-run=server together with OpenAPI validation to catch mismatched volume names before deployment.
Automated tests: Include an integration test that starts a pod with the same manifest and asserts that required files exist at the expected mount points.
Separate init container mounts: Reserve a distinct temporary directory (e.g., /tmp/pg-init) for init‑container work, then copy files into the final mount point after the Secret is mounted.
Explicit readOnly flag: Mark ConfigMap mounts as readOnly: true to prevent accidental overwrites by init containers.
Monitoring alerts: Alert on PostgreSQL container exit codes 1 or log patterns like “could not open configuration file”.

Related Topic Hub: Data Infrastructure Troubleshooting Hub

FAQ

Why does the pod start correctly in staging but not in the dev sandbox?
The staging manifest uses the correct volume name (pg-config) while the dev manifest was copied manually and introduced a typo. Environments that share the same manifest avoid this divergence.
Can I mount a ConfigMap and a Secret at the same path?
Yes, but they must be combined using subPath or a projected volume. Mounting them directly at the same path causes one to mask the other, as observed.
How do I debug a “could not open configuration file” error when the file exists on the host?
Inspect the container’s mount table with cat /proc/1/mountinfo and verify that the expected volume source (ConfigMap or Secret) is attached to the target path.
Is it safe to let an init container overwrite files in a ConfigMap volume?
No. ConfigMap volumes are read‑only from the perspective of the pod; an init container that writes to the same mount point will either fail (if readOnly) or, as in this case, replace the projected data with an empty directory.
What Kubernetes version introduced the projected volume type that could simplify this setup?
Projected volumes have been stable since Kubernetes 1.9. They allow you to combine multiple ConfigMaps, Secrets, and downward API items into a single mount point without name collisions.