Redis node fails to bind port on edge device after reboot

Problem

An edge computing device that runs a Redis instance as an AI inference cache fails to start after a power‑cycle. The Redis service exits with a fatal error indicating that the configured port (default 6379) cannot be bound.


2026-10-21T12:34:56.789Z * Fatal error, can't open port 6379: Address already in use
2026-10-21T12:34:56.789Z * Redis terminated

Because the node cannot bind the port, the distributed inference pipeline loses its local cache, leading to increased latency and higher upstream traffic.

Root Cause

The conflict originates from a stray process that automatically starts after reboot and binds to 0.0.0.0:6379 before the Redis service is launched. On this particular edge device the following conditions contributed:

  • Startup order: A custom telemetry agent (telemetryd) is enabled via rc.local and listens on the default Redis port for legacy metrics collection.
  • IPv4/IPv6 binding mismatch: Redis is configured to bind only to IPv4 (bind 0.0.0.0) while telemetryd binds to :::6379. The kernel treats the IPv6 wildcard as covering the IPv4 space, causing an “address already in use” error for any subsequent IPv4 listener.
  • Persisted PID file: In a previous run Redis crashed without cleaning /var/run/redis/redis.pid. Systemd interprets the stale PID as a running instance and skips the ExecStartPre check, allowing the stale PID to be reused after the reboot.

Debug

The following step‑by‑step investigation reproduces the findings on the edge device.

1. Verify the port is occupied

# ss -tulpn | grep :6379
LISTEN 0      128          0.0.0.0:6379          0.0.0.0:*      users:(("telemetryd",pid=3124,fd=5))

2. Identify the owning process

# lsof -iTCP:6379 -sTCP:LISTEN -Pn
COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
telemetryd 3124 root    5u  IPv6  12345      0t0  TCP *:6379 (LISTEN)

3. Check Redis configuration for bind directives

# cat /etc/redis/redis.conf | grep -E 'bind|port'
bind 0.0.0.0
port 6379

4. Inspect systemd unit ordering

# systemctl cat redis.service
[Unit]
Description=Redis In-Memory Data Store
After=network.target

[Service]
ExecStart=/usr/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/bin/redis-cli shutdown
PIDFile=/var/run/redis/redis.pid

Note the missing After=telemetryd.service dependency, allowing telemetryd to start earlier.

5. Review the stale PID file

# cat /var/run/redis/redis.pid

(File is empty – indicates a previous crash left the file in place.)

Solution

Three corrective actions were applied to guarantee Redis can bind its port after every reboot.

1. Reconfigure the conflicting service

Modify telemetryd to use an alternative port (6380) and restart it.

# /etc/telemetryd/telemetryd.conf
# Before
listen_port=6379

# After
listen_port=6380
# systemctl restart telemetryd
# ss -tulpn | grep :6379
(no output)

2. Adjust Redis bind settings to include IPv6 explicitly

Updating the bind line prevents the IPv6 wildcard from shadowing the IPv4 listener.

# cp /etc/redis/redis.conf /etc/redis/redis.conf.bak

# Before
bind 0.0.0.0

# After
bind 0.0.0.0 ::1

3. Harden the systemd unit

Add an explicit ordering dependency and a pre‑start check that aborts if the port is already in use.

# cat > /etc/systemd/system/redis.service.d/override.conf <<'EOF'
[Unit]
After=network-online.target telemetryd.service
Wants=network-online.target

[Service]
ExecStartPre=/usr/bin/bash -c 'ss -tulpn | grep -q ":6379" && exit 1 || exit 0'
EOF

# systemctl daemon-reload
# systemctl restart redis

4. Clean stale PID file on boot

Insert a tiny init script that removes the PID file before Redis starts.

# cat /etc/systemd/system/redis-clean-pid.service <<'EOF'
[Unit]
Description=Remove stale Redis PID file
Before=redis.service
ConditionPathExists=/var/run/redis/redis.pid

[Service]
Type=oneshot
ExecStart=/usr/bin/rm -f /var/run/redis/redis.pid

[Install]
WantedBy=multi-user.target
EOF

# systemctl enable redis-clean-pid.service

Verify

After applying the changes, perform the following checks:

  1. Confirm the port is free before Redis starts:
# ss -tulpn | grep :6379
(no output)
  1. Start Redis and inspect its logs:
# systemctl start redis
# journalctl -u redis -n 5
Oct 21 12:45:01 edge-device redis[4231]: * Ready to accept connections
Oct 21 12:45:01 edge-device redis[4231]: * DB loaded from disk: 0.000 seconds
  1. Run a health check via redis-cli:
# redis-cli ping
PONG
  1. Validate that telemetryd is listening on the new port:
# ss -tulpn | grep :6380
LISTEN 0      128          0.0.0.0:6380          0.0.0.0:*      users:(("telemetryd",pid=3124,fd=5))

Prevent

Implementing the following guardrails reduces the likelihood of recurrence:

  • Port reservation monitoring: Deploy a lightweight watchdog (e.g., a systemd PathExists unit) that alerts when any process binds to 6379 before Redis starts.
  • Explicit service dependencies: All services that may share ports must declare After=redis.service or Before=redis.service as appropriate.
  • Configuration audit: Include a CI check that parses redis.conf and other service configs for overlapping listen_port values.
  • PID file hygiene: Use RuntimeDirectory=redis in the systemd unit to let systemd manage the PID directory, eliminating stale files.
  • IPv6 awareness: When binding to IPv4 only, also bind to ::1 or explicitly set protected‑mode no if IPv6 is not required, to avoid wildcard shadowing.

Related Topic Hub: Data Infrastructure Troubleshooting Hub

FAQ

  1. Why does Redis sometimes start fine after a reboot but fail after a subsequent one?
    Because the conflicting service (telemetryd) may start before Redis only on a cold boot. If the device is rebooted quickly, the stale PID file can also cause a race condition where Redis thinks it is already running.
  2. Can I keep both services on the same port using SO_REUSEPORT?
    Redis does not enable SO_REUSEPORT and relies on exclusive binding for data integrity. Sharing the port would corrupt the in‑memory data structures and is unsupported.
  3. Is the IPv6 wildcard (::) always the cause of IPv4 port conflicts?
    On Linux, binding to :: without the ipv6only=1 socket option also reserves the corresponding IPv4 address. Explicitly binding to both 0.0.0.0 and ::1 or setting net.ipv6.bindv6only=1 avoids the shadowing.
  4. How can I automate detection of port collisions before a service starts?
    Add an ExecStartPre command in the systemd unit that runs ss -tulpn | grep -q ":PORT" and exits non‑zero if a match is found. Systemd will then abort the start and emit a clear error.
  5. What should I do if the conflicting process is a third‑party binary I cannot modify?
    Change Redis’s listening port (e.g., to 6381) and update all clients accordingly, or use a firewall rule (iptables -A INPUT -p tcp --dport 6379 -j DROP) to block the unwanted process from binding, then restart the system.