OpenAI GPT-3.5 API Server Unreachable In Hybrid Cloud Setup

Q: Investigation and Debugging Steps 1. Verify Network Reachability

# Simple TCP check from the on‑prem host nc -zv api.openai.com 443 # Expected output # Connection to api.openai.com 443 port [tcp/https] succeeded! 2. Inspect DNS Resolution # Resolve with the system resolver nslookup api.openai.com # Expected output (public IPs) # Non‑authoritative answer: # Name: api.openai.com # Addresses: 104.20.22.46, 104.20.23.46 If the response contains an internal IP range (e.g., 10.x.x.x) the internal DNS is overriding the public record.

Q: 3. Capture TLS Handshake

# Using openssl to view SNI and certificate chain openssl s_client -connect api.openai.com:443 -servername api.openai.com -tls1_2 Look for:

Q: 5. Examine Firewall and NAT Logs

# Example firewall log snippet (Cisco ASA) %ASA-6-106015: Deny tcp (no connection) from 10.1.2.3/54321 to 104.20.22.46/443 flags RST Presence of deny entries for port 443 confirms a blocking rule.

Q: 6. SDK‑Level Timeout and Proxy Settings

# Node.js example with openai-node SDK const { Configuration, OpenAIApi } = require("openai"); const configuration = new Configuration({ apiKey: process.env.OPENAI_API_KEY, // Explicit proxy configuration baseOptions: { proxy: { host: "proxy.corp.example.com", port: 3128, protocol: "http" }, timeout: 60000, // 60 s // Force HTTP/1.1 httpAgent: new (require("http")).Agent({ keepAlive: true }) } }); const openai = new OpenAIApi(configuration); Resolution – Making the API Reachable 1. Open Outbound Port 443 to api.openai.com Before (firewall rule missing):

Q: 4. Fix Split‑Horizon DNS

Update the internal DNS zone to forward api.openai.com queries to an external resolver (e.g., 8.8.8.8) or add a static A record with the public IPs.

Q: 5. Adjust Proxy Timeouts and Enable Keep‑Alive

# Squid proxy configuration request_timeout 120 seconds read_timeout 300 seconds client_persistent_connections on Validation – Confirming the Fix Re‑run the TCP check: nc -zv api.openai.com 443 should succeed. Execute the TLS handshake test; verify the SNI line appears. Run a simple OpenAI request with increased timeout: curl -X POST https://api.openai.com/v1/chat/completions \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Hello"}]}' \ --max-time 60 Expected JSON response containing a choices array.

In this article

Table of Contents

Problem – OpenAI GPT‑3.5 API Server Unreachable in a Hybrid Cloud

Client applications running in an on‑premises data centre repeatedly receive either a timeout or a 503 Service Unavailable response when calling https://api.openai.com/v1/chat/completions. The symptoms manifest as:

Log entry: Error: request timed out after 30 seconds
Log entry: 503 Service Unavailable: The server is currently unable to handle the request
SDK exception: OpenAIError: connection aborted
Occasional DNS error: NetworkError: getaddrinfo ENOTFOUND api.openai.com

The environment consists of:

On‑prem private network with strict outbound firewall rules.
Corporate API gateway that terminates TLS and forwards traffic to the public internet.
Optional HTTP proxy used for outbound traffic.
Hybrid‑cloud DNS that may resolve api.openai.com to internal IPs.

Root Cause Analysis

OpenAI’s API expects a direct TLS‑encrypted HTTP/1.1 request to api.openai.com on port 443, with a valid Host header and SNI matching the hostname. The official API reference defines the error semantics:

Error Code	Typical Meaning
429	Rate‑limit exceeded
503	Service overload or connectivity blockage (e.g., firewall, DNS, TLS termination failure)

Investigation of the evidence package reveals several recurring failure modes that map directly to the observed errors:

Outbound firewall blocking port 443 – Enterprise A’s outage was traced to a missing rule allowing api.openai.com traffic.
API gateway stripping SNI – Company B’s gateway performed TLS termination but omitted the SNI header when re‑establishing the outbound TLS session, causing OpenAI edge servers to reject the connection with a 503.
Split‑horizon DNS mis‑resolution – Internal DNS returned a private IP for api.openai.com, leading to ENOTFOUND or connection timeouts.
HTTP/2 incompatibility on outbound proxy – Data centre X observed intermittent 503s after enabling HTTP/2; forcing HTTP/1.1 resolved the issue.
Proxy timeout settings – The Stack Overflow discussion shows default proxy timeouts (often 15 s) causing premature aborts before OpenAI’s response is received.

In most hybrid‑cloud deployments the root cause is a combination of firewall/NAT timeouts and TLS termination that does not preserve the required SNI header.

Investigation and Debugging Steps

1. Verify Network Reachability

# Simple TCP check from the on‑prem host
nc -zv api.openai.com 443
# Expected output
# Connection to api.openai.com 443 port [tcp/https] succeeded!

2. Inspect DNS Resolution

# Resolve with the system resolver
nslookup api.openai.com
# Expected output (public IPs)
# Non‑authoritative answer:
# Name:    api.openai.com
# Addresses: 104.20.22.46, 104.20.23.46

If the response contains an internal IP range (e.g., 10.x.x.x) the internal DNS is overriding the public record.

3. Capture TLS Handshake

# Using openssl to view SNI and certificate chain
openssl s_client -connect api.openai.com:443 -servername api.openai.com -tls1_2

Look for:

Server Name Indication: api.openai.com
Valid certificate chain ending in DigiCert SHA2 Secure Server CA

4. Review API Gateway Configuration

Check that the gateway:

Preserves the original Host header.
Forwards the SNI value when establishing the outbound TLS session.
Uses HTTP/1.1 for outbound calls to OpenAI.

5. Examine Firewall and NAT Logs

# Example firewall log snippet (Cisco ASA)
%ASA-6-106015: Deny tcp (no connection) from 10.1.2.3/54321 to 104.20.22.46/443 flags RST

Presence of deny entries for port 443 confirms a blocking rule.

6. SDK‑Level Timeout and Proxy Settings

# Node.js example with openai-node SDK
const { Configuration, OpenAIApi } = require("openai");
const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
  // Explicit proxy configuration
  baseOptions: {
    proxy: {
      host: "proxy.corp.example.com",
      port: 3128,
      protocol: "http"
    },
    timeout: 60000, // 60 s
    // Force HTTP/1.1
    httpAgent: new (require("http")).Agent({ keepAlive: true })
  }
});
const openai = new OpenAIApi(configuration);

Resolution – Making the API Reachable

1. Open Outbound Port 443 to `api.openai.com`

Before (firewall rule missing):

# iptables -L OUTPUT -v -n
Chain OUTPUT (policy DROP)
target     prot opt source               destination

After (allow rule added):

# iptables -A OUTPUT -p tcp -d api.openai.com --dport 443 -j ACCEPT
# iptables -L OUTPUT -v -n
Chain OUTPUT (policy DROP)
target     prot opt source               destination
ACCEPT     tcp  --  0.0.0.0/0            api.openai.com      tcp dpt:443

2. Ensure SNI Propagation on the API Gateway

For an NGINX‑based gateway, add:

proxy_ssl_server_name on;
proxy_ssl_name $host;

This forces NGINX to include the original hostname as SNI when opening the upstream TLS connection.

3. Disable HTTP/2 for Outbound Calls

In an Envoy proxy configuration:

transport_socket:
  name: envoy.transport_sockets.tls
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
    common_tls_context:
      alpn_protocols: ["http/1.1"]

4. Fix Split‑Horizon DNS

Update the internal DNS zone to forward api.openai.com queries to an external resolver (e.g., 8.8.8.8) or add a static A record with the public IPs.

5. Adjust Proxy Timeouts and Enable Keep‑Alive

# Squid proxy configuration
request_timeout 120 seconds
read_timeout 300 seconds
client_persistent_connections on

Validation – Confirming the Fix

Re‑run the TCP check: nc -zv api.openai.com 443 should succeed.
Execute the TLS handshake test; verify the SNI line appears.
Run a simple OpenAI request with increased timeout:

curl -X POST https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"gpt-3.5-turbo","messages":[{"role":"user","content":"Hello"}]}' \
  --max-time 60

Expected JSON response containing a choices array.

Check application logs for the absence of 503 or timeout entries. Verify monitoring dashboards show successful request latency within the SLA.

Prevention – Operational Guardrails

Monitoring: Alert on any 5xx from api.openai.com and on outbound TCP connection failures to port 443.
Configuration as Code: Store firewall rules, DNS overrides, and gateway settings in version‑controlled manifests (e.g., Terraform, Ansible) and run CI checks for drift.
Health‑check job: Deploy a lightweight cron that periodically calls /v1/models and reports status to a central observability platform.
Proxy & Gateway Audits: Quarterly review to ensure SNI forwarding and HTTP/1.1 usage for all external APIs that require it.
Retry Strategy: Follow OpenAI’s recommended exponential back‑off for transient 429/503 responses.

FAQ – Common Follow‑Up Questions

Why does the OpenAI API work from my laptop but not from the on‑prem server?
Because the laptop uses the public internet without corporate firewall or gateway constraints, while the on‑prem server’s outbound traffic is filtered or altered by the firewall, DNS, and API gateway.
Can I use a corporate HTTP proxy with the OpenAI SDKs?
Yes. Configure the SDK’s proxy option (Node, Python, etc.) and ensure the proxy permits TLS CONNECT to api.openai.com:443. Increase the proxy’s request_timeout to exceed the SDK’s timeout.
What does “TLS handshake failed: certificate verify failed” indicate?
The gateway’s TLS termination presented a certificate chain that the OpenAI endpoint could not validate, often because the gateway used a self‑signed cert or omitted the required SNI. Enabling proxy_ssl_server_name on; (NGINX) or equivalent fixes it.
Is HTTP/2 ever supported for OpenAI calls?
OpenAI’s edge infrastructure currently expects HTTP/1.1 for API calls. Enabling HTTP/2 on outbound proxies can cause malformed frames and result in 503 responses.
How should I handle intermittent 503 errors?
Implement exponential back‑off with jitter as per OpenAI’s rate‑limit guidance, and verify that the network path remains stable (no firewall rule flaps, DNS TTL mismatches, or proxy resets).

Related Topic Hub: LLM Systems Troubleshooting Hub