RAG Query Decomposition Error After Adding a New Microservice on GCP Compute

Q: Problem

After deploying a new stateless microservice (order‑processor) on a GCP Compute Engine instance, the Retrieval‑Augmented Generation (RAG) service began returning DecompositionError: unable to split query into sub‑queries. The error appears only for multi‑step queries that require data from both the existing inventory service and the newly added order‑processor service.

Q: Debug 1. Inspect the orchestrator logs

2026-10-13 07:42:11,842 INFO orchestrator.graph_builder - Loading OpenAPI spec from http://order-processor.internal:8080/openapi.json 2026-10-13 07:42:12,013 ERROR orchestrator.graph_builder - DecompositionError: unable to split query into sub-queries (duplicate node id: /orders) 2. Verify the OpenAPI document of the new service curl -s http://order-processor.internal:8080/openapi.json | jq '.paths["/orders"]' { "post": { "summary": "Create a new order", "requestBody": { "content": { "application/json": { "schema": { "oneOf": [ { "$ref": "#/components/schemas/PhysicalOrder" }, { "$ref": "#/components/schemas/DigitalOrder" } ] } } } }, "responses": { "201": { "description": "Created" } } // NOTE: operationId missing } } 3. Compare with the existing inventory service spec curl -s http://inventory.internal:8080/openapi.json | jq '.paths["/orders"]' { "get": { "operationId": "listInventoryOrders", "summary": "List orders in inventory", "responses": { "200": { "description": "OK" } } } } 4. Check the reverse‑proxy configuration # /etc/nginx/conf.d/reverse-proxy.conf location /orders { proxy_pass http://inventory.internal:8080/v1/orders; } # No rule for order‑processor, so /orders resolves to the new service directly 5. Validate the service registration payload sent to the orchestrator curl -s http://orchestrator.internal:8080/services | jq '.[] | select(.name=="order-processor")' { "name": "order-processor", "baseUrl": "http://order-processor.internal:8080", "openApiUrl": "http://order-processor.internal:8080/openapi.json" } Solution 1. Add unique operationId values to the new service’s OpenAPI spec Update order‑processor/openapi.yaml:

Q: 2. Rename the endpoint to avoid path collision (optional but recommended)

Change the endpoint to /order‑v2 and update the client code accordingly.

Problem

After deploying a new stateless microservice (order‑processor) on a GCP Compute Engine instance, the Retrieval‑Augmented Generation (RAG) service began returning DecompositionError: unable to split query into sub‑queries. The error appears only for multi‑step queries that require data from both the existing inventory service and the newly added order‑processor service.

Typical request

curl -X POST https://rag.example.com/query \
  -H "Content-Type: application/json" \
  -d '{"question":"What is the stock level of product X and has it been ordered in the last 24h?"}'

Observed response

{
  "error": "DecompositionError: unable to split query into sub-queries"
}

Root Cause

The RAG orchestrator builds a directed acyclic graph (DAG) of sub‑queries by inspecting the OpenAPI specifications of all registered microservices. The new order‑processor service was registered with an OpenAPI document that contains a oneOf schema for the /orders endpoint but omits the operationId field. The orchestrator relies on operationId to generate unique node identifiers. When the field is missing, the DAG construction falls back to using the HTTP path as the identifier, which collides with the existing /orders path defined in the inventory service (both expose /orders for different purposes). The collision causes the graph builder to abort, emitting the DecompositionError.

Additional contributing factors:

The Compute Engine instance runs the service behind a nginx reverse proxy that rewrites /orders to /v1/orders only for the inventory service, leaving the new service reachable at the raw path.
Health‑check logs show intermittent 404 responses from the orchestrator when it attempts to fetch the OpenAPI spec of order‑processor during startup.

Debug

1. Inspect the orchestrator logs

2026-10-13 07:42:11,842 INFO  orchestrator.graph_builder - Loading OpenAPI spec from http://order-processor.internal:8080/openapi.json
2026-10-13 07:42:12,013 ERROR orchestrator.graph_builder - DecompositionError: unable to split query into sub-queries (duplicate node id: /orders)

2. Verify the OpenAPI document of the new service

curl -s http://order-processor.internal:8080/openapi.json | jq '.paths["/orders"]'

{
  "post": {
    "summary": "Create a new order",
    "requestBody": {
      "content": {
        "application/json": {
          "schema": {
            "oneOf": [
              { "$ref": "#/components/schemas/PhysicalOrder" },
              { "$ref": "#/components/schemas/DigitalOrder" }
            ]
          }
        }
      }
    },
    "responses": { "201": { "description": "Created" } }
    // NOTE: operationId missing
  }
}

3. Compare with the existing inventory service spec

curl -s http://inventory.internal:8080/openapi.json | jq '.paths["/orders"]'

{
  "get": {
    "operationId": "listInventoryOrders",
    "summary": "List orders in inventory",
    "responses": { "200": { "description": "OK" } }
  }
}

4. Check the reverse‑proxy configuration

# /etc/nginx/conf.d/reverse-proxy.conf
location /orders {
    proxy_pass http://inventory.internal:8080/v1/orders;
}
# No rule for order‑processor, so /orders resolves to the new service directly

5. Validate the service registration payload sent to the orchestrator

curl -s http://orchestrator.internal:8080/services | jq '.[] | select(.name=="order-processor")'

{
  "name": "order-processor",
  "baseUrl": "http://order-processor.internal:8080",
  "openApiUrl": "http://order-processor.internal:8080/openapi.json"
}

Solution

1. Add unique `operationId` values to the new service’s OpenAPI spec

Update order‑processor/openapi.yaml:

# Before
paths:
  /orders:
    post:
      summary: Create a new order
      requestBody:
        content:
          application/json:
            schema:
              oneOf:
                - $ref: '#/components/schemas/PhysicalOrder'
                - $ref: '#/components/schemas/DigitalOrder'
      responses:
        '201':
          description: Created
# operationId missing

# After
paths:
  /orders:
    post:
      operationId: createOrder
      summary: Create a new order
      requestBody:
        content:
          application/json:
            schema:
              oneOf:
                - $ref: '#/components/schemas/PhysicalOrder'
                - $ref: '#/components/schemas/DigitalOrder'
      responses:
        '201':
          description: Created

2. Rename the endpoint to avoid path collision (optional but recommended)

Change the endpoint to /order‑v2 and update the client code accordingly.

# Updated path
paths:
  /order-v2:
    post:
      operationId: createOrderV2
      ...

3. Adjust the Nginx reverse‑proxy to route the new path explicitly

# /etc/nginx/conf.d/reverse-proxy.conf
location /order-v2 {
    proxy_pass http://order-processor.internal:8080/order-v2;
}

4. Redeploy the updated microservice and reload Nginx

# Deploy new container image
gcloud compute instances update-container order-processor \
  --container-image=gcr.io/project/order-processor:20231013-v2

# Reload Nginx
sudo systemctl reload nginx

5. Re‑register the service with the orchestrator (if registration is not automatic)

curl -X POST http://orchestrator.internal:8080/register \
  -H "Content-Type: application/json" \
  -d '{
        "name":"order-processor",
        "baseUrl":"http://order-processor.internal:8080",
        "openApiUrl":"http://order-processor.internal:8080/openapi.json"
      }'

Verify

1. Confirm the orchestrator builds the DAG without errors

2026-10-13 08:12:04,211 INFO  orchestrator.graph_builder - Loaded OpenAPI spec from http://order-processor.internal:8080/openapi.json
2026-10-13 08:12:04,312 INFO  orchestrator.graph_builder - DAG construction completed successfully (nodes: 12)

2. Run a multi‑step query again

curl -X POST https://rag.example.com/query \
  -H "Content-Type: application/json" \
  -d '{"question":"What is the stock level of product X and has it been ordered in the last 24h?"}'

{
  "answer": "Product X has 42 units in stock and was ordered 3 times in the last 24 hours."
}

3. Check health endpoints

curl -s http://order-processor.internal:8080/health | jq .
{
  "status": "UP",
  "uptime": "5m12s"
}

4. Verify Nginx routing

curl -I http://gateway.example.com/order-v2
HTTP/1.1 200 OK
Server: nginx/1.22.0
...

Prevent

OpenAPI validation pipeline: integrate swagger-cli validate into CI to enforce presence of operationId for every operation.
Namespace paths: adopt a naming convention that prefixes service‑specific resources (e.g., /order‑v2, /inventory‑v1) to avoid accidental collisions.
Automated DAG sanity checks: after each service registration, run a lightweight orchestrator health probe that attempts to build the query graph and reports duplicate node IDs.
Centralised reverse‑proxy config management: store Nginx location blocks in a version‑controlled repository and lint for overlapping paths.
Observability: add a metric rag.graph_builder.errors_total and alert on any non‑zero value within 5 minutes of a new service deployment.

Related Topic Hub: Cloud Infrastructure Troubleshooting Hub

FAQ

Why does the error only appear for multi‑step queries?
Single‑step queries bypass the DAG construction because they target a single endpoint. The duplicate node ID is discovered only when the orchestrator attempts to merge sub‑queries into a graph.
Can I keep the original /orders path if I add a unique operationId?
Yes. The orchestrator uses operationId as the primary identifier; a unique value eliminates the collision even if the HTTP path is shared.
What if my service already defines operationId but the error persists?
Verify that the operationId values are globally unique across all registered services. Duplicates in different services still cause DAG construction failures.
How do I debug similar errors in a Kubernetes‑based deployment?
Export the orchestrator’s GRAPH_BUILDER_DEBUG flag, inspect the generated DOT file, and compare node identifiers against the combined OpenAPI specs of all pods.
Is there a way to make the orchestrator fallback to a safe mode instead of failing?
The orchestrator can be started with --allow‑duplicate‑ids, which will auto‑suffix colliding IDs. This is not recommended for production because it can mask genuine specification problems.