RAG citation JSON‑LD missing fields after model version upgrade

Problem: Malformed JSON‑LD Citations in RAG Responses after Model Version Upgrade

The AI service running on Google Compute Engine (GCE) VMs generates Retrieval‑Augmented Generation (RAG) answers that include citations. After upgrading the underlying LLM from v1.2.0 to v1.3.0, the citation strings no longer conform to the required JSON‑LD schema. Missing fields (@type, url), stray delimiters, and malformed arrays cause downstream services—particularly the documentation aggregation microservice—to reject the payload with JSON parsing errors.

Root Cause Analysis

The upgrade introduced a change in the CitationFormatter component:

• Previous version emitted a fully‑populated Citation object and serialized it with json.dumps(..., ensure_ascii=False).
• New version switched to a lazy‑construction path that only populates fields present in the prompt template. The template used in production omitted optional fields (author, datePublished) and, due to a bug, also omitted mandatory fields (@type, url).
• The lazy path concatenates individual field strings using "," without guarding against empty values, resulting in trailing commas (e.g., {"@type":"Article",, "title":"..."}).
• Because the service runs inside a Docker container on GCE, the new binary was built with a different jsonschema version that silently drops required validation when strict=False is set, allowing malformed JSON‑LD to slip through.

Debugging Steps

1. Capture the failing payloads

2026-09-28 14:32:11.842 INFO  rag_service: Received request id=7f3c9a2b
2026-09-28 14:32:12.013 ERROR rag_service: Citation serialization error
Traceback (most recent call last):
  File "/app/rag/citation.py", line 112, in format_citation
    json_str = json.dumps(citation_dict)
  File "/usr/local/lib/python3.11/json/__init__.py", line 238, in dumps
    **kw)
json.JSONDecodeError: Expecting ':' delimiter: line 1 column 27 (char 26)
Payload snippet:
{
  "@type":"Article",,
  "title":"Understanding GCP Load Balancing",
  "url":"https://cloud.google.com/load-balancing"
}

2. Verify the model version inside the container

$ docker exec -it rag-service cat /app/version.txt
v1.3.0

3. Compare the formatter implementation between versions

# v1.2.0 (working)
def format_citation(cite):
    citation = {
        "@type": "Article",
        "title": cite.title,
        "url": cite.url,
        "author": cite.author,
        "datePublished": cite.date,
    }
    return json.dumps(citation)

# v1.3.0 (current)
def format_citation(cite):
    fields = []
    if cite.title:
        fields.append(f'"title":"{cite.title}"')
    if cite.url:
        fields.append(f'"url":"{cite.url}"')
    # NOTE: @type and author are optional in this branch – bug!
    citation_str = "{" + ",".join(fields) + "}"
    return citation_str

4. Inspect the JSON‑LD schema validation configuration

# settings.yaml
jsonschema:
  strict: false   # <-- introduced in v1.3.0
  schema_path: /app/schemas/citation_schema.json

5. Reproduce locally with the same input

$ python - <<'PY'
from rag.citation import format_citation
class Cite: pass
c = Cite()
c.title = "Understanding GCP Load Balancing"
c.url = "https://cloud.google.com/load-balancing"
print(format_citation(c))
PY
{"title":"Understanding GCP Load Balancing","url":"https://cloud.google.com/load-balancing"}

Note the missing @type field, which downstream services require.

Solution

1. Restore mandatory field population

Update CitationFormatter.format_citation to always emit required JSON‑LD keys and to use a safe serializer.

# Before (v1.3.0)
def format_citation(cite):
    fields = []
    if cite.title:
        fields.append(f'"title":"{cite.title}"')
    if cite.url:
        fields.append(f'"url":"{cite.url}"')
    citation_str = "{" + ",".join(fields) + "}"
    return citation_str

# After (fixed)
import json

REQUIRED_FIELDS = {
    "@type": "Article",
    "title": None,
    "url": None,
}

def format_citation(cite):
    # Start with required skeleton
    citation = {k: v for k, v in REQUIRED_FIELDS.items()}
    # Populate optional values if present
    if cite.title:
        citation["title"] = cite.title
    if cite.url:
        citation["url"] = cite.url
    if getattr(cite, "author", None):
        citation["author"] = cite.author
    if getattr(cite, "datePublished", None):
        citation["datePublished"] = cite.datePublished
    # Use json.dumps to guarantee valid JSON‑LD
    return json.dumps(citation, ensure_ascii=False)

2. Reinstate strict schema validation

Set strict: true in settings.yaml and add a CI test that validates a sample citation against the schema.

# settings.yaml (updated)
jsonschema:
  strict: true
  schema_path: /app/schemas/citation_schema.json

3. Redeploy the corrected container

# Build and push
docker build -t gcr.io/my-project/rag-service:1.3.1 .
docker push gcr.io/my-project/rag-service:1.3.1

# Rolling update via instance group manager
gcloud compute instance-groups managed rolling-action replace \
  rag-igm --version=1.3.1 --max-surge=1 --max-unavailable=0

Verification

Functional test

$ curl -s -X POST https://rag.mycompany.com/v1/generate \
  -H "Content-Type: application/json" \
  -d '{"query":"Explain GCP load balancing"}' | jq '.citations[0]'

{
  "@type": "Article",
  "title": "Understanding GCP Load Balancing",
  "url": "https://cloud.google.com/load-balancing",
  "author": "Google Cloud Docs",
  "datePublished": "2024-07-15"
}

Schema validation log

2026-09-29 08:14:02.331 INFO  citation_validator: Payload validated against citation_schema.json (status=OK)

Downstream service health check

$ curl -s http://doc-aggregator.internal/healthz
{"status":"healthy","citations_parsed":12457}

Prevention Strategies

• Schema‑first development: Keep the JSON‑LD schema in a shared repository and generate data‑classes from it. Any change must be accompanied by a schema version bump and CI validation.
• Strict validation in production: Never disable strict mode in jsonschema. Use jsonschema.validate at the service boundary.
• Automated contract tests: Add an integration test that sends a typical RAG request and asserts that every citation contains @type, title, and url.
• Version pinning of third‑party libraries: Lock jsonschema and pydantic to known‑good versions in requirements.txt to avoid silent behavior changes.
• Observability: Emit a metric citation_malformed_total whenever serialization fails; set an alert threshold of >0 per minute.

Related Topic Hub: Cloud Infrastructure Troubleshooting Hub

FAQ

1. Why did the citation format break only after the model upgrade? The upgrade introduced a new lazy‑construction path in CitationFormatter that omitted mandatory fields. The previous eager‑construction path always populated the required keys.
2. Can I keep the lazy‑construction logic and still meet the schema? Yes, but you must explicitly insert required fields before joining strings, or better, build a dict and let json.dumps handle serialization.
3. What if downstream services need additional fields like publisher? Extend REQUIRED_FIELDS and update the shared JSON‑LD schema. The strict validator will enforce presence across all services.
4. How do I detect malformed citations in real time? Increment a Prometheus counter in the exception handler of format_citation and configure an alert on non‑zero values.
5. Is there a way to test schema compliance without deploying? Use the jsonschema CLI in CI pipelines:

   
   jsonschema -i sample_citation.json /app/schemas/citation_schema.json