Weaviate RAG prompt rendering error after updating template placeholders

Problem – RAG Prompt Rendering Failure After Updating Template Placeholders

In a staging deployment of Weaviate (Docker Compose, PostgreSQL vector store, generative AI module v1.22), updating the prompt_template.yaml to replace the built‑in {{question}} placeholder with a custom name (e.g., {{user_query}}) caused the generative endpoint to return HTTP 500. The logs contain errors such as:

Error rendering prompt: KeyError: 'user_query'
InvalidRequestError: Prompt contains unresolved placeholders
TemplateRuntimeError: Undefined variable 'user_query'

Result: the RAG query never reaches the LLM because the prompt sent to the model is malformed.

Root Cause Analysis

Supported placeholder syntax

The official documentation for the Generative AI module states that only the placeholders defined by the module are guaranteed to be substituted: {{question}}, {{context}}, and a few class‑property tokens (Weaviate Generative AI – Prompt templates).

Template versioning and strict Jinja2 mode

Starting with v1.22 the internal Jinja2 engine runs in strict mode. Any variable that is not present in the rendering context raises a TemplateRuntimeError (GitHub issue #2319). This change turned a previously silent omission into a hard failure.

Configuration mismatch

• The environment variable WEAVIATE_ENABLE_PROMPT_TEMPLATES must be set to true for custom files to be loaded (community forum thread).
• After the Docker Compose update, the PROMPT_TEMPLATE_FILE path still pointed to the old file, causing Weaviate to fall back to the default template where {{question}} is still expected (GitHub issue #2984).

Combined, these factors mean that the new placeholder name is never supplied to the rendering engine, leading to the observed KeyError and InvalidRequestError.

Investigation & Debugging Steps

1. Confirm the template file is being read.

   docker exec -it weaviate cat /etc/weaviate/prompt_template.yaml

   Expected output should show the updated placeholders.

2. Check environment variables in the running container.

   docker exec -it weaviate env | grep -E 'WEAVIATE|PROMPT'

   Look for:

   WEAVIATE_ENABLE_PROMPT_TEMPLATES=true
   PROMPT_TEMPLATE_FILE=/etc/weaviate/prompt_template.yaml

   If the flag is missing or set to false, the custom file is ignored.

3. Inspect the rendering context. Enable debug logging for the generative module (add LOG_LEVEL=DEBUG to the container env) and search for the line:

   DEBUG generative.rendering: rendering context: {'question': '…', 'context': '…'}

   If user_query is absent, the placeholder will fail.

4. Validate Jinja2 strict mode behavior. Reproduce the error with a minimal template:

   template: "Answer: {{user_query}}"

   Run a curl request:

   curl -X POST http://localhost:8080/v1/generate \
     -H "Content-Type: application/json" \
     -d '{"question":"What is Weaviate?"}'

   Observe the same TemplateRuntimeError.

5. Check for stale template cache. Restart the Weaviate container to force a cache refresh:

   docker compose restart weaviate

   If the error disappears after restart, a race condition was caching the old template (issue #2984).

Resolution – Align Placeholder Names and Configuration

Option 1 – Use Built‑in Placeholders

Revert the template to the supported {{question}} placeholder.

# before (broken)
prompt: |
  Answer the following question using the context:
  {{context}}
  Question: {{user_query}}

# after (working)
prompt: |
  Answer the following question using the context:
  {{context}}
  Question: {{question}}

Why it works: The rendering engine always provides question and context variables, so no KeyError occurs.

Option 2 – Extend the Rendering Context

If a custom name is required, expose it via the additionalProperties field in the query payload and map it in the module configuration.

# prompt_template.yaml (custom placeholder)
prompt: |
  Answer using the provided context:
  {{context}}
  Query: {{user_query}}

# weaviate.yaml (module config)
generative:
  enabled: true
  promptTemplateFile: /etc/weaviate/prompt_template.yaml
  additionalVariables:
    - name: user_query
      source: request.question

After adding additionalVariables, restart the container.

Update Docker Compose

# docker-compose.yml snippet
services:
  weaviate:
    image: semitechnologies/weaviate:1.22.0
    environment:
      - WEAVIATE_ENABLE_PROMPT_TEMPLATES=true
      - PROMPT_TEMPLATE_FILE=/etc/weaviate/prompt_template.yaml
      - LOG_LEVEL=DEBUG
    volumes:
      - ./prompt_template.yaml:/etc/weaviate/prompt_template.yaml:ro

Restart:

docker compose up -d weaviate

Verification – Confirm Prompt Renders Correctly

1. Send a RAG request:

   curl -X POST http://localhost:8080/v1/generate \
     -H "Content-Type: application/json" \
     -d '{"question":"Explain vector search"}'

2. Inspect the response payload. The prompt field (if logged) should contain the substituted values, e.g.:

   {
     "result": "Vector search allows..."
   }
   

3. Check the Weaviate logs for the absence of rendering errors:

   2026-08-31T12:34:56Z INFO generative.rendering: rendered prompt successfully

4. Optionally, enable the WEAVIATE_PROMPT_RENDER_DEBUG=true flag to log the final prompt string for audit.

Prevention – Guardrails for Future Template Changes

• Validate template syntax before deployment. Use a local Jinja2 linter:

  python -c "import jinja2, sys; jinja2.Environment(undefined=jinja2.StrictUndefined).from_string(open('prompt_template.yaml').read())"

• Automate placeholder verification. In CI, run a smoke test that sends a dummy query and asserts that the response does not contain raw {{.*}} tokens.
• Pin the generative module version. Avoid unexpected strict‑mode switches by specifying the exact Weaviate image tag.
• Keep WEAVIATE_ENABLE_PROMPT_TEMPLATES enabled. Document this requirement in the deployment README.
• Version the template file. Include a template_version field (as recommended in the docs) and bump it on every change; the module will reject stale caches if the version mismatches.

FAQ – Common Follow‑Up Questions

1. Why does the error appear only after upgrading to v1.22?
   Because v1.22 enables Jinja2 strict mode, turning missing variables into hard failures (issue #2319).
2. Can I use arbitrary placeholder names like {{my_input}}?
   Only if you declare them in the module’s additionalVariables mapping. Otherwise the engine will raise KeyError or TemplateRuntimeError.
3. How do I know which variables are available for substitution?
   The documentation lists the built‑in variables (question, context, class properties). Runtime logs at DEBUG level also print the rendering context.
4. What does “InvalidRequestError: Prompt contains unresolved placeholders” mean?
   It indicates that after rendering, the prompt string still contains {{…}} tokens, meaning at least one placeholder was not supplied.
5. Is there a way to fallback to a default value when a variable is missing?
   Yes. Use Jinja2’s default filter in the template, e.g., {{user_query | default('') }}. This works only when strict mode is disabled or the variable is optional.

Related Topic Hub: Vector Databases Troubleshooting Hub