Weaviate Similarity Query Returns Empty Results During RAG Inference

Q: 1. Verify that objects exist

curl -s http://localhost:8080/v1/objects?class=DocumentChunkV1 | jq '.objects | length' Expected output: a non‑zero integer (e.g., 1245). If 0, the import failed or the class name is wrong.

Q: 2. Check that vectors are present

curl -s http://localhost:8080/v1/objects/DocumentChunkV1/12345 | jq '.properties["_additional"].vector | length' A missing vector field indicates the vectorizer did not run.

Q: 3. Inspect the class definition

curl -s http://localhost:8080/v1/schema/DocumentChunkV1 | jq '.classes[0].vectorIndexConfig' Sample output:

Q: 4. Review server logs for index refresh errors

journalctl -u weaviate -f | grep -i "refresh" Typical error:

Q: 1. Re‑enable and run the vectorizer (if disabled)

# Before (module disabled) curl -X PATCH http://localhost:8080/v1/modules/text2vec-contextionary \ -H "Content-Type: application/json" \ -d '{"enabled": false}' # After (enable module) curl -X PATCH http://localhost:8080/v1/modules/text2vec-contextionary \ -H "Content-Type: application/json" \ -d '{"enabled": true}' 2. Re‑import missing vectors (if objects lack vectors) python -

Q: 4. Align query distance metric with class configuration

If the class uses cosine, ensure the query does not specify distance (defaults to class metric) or explicitly set it:

In this article

Table of Contents

Problem

During Retrieval‑Augmented Generation (RAG) inference, the Weaviate NearVector similarity query returns an empty result set even though the target documents are present in the vector store. The downstream LLM receives no context, leading to hallucinations or generic answers.

Root Cause

The empty results stem from one or more of the following intertwined issues:

Vector index not synchronized: Documents were bulk‑imported with batch but the index was never rebuilt because autoRefresh was disabled or the refresh call failed.
Incompatible distance metric: The class was defined with cosine distance, but the query used euclidean (default) causing the similarity threshold to be unreachable.
Too‑high certainty or distance filter: The query specified certainty: 0.95 while the vectors of the relevant documents only achieve ~0.78 similarity.
Schema drift: The class name used in the inference code (DocumentChunk) differed in case or spelling from the actual class (DocumentChunkV1), so the query was executed against an empty class.
Missing vectorizer: The text2vec-contextionary module was disabled after a recent upgrade, leaving newly added objects without a vector representation.

Debug

Below is a step‑by‑step debugging workflow that isolates the cause.

1. Verify that objects exist

curl -s http://localhost:8080/v1/objects?class=DocumentChunkV1 | jq '.objects | length'

Expected output: a non‑zero integer (e.g., 1245). If 0, the import failed or the class name is wrong.

2. Check that vectors are present

curl -s http://localhost:8080/v1/objects/DocumentChunkV1/12345 | jq '.properties["_additional"].vector | length'

A missing vector field indicates the vectorizer did not run.

3. Inspect the class definition

curl -s http://localhost:8080/v1/schema/DocumentChunkV1 | jq '.classes[0].vectorIndexConfig'

Sample output:

{
  "distance": "cosine",
  "efConstruction": 128,
  "ef": 64,
  "maxConnections": 64
}

4. Review server logs for index refresh errors

journalctl -u weaviate -f | grep -i "refresh"

Typical error:

2024-09-12T14:32:07.123Z ERROR weaviate.index.refresh Failed to refresh index for class DocumentChunkV1: index not built

5. Reproduce the failing query locally

python - <<'PY'
import weaviate
client = weaviate.Client("http://localhost:8080")
result = client.query.get("DocumentChunkV1", ["content"]).with_near_vector({
    "vector": [0.12, -0.34, 0.56, ...],
    "certainty": 0.95
}).do()
print(result)
PY

Observed output:

{'data': {'Get': {'DocumentChunkV1': []}}}

6. Examine the similarity score of a known matching document

python - <<'PY'
import weaviate, numpy as np
client = weaviate.Client("http://localhost:8080")
vec = client.data_object.get_vector("DocumentChunkV1", "12345")
query_vec = np.array([0.12, -0.34, 0.56, ...])
cos_sim = np.dot(vec, query_vec) / (np.linalg.norm(vec) * np.linalg.norm(query_vec))
print("cosine similarity:", cos_sim)
PY

Typical result: 0.78, which is below the certainty: 0.95 threshold.

Solution

Apply the fixes that correspond to the identified root cause. The most common combination is rebuilding the index and aligning the distance metric and certainty threshold.

1. Re‑enable and run the vectorizer (if disabled)

# Before (module disabled)
curl -X PATCH http://localhost:8080/v1/modules/text2vec-contextionary \
  -H "Content-Type: application/json" \
  -d '{"enabled": false}'

# After (enable module)
curl -X PATCH http://localhost:8080/v1/modules/text2vec-contextionary \
  -H "Content-Type: application/json" \
  -d '{"enabled": true}'

2. Re‑import missing vectors (if objects lack vectors)

python - <<'PY'
import weaviate, json
client = weaviate.Client("http://localhost:8080")
with open("missing_ids.json") as f:
    ids = json.load(f)
for obj_id in ids:
    obj = client.data_object.get_by_id("DocumentChunkV1", obj_id)
    # Force vectorization
    client.batch.add_data_object(
        data_object=obj["properties"],
        class_name="DocumentChunkV1",
        uuid=obj_id,
        vectorize=True
    )
client.batch.create_objects()
PY

3. Rebuild the vector index

# Trigger a manual refresh
curl -X POST http://localhost:8080/v1/schema/DocumentChunkV1/refresh

Verify the log shows:

2024-09-12T14:45:12.876Z INFO weaviate.index.refresh Index refreshed for class DocumentChunkV1

4. Align query distance metric with class configuration

If the class uses cosine, ensure the query does not specify distance (defaults to class metric) or explicitly set it:

client.query.get("DocumentChunkV1", ["content"]).with_near_vector({
    "vector": query_vec,
    "certainty": 0.80   # lower than observed similarity
})

5. Correct class name mismatches

# Before (incorrect class name)
client = weaviate.Client("http://localhost:8080")
client.query.get("DocumentChunk", ["content"]).with_near_vector({...}).do()

# After (exact class name)
client.query.get("DocumentChunkV1", ["content"]).with_near_vector({...}).do()

Verify

Run the same query used in production and confirm that at least one document is returned.

python - <<'PY'
import weaviate
client = weaviate.Client("http://localhost:8080")
result = client.query.get("DocumentChunkV1", ["content"]).with_near_vector({
    "vector": [0.12, -0.34, 0.56, ...],
    "certainty": 0.80
}).do()
print(result)
PY

Expected output (truncated):

{
  "data": {
    "Get": {
      "DocumentChunkV1": [
        {
          "content": "The quick brown fox..."
        },
        {
          "content": "Lorem ipsum dolor..."
        }
      ]
    }
  }
}

Additional validation steps:

Check /_/metrics for weaviate_vector_index_refresh_seconds to ensure the refresh succeeded.
Run a health check endpoint (/v1/.well-known/ready) and confirm a 200 response.
Execute a synthetic RAG request through the inference service and verify the LLM receives the retrieved context.

Prevent

Implement the following guardrails to avoid recurrence:

Automated index refresh: Enable autoRefresh: true in the class config or schedule a nightly /refresh call via a cron job.
Threshold sanity check: Deploy a pre‑flight validation that computes the 95th percentile similarity of a sample query and caps the certainty value accordingly.
Schema consistency CI: Add a CI step that fetches the live schema and asserts that class names used in code match exactly (case‑sensitive).
Vectorizer health monitor: Export a Prometheus metric from the text2vec-contextionary module indicating the number of vectors generated per minute; alert if it drops to zero.
Batch import verification: After any bulk load, run a quick count query (GET /v1/objects?class=…&limit=0) and compare the result with the expected document count.

Related Topic Hub: Vector Databases Troubleshooting Hub

FAQ

Why does the query work in a local dev environment but return empty results in production?
Production often runs with autoRefresh: false for performance, so the index is stale after a bulk import. Dev clusters usually have auto‑refresh enabled, keeping the index up‑to‑date.
Can a mismatched distance metric cause zero results even if the vectors are correct?
Yes. If the class is indexed with cosine but the query uses the default euclidean, the similarity scores are computed in incompatible spaces, making the certainty threshold unreachable.
How can I determine the appropriate certainty value for my dataset?
Sample a set of known relevant queries, compute their cosine similarity, and set certainty to a value slightly below the 5th percentile of those scores (e.g., 0.80 for a 0.78 median).
Is there a way to force Weaviate to recompute vectors for existing objects?
Yes. Use the /v1/objects/{class}/{id}/vectorize endpoint or re‑batch the objects with vectorize: true. This triggers the active vectorizer module to generate fresh vectors.
What should I monitor to catch this issue before it impacts users?
Set alerts on:
- Drop in weaviate_objects_total vs. expected count.
- Zero increments in weaviate_vectorizer_vectors_generated_total over a 5‑minute window.
- High latency or error rate on /v1/query with NearVector filters.