A person holding a smart phone in their hand

Mistral AI CRD rejected missing spec.modelVersion during Knative deployment

by

Problem

When deploying a Mistral AI inference job through a Knative Eventing pipeline, the custom resource (CR) creation fails with a validation error from the Kubernetes API server:


error: admission webhook "mistral-ai-validation.k8s.io" denied the request: 
spec.modelVersion: Required value: missing required field "spec.modelVersion"
spec.resources.limits.cpu: Invalid value: "500m": must be an integer
spec.parameters: Additional property foo is not allowed

The CRD for the Mistral AI inference controller rejects any custom resource that does not conform to the declared OpenAPI v3 schema. As a result, the Knative Trigger never creates the MistralInferenceJob objects, breaking the end‑to‑end event‑driven inference pipeline.

Root Cause

  • Missing required field: The CRD schema marks spec.modelVersion as required. The manifest generated by the CI pipeline omitted this field, causing immediate rejection.
  • Type mismatch for CPU limits: The schema defines spec.resources.limits.cpu as an integer (representing millicores). The manifest used the string "500m", which is valid for a Quantity type but not for the integer type enforced by the CRD.
  • Disallowed additionalProperties: The spec.parameters map is declared with additionalProperties: false. The manifest included a user‑defined key foo, which the schema rejects.

These validation failures stem from a mismatch between the CRD’s OpenAPI schema (generated by the Mistral AI controller) and the manifest produced by the Knative event source.

Debug

1. Inspect the CRD schema

kubectl get crd mistralinferencejobs.mistral.ai -o yaml | \
  yq '.spec.validation.openAPIV3Schema.properties.spec'

Key excerpts:


properties:
  modelVersion:
    type: string
  resources:
    type: object
    properties:
      limits:
        type: object
        properties:
          cpu:
            type: integer   # <-- expects integer millicores
  parameters:
    type: object
    additionalProperties: false
required:
- modelVersion

2. Reproduce the failure

kubectl apply -f examples/mistral-inference-job.yaml
Error from server (BadRequest): error when creating "examples/mistral-inference-job.yaml":
admission webhook "mistral-ai-validation.k8s.io" denied the request:
spec.modelVersion: Required value: missing required field "spec.modelVersion"
spec.resources.limits.cpu: Invalid value: "500m": must be an integer
spec.parameters: Additional property foo is not allowed

3. Review Knative Trigger logs

kubectl logs -l app=eventing-controller -n knative-eventing -c controller | \
  grep "MistralInferenceJob"

No additional errors appear because the controller never receives a valid CR; the failure occurs at API‑server admission time.

4. Verify the API‑server audit log (optional)

journalctl -u kube-apiserver | grep "admission webhook" | tail -n 5

Solution

Option A – Fix the custom resource manifest

Update the manifest to satisfy the schema:

Before (invalid)

apiVersion: mistral.ai/v1
kind: MistralInferenceJob
metadata:
  name: sentiment-analysis
spec:
  # modelVersion omitted
  resources:
    limits:
      cpu: "500m"          # string, not integer
  parameters:
    foo: "bar"            # disallowed extra key
    temperature: 0.7

After (valid)

apiVersion: mistral.ai/v1
kind: MistralInferenceJob
metadata:
  name: sentiment-analysis
spec:
  modelVersion: "v1.2.3"
  resources:
    limits:
      cpu: 500            # integer millicores
  parameters:
    temperature: 0.7

Explanation:

  • modelVersion is now explicitly set.
  • cpu uses an integer (500 millicores) as required.
  • Only allowed keys remain in parameters.

Option B – Adjust the CRD schema (if the controller should accept string quantities)

If the controller’s intent is to allow standard Kubernetes Quantity strings, modify the CRD to use type: string and a pattern that matches the quantity format.

Patch the CRD

kubectl patch crd mistralinferencejobs.mistral.ai \
  --type='json' -p='[
  {"op":"replace","path":"/spec/validation/openAPIV3Schema/properties/spec/properties/resources/properties/limits/properties/cpu/type","value":"string"},
  {"op":"add","path":"/spec/validation/openAPIV3Schema/properties/spec/properties/resources/properties/limits/properties/cpu/pattern","value":"^[0-9]+m?$"}
]'

After the patch, the original manifest with "500m" will be accepted, but you still need to provide modelVersion and remove disallowed parameters.

Verify

  1. Apply the corrected manifest:
kubectl apply -f corrected/mistral-inference-job.yaml

Expected output:

mistralinferencejob.mistral.ai/sentiment-analysis created
  1. Confirm the resource reaches Ready state:
kubectl get mistralinferencejob sentiment-analysis -o yaml | \
  yq '.status.conditions[] | select(.type=="Ready")'

Sample output:

- type: Ready
  status: "True"
  reason: InferenceJobCreated
  message: Inference job successfully scheduled
  1. Check that the Knative Trigger has delivered the event:
kubectl get broker default -o yaml | yq '.status' | grep -i "eventsReceived"
  1. Inspect controller logs for the newly created job:
kubectl logs -l app=mistral-inference-controller -n mistral-system | \
  grep "sentiment-analysis"

Log snippet confirming processing:

2026-08-24T12:34:56Z INFO  controller: Created inference pod for job sentiment-analysis (modelVersion=v1.2.3)

Prevent

  • Schema‑driven CI validation: Add a step that runs kubeval or kubectl apply --dry-run=client against the generated CRD schema before merging.
  • Helm chart defaults: Define modelVersion as a required value in values.yaml with a sensible default.
  • OpenAPI linting: Use openapi‑validator to ensure that spec.resources.limits.cpu is declared with the correct type.
  • Strict parameter handling: If arbitrary parameters are needed, change additionalProperties to true or define an explicit map schema.
  • Automated tests: Include an integration test that creates a minimal MistralInferenceJob and asserts successful reconciliation.

Related Topic Hub: LLM Systems Troubleshooting Hub

FAQ

  1. Why does the API server reject cpu: "500m" while a normal Pod spec accepts it?
    The Mistral AI CRD explicitly defines cpu as an integer. Kubernetes core resources use the Quantity type, but custom resources must follow the schema the controller author provides. Align the schema with Quantity or use integer millicores.
  2. Can I keep extra keys in spec.parameters without editing the CRD?
    No. The CRD sets additionalProperties: false. Either remove the extra keys or modify the CRD to allow a free‑form map (e.g., type: object, additionalProperties: {type: string}).
  3. Is modelVersion always required?
    Yes, the controller uses it to select the correct model artifact. If you need a default, set a default value in the CRD’s default field or provide it via a Helm value.
  4. How do I discover the exact schema a CRD expects?
    Run kubectl get crd <name> -o jsonpath='{.spec.validation.openAPIV3Schema}' or use kubectl explain <resource> --recursive to view the OpenAPI definition.
  5. Will upgrading the Mistral AI controller automatically fix the schema?
    Only if the new controller version updates the CRD to match the intended manifest format. Verify the CRD after any upgrade; do not assume backward compatibility.