Mistral AI CRD validation error missing spec fields during event trigger

Expected error:

error: error validating "payload.yaml": error validating data: ValidationError(ModelDeployment.spec.replicas): invalid type for v1beta1.IntOrString: got string, expected int; ValidationError(ModelDeployment.spec.resources.cpu): invalid type for v1beta1.IntOrString: got string, expected int; ValidationError(ModelDeployment.spec.resources.memory): Invalid value: "2GB": must be a quantity (e.g., "2Gi")

4. Capture the raw event payload as it arrives at the router (e.g., CloudWatch logs, Kafka consumer logs):

2026-08-28T12:03:58Z INFO Received event: {"modelName":"my-model","modelVersion":"1.0","replicas":"2","resources":{"cpu":"1","memory":"2GB"}}

5. Check for race conditions by enabling the operator’s --debug flag and watching the order of CREATE requests in the API server audit log.

kubectl get --raw "/apis/authorization.k8s.io/v1/selfsubjectaccessreviews" -H "Accept: application/json"

Resolution – Correcting the CRD Payload

Two complementary fixes are required: ensure required fields are always present, and enforce correct data types before the object reaches the API server.

1. Enforce a strict JSON schema in the event producer

Update the Lambda (or equivalent) to marshal numeric fields as integers and to inject defaults for missing fields.

// Example Node.js Lambda snippet
const payload = {
  modelName: event.modelName || "default-model",   // guarantee presence
  modelVersion: event.modelVersion || "latest",
  replicas: Number(event.replicas) || 1,          // cast to int
  resources: {
    cpu: Number(event.resources?.cpu) || 1,
    memory: event.resources?.memory || "2Gi"     // use Kubernetes quantity format
  }
};

await k8sApi.createNamespacedCustomObject(
  "mistral.ai",
  "v1",
  "prod",
  "modeldeployments",
  payload
);

2. Add a mutating admission webhook to coerce types and fill defaults

If the event source cannot be changed (e.g., third‑party system), deploy a lightweight mutating webhook that:

• Converts stringified numbers to integers.
• Rewrites memory strings to valid Quantity format.
• Adds missing mandatory fields with sensible defaults.

Sample webhook configuration (YAML):

apiVersion: admissionregistration.k8s.io/v1
kind: MutatingWebhookConfiguration
metadata:
  name: mistral-modeldeployment-mutate
webhooks:
  - name: mutate.mistral.ai
    admissionReviewVersions: ["v1"]
    sideEffects: None
    clientConfig:
      service:
        name: modeldeployment-mutate-svc
        namespace: mistral-system
        path: /mutate
    rules:
      - apiGroups: ["mistral.ai"]
        apiVersions: ["v1"]
        operations: ["CREATE","UPDATE"]
        resources: ["modeldeployments"]

3. Update the CRD to include default values where appropriate

Starting with Mistral AI Operator v1.4, the CRD supports x-kubernetes-defaults. Adding defaults reduces the chance of missing fields.

spec:
  modelName:
    type: string
  modelVersion:
    type: string
    default: "latest"
  replicas:
    type: integer
    default: 1
  resources:
    type: object
    properties:
      cpu:
        type: integer
        default: 1
      memory:
        type: string
        pattern: "^[0-9]+(Mi|Gi)$"
        default: "2Gi"

Before / After Comparison

Before (failing payload):

apiVersion: mistral.ai/v1
kind: ModelDeployment
metadata:
  name: my-model-deployment
spec:
  modelVersion: "1.0"
  replicas: "2"
  resources:
    cpu: "1"
    memory: "2GB"

After (validated and mutated payload):

apiVersion: mistral.ai/v1
kind: ModelDeployment
metadata:
  name: my-model-deployment
spec:
  modelName: "my-model"          # injected default
  modelVersion: "1.0"
  replicas: 2                    # integer
  resources:
    cpu: 1                       # integer
    memory: "2Gi"                # valid quantity

Verification – Confirming the Fix

1. Re‑trigger the event and watch the operator logs for a successful admission:

2026-08-28T12:10:45Z INFO admission webhook "mistral-operator.kb.io" allowed the request: ModelDeployment my-model-deployment created

2. Check the resource status:

kubectl get modeldeployment my-model-deployment -n prod -o jsonpath="{.status.phase}"

Expected output: Ready

3. Validate that the spec fields have the correct types:

kubectl get modeldeployment my-model-deployment -n prod -o yaml | grep -A5 "spec:"

Result should show integers for replicas and cpu, and a quantity string for memory.

Prevention – Operational Guardrails

• Schema validation in CI: Use kubectl apply --dry-run=client on generated manifests before they are committed.
• Event payload contracts: Publish an OpenAPI spec for the event schema and enforce it with API‑gateway validation (e.g., AWS API Gateway, Kong).
• Admission webhook testing: Deploy a staging environment with the same mutating webhook and run integration tests that simulate high‑concurrency event bursts.
• Monitoring & alerts: Create Prometheus alerts on admission_webhook_rejection_total{webhook="mistral-operator.kb.io"} to catch spikes in validation failures.
• Defaulting in the CRD: Keep the CRD version up‑to‑date and enable preserveUnknownFields: false to avoid silent schema drift.

FAQ – Related Questions

1. Why does the webhook reject spec.replicas when I pass “2” as a string?
   The CRD schema defines replicas as an integer. JSON strings are not coerced by the Kubernetes API server, so the admission webhook reports “Invalid type: string (expected int)”. Cast the value to an integer before creating the resource or use a mutating webhook to perform the conversion.
2. Can I rely on the operator to fill missing fields automatically?
   Only fields that have a default defined in the CRD are auto‑populated. Required fields without defaults (e.g., modelName) must be supplied by the event producer; otherwise the request is denied.
3. How do I express memory limits correctly for Mistral AI?
   The spec.resources.memory field expects a Kubernetes Quantity (e.g., 2Gi, 512Mi). Values like “2GB” are rejected because they do not match the quantity pattern defined in the CRD.
4. My event router sometimes sends numeric fields as strings only under high load. What should I do?
   Implement a mutating admission webhook that normalizes the payload, or adjust the router’s serialization library to emit native JSON numbers. Adding a JSON schema validation step before the router forwards the payload can also catch malformed events early.
5. Is there a way to see which fields are required without reading the full CRD documentation?
   Run kubectl explain modeldeployment.spec --recursive. The output lists each field, its type, and whether it is required (marked with “*”).

Related Topic Hub: LLM Systems Troubleshooting Hub