Written by Nelson Koning
on 

GitOps, OCI artifacts and signed models: a practical path to automated ML deployments

Shipping ML models to production has started to look less like a one-off “copy the file and hope” task and more like releasing a piece of software: you want reproducibility, auditable history, secure provenance, and a predictable pipeline that can be recovered when it misbehaves. GitOps gives us that predictable pipeline by treating Git as the single source of truth, and recent work in packaging models as OCI artifacts — plus signing them with Sigstore — makes it possible to run end-to-end automated model deployments with much stronger trust guarantees than before.

This article explains a practical pattern: package your model as an OCI artifact, sign and attest it, store the model and metadata in a registry, and let a GitOps operator (Flux or Argo) reconcile the deployment to a serving layer such as KServe. Along the way I’ll highlight where security and auditability fit, and give small examples of what the pieces look like in YAML.

Why OCI artifacts matter for models

Putting trust on the model: signing and validation A model in a registry is only as good as your trust in it. Sigstore’s ecosystem — and specifically the work around a Model Validation Operator — makes it possible to sign model artifacts and validate those signatures as part of the deployment chain. That means you can enforce policy such as “only signed models from the model-registry team can be deployed to production clusters,” or require attestations that tests passed before deployment. This avoids a common headache where a model binary shows up in production with no clear provenance. (blog.sigstore.dev)

How a GitOps-driven model deployment pipeline looks (high level)

  1. Training and packaging (CI stage)
    • Train the model in your usual environment.
    • Package the model into an OCI artifact (a layer or an image), include metadata (framework, version, training commit hash, metrics) and push to a registry.
    • Sign the artifact (cosign / Sigstore) and optionally attach an attestation that unit/integration/validation tests passed.
  2. Declarative reference in Git
    • A manifest in Git — for example a KServe InferenceService manifest or a small YAML that references the OCI artifact digest — points to the specific model artifact and desired serving config (resources, canary weights, autoscaling hints).
  3. GitOps reconciliation
    • Flux or ArgoCD watches the Git repo and reconciles the cluster state. Where Flux has built-in support for OCI sources and composition, ArgoCD can be used with similar patterns to keep serving manifests in sync. The controller ensures the cluster matches the manifest in Git. (fluxcd.io)
  4. Admission and validation
    • A cluster-side validation step (for example a Sigstore-based operator) verifies the artifact signature and attestation before allowing the InferenceService to consume the model. If validation fails, the deployment is blocked and the Git history documents the attempted change. (blog.sigstore.dev)

KServe as the serving layer KServe is a Kubernetes-native inference solution that maps well to the GitOps model: model-serving resources are expressed as CRDs (e.g., InferenceService) and can be fully declared in Git. That makes it straightforward for GitOps controllers to create, update, or roll back serving endpoints based on the model reference in your repo. KServe also continues to evolve toward LLM runtimes and model caching improvements, which fits teams deploying heavier inference workloads. (kserve.github.io)

A tiny example: what the pieces might look like

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: my-vision-model
spec:
  predictor:
    model:
      storageUri: oci://registry.example.com/models/my-vision-model@sha256:<digest>
      # metadata/annotations would include signature/attestation references

This manifest is just a pointer; the heavy lifting (pulling the model, validating the signature, provisioning resources) happens in-cluster under the watchful eye of GitOps.

A defensive, pragmatic stance

Why this combination is appealing

Real-world adoption and ecosystem momentum Flux and other GitOps tools have been investing in OCI support and improvements for composing distributed sources, which makes the pattern above more practical for teams that want to treat models like first-class artifacts in a registry rather than opaque files. At the same time, open-source efforts around model signing and validation are explicitly solving the trust problem for ML artifacts — a sign that the ecosystem is maturing toward production-grade practices. (fluxcd.io)

Closing analogy Think of a model release as publishing a new record. The OCI registry is the record store, Sigstore is the stamp of authenticity and provenance in the sleeve, Git is the catalogue that lists which records should be on the store shelf, and the GitOps controller is the clerk who keeps the shelf matching the catalogue. When everyone agrees on that rhythm, the store stays tidy, customers get predictable behavior, and when a bad pressing slips through you can trace it back to the pressing plant and pull it off the shelf quickly.

Caveats and trade-offs

References