Written by Albert Friedman
on 

Push, Sign, and Ship: GitOps for ML Models with OCI Images, Argo CD, and KServe

GitOps for ML has often stalled on one messy detail: how to version, transport, and roll out big model artifacts safely and repeatably. Two recent moves make this much easier:

Pair those with signature verification and registry-native distribution patterns from Flux/ORAS, and you get a clean, automated pipeline for ML model deployment that is fast, auditable, and boring (in the best way). (fluxcd.io)

This article gives you a practical blueprint: package a model as an OCI image, sign it, declare a KServe InferenceService that points at that image via oci://, and have Argo CD continuously reconcile the manifests (also stored in an OCI registry). We’ll close with a canary rollout pattern using KServe’s built‑in traffic splitting.

What you’ll build

Why OCI for models?

OCI registries are ubiquitous, cache-friendly, and support signatures and referrers. KServe’s Modelcars feature mounts model data from an OCI image, eliminating the “download model over and over” tax. For large models and auto-scaling workloads, that is a big win. (kserve.github.io)

High-level architecture

Step 1 — Package the model as an OCI image and push to your registry

Create a minimal Dockerfile that only bakes in the model payload:

Dockerfile FROM busybox RUN mkdir /models && chmod 775 /models COPY data/ /models/

Where data/ holds your serialized model. Build and push:

build and push

docker build -t REGISTRY/ORG/iris-model:1.0 . docker push REGISTRY/ORG/iris-model:1.0

KServe’s Modelcars expects a normal OCI image; it then makes the model available under /mnt/models in the serving container via a clever shared-process-namespace link. Avoid latest tags; pin concrete tags to leverage node-local caching. (kserve.github.io)

Step 2 — Sign the model image (supply chain hygiene)

Use Cosign to sign the image during CI. Example (key-based; keyless is also supported by many orgs):

generate a keypair once, store the private key securely

cosign generate-key-pair

sign the pushed image

cosign sign –key cosign.key REGISTRY/ORG/iris-model:1.0

You can verify signatures in-cluster using policy engines such as Kyverno or Ratify to block unsigned images. This is a common pattern for enforcing that only signed images run in your clusters. (release-1-10-0.kyverno.io)

Step 3 — Enable KServe Modelcars

Modelcars is off by default. Flip the switch in the inferenceservice-config ConfigMap (namespace kserve) to enable oci:// storage:

enable Modelcars (snippet from KServe docs)

config=$(kubectl get cm -n kserve inferenceservice-config -o jsonpath=’{.data.storageInitializer}’) newValue=$(echo $config | jq -c ‘. + {“enableModelcar”: true, “uidModelcar”: 1010}’) kubectl patch cm -n kserve inferenceservice-config –type=json
-p=”[{"op":"replace","path":"/data/storageInitializer","value":$newValue}]” kubectl delete pod -n kserve -l control-plane=kserve-controller-manager

The docs also include a worked example and explain why tags (not latest) matter for caching. (kserve.github.io)

Step 4 — Declare the InferenceService with an oci:// storageUri

Here’s a minimal example for a scikit-learn model:

apiVersion: serving.kserve.io/v1beta1 kind: InferenceService metadata: name: iris-sklearn spec: predictor: model: modelFormat: name: sklearn storageUri: oci://REGISTRY/ORG/iris-model:1.0

When the pod starts, KServe mounts the model from the image into /mnt/models without a separate download container. (kserve.github.io)

Step 5 — Store your manifests in an OCI registry and let Argo CD sync them

Argo CD now supports OCI as a first-class application source, so you can publish plain Kubernetes YAML as an OCI artifact and point Argo CD at an oci:// repo URL. This is great for air‑gapped environments or where you want “all artifacts in the registry.” (argo-cd.readthedocs.io)

Push the manifests to a registry. You can use tools like ORAS to package and push non-image artifacts:

tar up your manifests and push as an OCI artifact

tar -czf svc.tar.gz inferenceservice.yaml oras push REGISTRY/ORG/iris-manifests:v1 svc.tar.gz

ORAS is designed for pushing and pulling arbitrary artifacts to registries and works well for YAML bundles. (oras.land)

Then, configure Argo CD to track that OCI source:

apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: iris-inference namespace: argocd spec: project: default source: repoURL: oci://REGISTRY/ORG/iris-manifests targetRevision: v1 path: . destination: server: https://kubernetes.default.svc namespace: ml-inference syncPolicy: automated: prune: true selfHeal: true

The Argo CD docs show this pattern: repoURL uses an oci:// scheme, and targetRevision pins a tag or digest. (argo-cd.readthedocs.io)

Note: If you prefer Flux, it has long supported reconciling manifests from OCI and can verify Cosign signatures on OCI artifacts before applying them—a strong control for “config must be signed.” (fluxcd.io)

Step 6 — Gradual rollouts with KServe canaries

To ship a new model version, publish iris-model:1.1 and update storageUri in the InferenceService. Then, add canaryTrafficPercent to shift only a slice of live traffic to the new revision:

apiVersion: serving.kserve.io/v1beta1 kind: InferenceService metadata: name: iris-sklearn spec: predictor: canaryTrafficPercent: 10 model: modelFormat: name: sklearn storageUri: oci://REGISTRY/ORG/iris-model:1.1

KServe tracks the latest good revision and will split traffic accordingly. Increase canaryTrafficPercent in small steps until you reach 100%. If a step fails, KServe can roll traffic back to the previous rolled‑out revision. (kserve.github.io)

Security and policy tips

Operational gotchas

Why this pattern works

Wrap-up

Automating ML model deployment with GitOps is much simpler when models are first-class OCI citizens. Package your model as a small OCI image, sign it, and point KServe at it with an oci:// URL. Store InferenceService manifests in the registry as well and let Argo CD sync them continuously. Add signature verification and canary steps, and you’ve got a fast, compliant, and low-drama path from “new model artifact” to production traffic. (kserve.github.io)

Further reading