Written by Albert Friedman
on 

Managing Dashboards with GitOps: an intro to observability as code

Observability as code treats dashboards, alerts, data-source configs, and other observability artifacts as versioned, reviewable, and automatable code. For teams adopting GitOps, dashboards become declarative assets in a Git repository, synchronized to Grafana or another visualization platform by a continuous delivery system (Argo CD, Flux, or a provider-specific Git sync). The result is reproducible dashboards, audit trails for changes, and safer rollouts of visualization changes across environments. (grafana.com)

This article explains the core ideas, common implementation patterns, and practical gotchas for managing dashboards with GitOps. Examples use Grafana terminology because its “observability as code” features and ecosystem shine for this pattern, but the patterns apply to other dashboarding systems as well. (grafana.com)

Why treat dashboards as code?

Grafana Labs has been explicitly building features and workflows for observability-as-code—integrating API-driven provisioning, Git Sync, and resource schemas to make dashboards first-class as-code resources. Recent product work expanded APIs and Terraform resources to support schema-based validation and programmatic generation of dashboards. (grafana.com)

Core concepts (short)

Common GitOps patterns for dashboards

  1. ConfigMap provisioning (Kubernetes + GitOps)
    • Store dashboard JSON files in a Git repo.
    • A CI process or GitOps controller (Argo CD/Flux) applies ConfigMaps that contain the JSON to the cluster.
    • Grafana reads those ConfigMaps via the provisioning mechanism or via a sidecar that writes them to the filesystem for provisioning.
    • Good for small-to-medium setups because it’s simple and Kubernetes-native. (grafana.com)
  2. Grafana Operator + CRDs (Kubernetes + GitOps)
    • The Grafana Operator exposes CRDs (custom resources) for dashboards and other Grafana resources.
    • GitOps tools reconcile YAML CRs to the cluster, and the operator pushes those resources into Grafana, keeping the two in sync.
    • Scales well and models Grafana resources as Kubernetes-native objects. (grafana.com)
  3. Git Sync (Grafana built-in)
    • Grafana’s Git Sync can pull a repository of dashboard JSON files directly into Grafana; dashboards are provisioned from that folder.
    • This removes the need for Kubernetes manifests and works well when teams prefer managing dashboards directly in a repo without an intermediate Operator. (grafana.com)
  4. Terraform / SDK-driven generation
    • For generated or templated dashboards, an SDK or generator (jsonnet, grafonnet, or Grafana’s Foundation SDK) produces JSON that CI commits or pushes to the repo or applies directly via APIs or Terraform provider.
    • Useful for reproducible, parameterized dashboards across many teams. (grafana.com)

Minimal repo layout (example)

A simple Git repo layout for a GitOps-driven Grafana provisioning approach:

This keeps environment variants and provisioning config explicit and discoverable. Grafana renders README files inline for provisioned folders, which helps teams discover what each folder contains. (grafana.com)

Example: provisioning provider snippet

Grafana’s provisioning allows pointing to a filesystem path where dashboard JSON files live. In a Kubernetes setup that writes files into /var/lib/grafana/provisioning/dashboards, a provider might look like this:

apiVersion: 1
providers:
  - name: 'gitops-dashboards'
    orgId: 1
    folder: 'GitOps'
    type: file
    options:
      path: /var/lib/grafana/provisioning/dashboards/gitops

After provisioning, Grafana watches the filesystem and updates dashboards when JSON files change. The provisioning system strips IDs and manages version handling to avoid accidental overwrites. (grafana.com)

GitOps example: Argo CD + Grafana Operator (conceptual)

Grafana documentation provides step-by-step examples for this pattern and sample manifests to check sync status in Argo CD. This flow gives strong reconciliation guarantees and fits well with Kubernetes-native delivery. (grafana.com)

Practical considerations and gotchas

Testing and review workflow

Typical teams use pull requests for dashboard changes. Useful checks in CI:

Grafana’s move toward programmatic APIs and Terraform resources makes incorporating validation into CI/CD pipelines easier and more reliable. (grafana.com)

When to choose which pattern

Short recipe: what a minimal GitOps dashboard change looks like (conceptual)

That pipeline replaces manual dashboard edits and gives both traceability and a rollback path through Git history. (grafana.com)

Final notes on maturity and toollandscape

Grafana and the broader observability ecosystem have been investing in “dashboards as code” primitives—APIs, providers, and SDKs—that move the industry toward treating observability artifacts like other infrastructure. Official guidance, tutorials, and new Terraform resources make it easier to validate and distribute dashboards from code. For teams adopting GitOps, the key is to pick a pattern that matches operational scale and the team’s current platform (Kubernetes vs. non-Kubernetes) and to bake validation and review into the CI pipeline. (grafana.com)

Observability as code does more than protect dashboards from accidental edits: it makes dashboards reproducible infrastructure, opens observability to collaboration, and makes visualization changes traceable—core capabilities for reliable, scalable monitoring and SRE practices. (grafana.com)

References (examples)