on
Making Vault + Kubernetes CSI safer after the CSI service-account token changes
Kubernetes’ handling of secrets and service account tokens has continued to evolve, and that evolution matters if you use HashiCorp Vault to deliver secrets into pods. This article walks through why the recent CSI-related changes matter, what they mean for Vault deployments, and a practical set of best practices to keep secrets short‑lived, least‑privileged, and hard to exfiltrate when using the Vault CSI or Secrets Store integration.
Why this matters right now
- Kubernetes v1.35 introduced a refinement for CSI drivers that use service account tokens: a new, safer way to pass tokens to CSI drivers using the NodePublishVolumeRequest “secrets” field and a CSIDriver flag (serviceAccountTokenInSecrets) to opt into the behavior. This helps avoid exposing tokens via files on disk or other less controlled channels. (kubernetes.io)
- HashiCorp provides a Vault CSI/Secrets Operator integration that can inject secrets into pods as protected mounts and supports patterns for dynamic credentials and least‑privilege access. Adopting the safer Kubernetes behavior together with Vault’s dynamic secrets model materially reduces the blast radius of compromised workloads. (developer.hashicorp.com)
What changed in Kubernetes (short version)
- Historically, some CSI drivers obtained service account tokens via projected service account files or other mechanisms that could leave tokens exposed on the node filesystem or make it hard to control token lifetimes.
- Kubernetes v1.35 formalized a safer path: drivers that need a service account token can request it through the CSI “secrets” field during NodePublishVolume, and cluster admins opt a driver into this mode by setting serviceAccountTokenInSecrets: true on the CSIDriver object. This reduces scope for accidental token exposure and gives operators clearer upgrade/rollout steps. (kubernetes.io)
Why that matters for Vault deployments
- Vault integrations that mount secrets into pods via CSI (or via the Secrets Store CSI Driver) frequently require some authentication material for the driver to fetch secrets on behalf of the pod. If that credential is resident somewhere predictable and long‑lived, an attacker who compromises the node or container may get a path to Vault.
- Moving to the new Kubernetes CSI token flow narrows the window of exposure and enables drivers to rely on well-scoped, short‑lived tokens issued per-volume mount, rather than long‑lived service account keys living on disk. The Vault side remains responsible for applying least‑privilege policies and for issuing short TTLs for secrets. (developer.hashicorp.com)
Recommended high‑level architecture
- Use workload identity / bound service accounts for pod→Vault authentication: keep a binding between a Kubernetes service account and a Vault role (or OIDC client) so a token presented by the pod can only request the secrets it needs. This avoids static credentials in container images or environment variables. (developer.hashicorp.com)
- Prefer the Vault CSI provider (Secrets Store CSI) over writing secrets into Kubernetes Secret objects when you need stronger protection: the CSI approach can mount secrets directly into the container’s filesystem or provide them as memory-backed files without creating cluster-wide secrets that are discoverable via the Kubernetes API. (developer.hashicorp.com)
- Issue short‑lived, dynamic secrets from Vault where possible (databases, cloud IAM tokens, certificates). Short TTLs and automatic revocation reduce exposure if a secret is leaked. Vault’s secrets engines are designed for this use case. (developer.hashicorp.com)
Concrete checklist before you deploy or upgrade
- Audit how your CSI driver authenticates
- Does it use projected service account tokens, a node-level credential, or an in-cluster static secret? If it’s not using short-lived, bound tokens, plan to change that.
- If you run your own CSI driver for Vault, prepare to set serviceAccountTokenInSecrets and update to a driver version that supports the new flow. (kubernetes.io)
- Map Vault roles to Kubernetes service accounts (workload identity)
- Create Vault roles with policies that grant the minimum capabilities (read specific paths, create ephemeral DB creds, or sign certs).
- Bind each role to a specific Kubernetes service account name and namespace so tokens from other pods can’t impersonate the role. (developer.hashicorp.com)
- Prefer dynamic secrets and enforce short TTLs
- For databases, use Vault’s database secrets engine to generate credentials on demand with automatic revocation.
- For cloud APIs, prefer generated IAM tokens/keys with tight TTLs instead of long-lived keys in pods. (developer.hashicorp.com)
- Use CSI mounts instead of Kubernetes Secrets where secrecy matters
- CSI mounts avoid placing secrets in the Kubernetes API server as Secret objects, which are accessible to anyone with API read permissions in the namespace.
- Ensure your CSI driver is upgraded to support the serviceAccountTokenInSecrets flow so nodal exposure is minimized. (developer.hashicorp.com)
- Enforce granular Vault policies and audit logging
- Implement policies that only allow the required operations for each role.
- Enable audit logging on Vault and collect those logs centrally to detect anomalous access patterns quickly. (developer.hashicorp.com)
- Consider Pod Security and Node hardening
- Limit hostPath mounts and other privileges that could let a compromised container read node-level files.
- Keep node images patched and minimize the set of system tools available to containers.
Example: what an operator change looks like (conceptual)
- Update your CSIDriver manifest to explicitly opt into the new token-in-secrets behavior:
apiVersion: storage.k8s.io/v1
kind: CSIDriver
metadata:
name: vault.csi.hashicorp.com
spec:
attachRequired: true
podInfoOnMount: true
serviceAccountTokenInSecrets: true
- Use a SecretsProviderClass (or the Vault CSI configuration) that instructs the driver how to exchange the provided token for Vault secrets and which Vault role/policy to use. The exact driver config depends on the provider you run; consult the Vault CSI docs for the provider-specific fields. (developer.hashicorp.com)
Operational notes and gotchas
- Rolling upgrades: change driver behavior in two steps—deploy a new driver that understands both modes with fallback logic, then update the CSIDriver object to enable serviceAccountTokenInSecrets. This avoids downtime during rollout. The Kubernetes project and driver maintainers recommend this pattern for a smooth transition. (kubernetes.io)
- Don’t confuse CSI mounts with in‑pod client-side caching: even when secrets are mounted, applications should treat them as ephemeral and be resilient to secret rotation/revocation.
- RBAC considerations: even if CSI avoids writing Kubernetes Secrets, cluster role bindings that allow listing pods, service accounts, or CSIDriver objects can still give attackers useful information—apply least privilege at the Kubernetes RBAC level as well. (developer.hashicorp.com)
Why short‑lived credentials remain the single best mitigation Short TTLs and dynamic generation change the economics for attackers. A leaked database credential that auto-expires reduces the window for exploitation; a certificate signed for minutes is far less useful than a static key. Vault was built to enable this pattern and the Vault team’s guidance emphasizes dynamic credentials as a priority to prevent leaked secrets. Pair that with the safer CSI token handling in Kubernetes and you significantly reduce both how and how long credentials can be used if compromised. (hashicorp.com)
Summary (practical takeaways)
- Upgrade and configure your CSI driver to use the Kubernetes v1.35+ serviceAccountTokenInSecrets flow where possible—this reduces token exposure during volume publish. (kubernetes.io)
- Use Vault’s CSI/Secrets Store integration to mount secrets into pods instead of writing them as Kubernetes Secrets when you need higher protection. (developer.hashicorp.com)
- Design Vault roles and policies around workload identity, and favor dynamic, short‑lived credentials so leaked secrets have minimal value. (developer.hashicorp.com)
Adopting the above patterns aligns platform and application teams: Kubernetes gives a safer mechanism to hand credentials to node components, and Vault focuses on delivering least‑privilege, short‑lived secrets. The combination reduces both the surface area and the lifetime of exposed secrets—two of the most effective controls in real‑world incident containment.