on
Ephemeral secrets and state encryption: a practical intro to OpenTofu’s safer secret handling
OpenTofu has quickly evolved beyond being a drop-in Terraform fork — it’s adding features that rethink how secret and transient data are handled in infrastructure-as-code. Two recent additions are especially important for anyone managing secrets with Terraform-style workflows: built-in state encryption (available since early releases) and a new “ephemeral” model that avoids persisting confidential values to state at all (available in 1.11 beta/nightlies). This article explains what these features do, when to use each, and practical patterns and migration tips to get started safely. (linuxfoundation.org)
Why caring about state and secrets matters
- Your state file is the single source of truth for what OpenTofu manages. It often contains IPs, IDs — and sometimes credentials and keys. Without protections, that file is a sensitive target.
- Historically, teams relied on marking values “sensitive” (hides them from CLI output) and on backend protections (remote storage, IAM, encryption-at-rest). Those help but don’t stop plaintext values from being stored in state. (opentofu.org)
OpenTofu’s two complementary approaches
- State encryption — encrypt the entire state (and plan) so stored values are protected at rest and in transit. This reduces the blast radius if a backend leak occurs. OpenTofu added built-in state encryption (AES-GCM, multiple key providers) in the v1.7 series. (opentofu.org)
- Ephemeral values/resources (beta in 1.11 / available in nightlies) — avoid persisting secret or transient values at all. Ephemeral resources are opened at runtime, used during an operation, and never written to plan or state. This is ideal for short-lived credentials, generated passwords, or temporary tunnels. Use ephemeral first when possible; fall back to encrypted state when persistence is required. (opentofu.org)
How ephemeral values work (conceptual)
- An ephemeral block requests a value from a provider at runtime (for example, a one-time secret from a KMS or a temporary credential).
- The attributes returned are marked ephemeral and can only be used in specific contexts (locals, providers, provisioners, connection blocks, write-only attributes, and ephemeral outputs). Any attempt to use them in a place that would require persistent storage will error. (opentofu.org)
Simple ephemeral example (This is a compact illustration based on current OpenTofu docs — don’t run this in production without reading provider docs and release notes.)
ephemeral "aws_secretsmanager_random_password" "pw" {
# provider-specific schema / args, if applicable
}
resource "kubernetes_secret_v1" "admin" {
metadata {
name = "admin"
namespace = "my-app"
}
# write-only attribute: send password into the secret, but don't store it in state
data_wo = {
username = "admin"
password = ephemeral.aws_secretsmanager_random_password.pw.random_password
}
data_wo_revision = 1
type = "kubernetes.io/basic-auth"
}
Notes:
- data_wo is a write-only attribute: the secret is sent to Kubernetes but not retained in OpenTofu’s state or plan outputs. Ephemeral values are only available during the command run that creates or updates the secret. (opentofu.org)
State encryption: when you still need persistence There are valid reasons to store secrets in state (e.g., long-lived credentials that must be tracked, or outputs that other stacks read via remote state). OpenTofu’s state encryption protects those values by encrypting the plan/state blobs client-side before they are sent to the backend. Key takeaways:
- OpenTofu supports multiple encryption methods (AES-GCM) and many key providers (PBKDF2, AWS KMS, GCP KMS, external commands). You can chain key providers for layered security. (opentofu.org)
- Encryption is configured in Terraform/OpenTofu language blocks (not as a backend toggle), which makes it portable across different backends. Example (simplified):
terraform {
encryption {
key_provider "external" "my_k" {
command = ["op", "read", "op://my/statekey"]
}
method "aes_gcm" "state" {
keys = key_provider.external.my_k
}
state {
method = method.aes_gcm.state
}
}
}
- Use external key providers to integrate with your existing KMS or secret manager so actual encryption keys aren’t hardcoded into configuration. (opentofu.org)
Practical patterns — when to choose ephemeral vs encrypted state
- Use ephemeral values when:
- The secret is short-lived (one-time password, ephemeral API token, temporary SSH tunnel).
- You can generate or fetch the secret at apply-time and you do not need to reference it later from state.
- You want to reduce the surface area of sensitive data stored by OpenTofu. (opentofu.org)
- Use state encryption when:
- Values must be stored and later read (e.g., outputs consumed by another stack via terraform_remote_state).
- Your provider or workflow cannot yet support ephemerality (ephemeral support requires provider cooperation and may not be immediately available for all resources). (opentofu.org)
- Combine both:
- Generate a secret ephemeral, write it to the external system (e.g., Vault or Secrets Manager), then store only a non-sensitive reference or resource ID in the state (and encrypt the state as an additional safeguard).
Limitations and cautions
- Ephemeral relies on provider support. Not all providers/resources expose ephemeral interfaces immediately — check provider docs or schema. If a provider doesn’t support ephemeral semantics, you can’t simply “make” its fields ephemeral. (opentofu.org)
- Ephemeral values are not available later. If you need to rotate or retrieve that same secret later via the state, ephemeral is the wrong choice. Plan your lifecycle: rotate in the secret store (Vault, Secrets Manager), not by reading it from state. (opentofu.org)
- Ephemeral features (as of writing) reached beta in 1.11. Beta/nightly features should be tested in non-production first. The project explicitely warns against running prereleases in production. (opentofu.org)
Migration and operational tips (safe steps)
- Back up state before you touch anything. Always. Use S3 versioning, DB snapshots, or copy local tfstate files. If something goes wrong, you’ll want to revert. The OpenTofu migration guides emphasize backup and careful testing. (opentofu.org)
- Try the ephemeral path in a disposable environment:
- Build a small test module that generates a secret ephemeral, writes it into a secrets backend (Vault/SecretsManager/Kubernetes), and does not persist it in state.
- Verify you can rotate and recover via the backend, not via OpenTofu state. (opentofu.org)
- If you must store values, enable state encryption and integrate an external key provider (KMS or command). Test key rotation and decryption in CI and long-lived workers. (opentofu.org)
- CI/CD considerations:
- Ensure transient ephemeral operations in CI have access to the providers they need (e.g., credentials to fetch ephemeral tokens), but don’t leak them into logs. Use masked variables and restrict logs.
- For global provider caches and parallel runs, OpenTofu 1.10+ improved provider cache locking to be safe in CI. If you run many parallel inits, use the global provider cache and the filesystem locking options. (opentofu.org)
- Read provider docs — provider authors decide whether to support ephemeral semantics. If your team depends on ephemeral, engage provider maintainers or submit PRs. (opentofu.org)
A short checklist before you go to production
- Back up current state and commit all config. (opentofu.org)
- Test ephemeral flows in a dev workspace — confirm secret is not written anywhere in plan or state. (opentofu.org)
- If you keep secrets in state, enable OpenTofu encryption and configure a key provider (preferably external KMS). Validate decryption in your CI and recovery process. (opentofu.org)
- Review provider compatibility for ephemeral/write-only attributes. If not supported, evaluate moving secret management outside of OpenTofu (Vault, cloud secret manager). (opentofu.org)
Why this is a meaningful shift OpenTofu’s combination of ephemeral values and built-in state encryption is an explicit attempt to reduce the risk profile of IaC workflows:
- Ephemeral values change the model: treat infra code as orchestration that can request secrets at runtime, rather than as a datastore for secrets.
- State encryption gives teams the safety net to keep necessary persisted secrets protected when persistence is unavoidable. These approaches complement each other and give practitioners options depending on operational constraints. (opentofu.org)
Where to learn more and next steps
- OpenTofu docs: ephemerality, ephemeral resources, and encryption pages (start with the language docs and release notes). (opentofu.org)
- Try the 1.11 beta or nightly builds in a sandbox to exercise ephemeral examples (don’t use beta in production). (opentofu.org)
- If you’re migrating from Terraform, follow the OpenTofu migration guide (it emphasizes backups and staged testing). (opentofu.org)
Closing thoughts Secrets are the most sensitive part of your infrastructure graph. OpenTofu’s ephemeral and write-only concepts represent a practical way to reduce the attack surface by not storing secrets when you don’t have to — while state encryption protects the cases where persistence is necessary. Start small: prototype ephemeral flows in a test workspace, validate your secret store workflows, and only then roll changes into critical workloads. And as always, back up your state before making changes. (opentofu.org)
Further reading and references
- OpenTofu 1.11.0-beta1 announcement (ephemeral + enabled meta-arg). (opentofu.org)
- Ephemerality & Ephemeral resources docs (usage, lifecycle, examples). (opentofu.org)
- OpenTofu 1.7.0 state encryption release notes and docs. (opentofu.org)
- OpenTofu 1.10 release notes (OCI registry, external key providers, S3 locking improvements). (opentofu.org)
- Migration guide: safe steps for switching from Terraform to OpenTofu. (opentofu.org)
If you’d like, I can:
- Create a small demo repository (with ephemeral + write-only attributes) that you can run in a disposable cluster, or
- Walk you through configuring an external key provider (e.g., AWS KMS or an external script) and validating decryption in CI. Which would help you more right now?