Secret storage and recovery

LyftData encrypts stored variables and Credential Manager material. The matching master keys are separate from the encrypted state: a database backup without its original matching keys cannot be decrypted.

Prerequisites

Keep the complete staging directory (the state root) on a supported persistent local filesystem, such as ext4 or XFS. A container’s writable overlay layer is not eligible for automatic file-backed secret storage. A bind mount or named volume must also use a supported underlying filesystem; mounting another overlay-backed directory does not establish that boundary.

Depending on the installed version, an unsupported location reports backend is unavailable or automatic secret-storage selection cannot use this state-root filesystem. For a fresh evaluation, mount the complete Product state at a persistent path before starting LyftData. For a container with its actual state volume mounted at /data, declare that mount and keep staging inside it:

lyftdata run server --staging-dir /data --secret-storage-runtime container --secret-storage-state-mount /data

The runtime user must own the private state and be able to write it. See the Docker guide for the image’s runtime UID and volume setup. Mounting only the key directory does not preserve the complete application. For existing state, stop its owner and use the documented offline backup and recovery procedure to preserve databases, original keys and metadata together. Do not delete secret-storage records or generate replacement keys to resolve a filesystem diagnostic.

Key sources

Keep the original key value and source for each family. Server Variables use LYFTDATA_VARIABLES_MASTER_KEY_SOURCE and LYFTDATA_VARIABLES_MASTER_KEY; Credential Manager uses LYFTDATA_CREDENTIAL_MANAGER_MASTER_KEY_SOURCE and LYFTDATA_CREDENTIAL_MANAGER_MASTER_KEY; Worker Settings, including the built-in Worker, use LYFTDATA_SETTINGS_MASTER_KEY_SOURCE and LYFTDATA_SETTINGS_MASTER_KEY. The source choices are auto, env, and keyring. Deliver environment-backed keys through the runtime account’s approved secret-injection path.

Server also accepts --variables-master-key-file, --credential-manager-master-key-file, and --settings-master-key-file; lyftdata run worker accepts --settings-master-key-file. Keep the corresponding source at auto and its master-key environment value absent. Each file holds an existing 32-byte key encoded as 64 hexadecimal characters or base64; on Unix it must be a regular, non-symlink file owned by the runtime user with one link and mode 0600. Use a different key for each family.

Preserve both encrypted state and these original keys under separate access controls. If a source becomes unavailable, restore its original account, session or secret-delivery path. Generating a replacement key cannot recover existing encrypted data.

Automatic storage keeps a recorded owner

On first use of an empty profile, LyftData chooses persistent storage from the observed launch context. It does not treat a desktop login, service account, container user, WSL user, or scheduled task as the same runtime identity. Once a profile is ready, its secret-storage record preserves the exact key reference and backend. Normal startup reuses that recorded reference; it does not create a replacement key or silently switch source because the account, path, or platform looks different.

For a Server, the recovery set contains all of the following profiles:

Server Variables and Auth: the Variables and Credential Manager key families;
Server Settings; and
built-in Worker Settings.

A standalone Worker has one separate Worker Settings profile. A Server export always covers all three Server profiles together; it is not a selection of individual secrets.

The state root keeps the protected secret-storage records. A managed-file choice can also keep its private material there; an account-bound choice can keep material outside that directory. The record is part of the installation’s durable state. Do not copy, edit, or re-point a recorded reference by hand.

Legacy environment, supplied-file, and legacy-keyring sources remain read-only compatibility inputs. They can be used to read the exact original material when it is still available, but they are not destinations for a restore. A copied state root does not make an account-bound key available on another machine or under another account.

Back up encrypted state and recovery material separately

Plan for two related but separately protected artifacts:

A matching, offline backup of the complete Server staging directory, including all three Server profiles and their secret-storage records; or the complete standalone Worker jobs/state directory and its records. Do not back up Server profiles one at a time.
A protected recovery bundle created while the original keys and matching encrypted state are still available. Keep the bundle’s recovery secret separately from both the bundle and the state backup.

Stop the managed Product owner before making the state backup or using the offline commands below. Use the supported Product backup and offline ownership path for the deployment; do not copy a running database or inspect it with generic SQLite tools. A recovery bundle contains the original keys and their identity, not the databases. A state backup alone is incomplete when the original key material is account-bound or outside the state directory, and a bundle alone cannot recreate the encrypted state it verifies.

Protect each artifact with access controls appropriate to the runtime account and retention policy. In particular, do not place the recovery secret in a database archive, command history, normal environment, log, ticket, or support bundle. Rehearse recovery only against an isolated matching state copy and synthetic credentials.

See Backup and Recovery for the wider operational backup set, including startup configuration and job definitions, and Credential lifecycle and recovery for the credential incident boundary.

Use the offline recovery commands

lyftdata secret-storage is an offline local command. Stop every managed Product owner for the selected state root and ensure there is no live owner before running status, export, or restore. The command requires the existing matching local databases; it does not create replacement databases.

Use the actual role, absolute state-root path, and launch context. These are the Server commands for a Linux service installation:

# Inspect existing profiles without creating or changing keys.
lyftdata secret-storage status --role server --state-root /srv/lyftdata --runtime service --json

# Verify every existing Server profile, then write a new protected bundle.
lyftdata secret-storage export --role server --state-root /srv/lyftdata --runtime service --output /backup/lyftdata-recovery.age

# Restore the original keys beside matching database backups in the destination root.
lyftdata secret-storage restore --role server --state-root /srv/lyftdata-restored --runtime service --input /backup/lyftdata-recovery.age

Use --role worker with a standalone Worker’s existing jobs/state directory. --runtime accepts desktop, service, container, or wsl. A container also needs its declared persistent mount when applicable:

lyftdata secret-storage status --role worker --state-root /var/lib/lyftdata-worker --runtime container --state-mount /var/lib/lyftdata-worker --json

status is existing-only: it reports the readiness of each independently inspectable profile and never provisions a fresh key. export verifies every profile in the selected group before it writes a bundle; an existing output path is never overwritten. restore authenticates the whole bundle and verifies all matching ciphertext before it changes any destination profile. If a profile is missing, mismatched, locked, unavailable, or conflicts with unrelated destination state, follow the reported action and do not reset keys.

Before restore, place the matching database backup under the explicit destination state root using the supported offline backup procedure. A valid bundle can restore the verified original keys into a writable supported destination after loss of the original machine, account, or source, but it cannot decrypt unrelated state. If interruption occurs after recovery begins, repeat the same restore command with the same bundle and destination; do not substitute fresh keys or a different source.

Protect the recovery secret

Without an input file, export prompts twice for a non-echoed recovery secret and restore prompts once. For a headless service, use a protected file instead:

lyftdata secret-storage export --role server --state-root /srv/lyftdata --runtime service --output /backup/lyftdata-recovery.age --recovery-secret-file /run/lyftdata/recovery-secret

lyftdata secret-storage restore --role server --state-root /srv/lyftdata-restored --runtime service --input /backup/lyftdata-recovery.age --recovery-secret-file /run/lyftdata/recovery-secret

The file contains at most 128 UTF-8 secret bytes; one final LF or CRLF is ignored. The protected reader rejects a file that does not meet the local private-file rules, including restrictive Unix ownership and mode plus compatible Windows and macOS access controls. Keep the path private and give it to export or restore only. status does not accept --recovery-secret-file.

Never supply the recovery secret through a command argument, normal environment variable, log, support bundle, or shell history. The bundle is authenticated, but that does not remove the need to protect the secret.

Original sources, supplied files, and loss

Export can protect only keys that are already ready and can be verified against their owning encrypted data. It cannot reconstruct a missing original key. For a supplied source, status and export need the exact original file inputs whose recorded identities match the profile:

lyftdata secret-storage status --role server --state-root /srv/lyftdata --runtime service \
  --variables-master-key-file /secure/variables.key \
  --credential-manager-master-key-file /secure/credentials.key \
  --settings-master-key-file /secure/settings.key --json

For a standalone Worker, only --settings-master-key-file applies. An environment or supplied-file source that has already been lost cannot be recovered by guessing a value, copying metadata, or generating a new key. If a previous protected export and its recovery secret are available, use restore with the matching state backup. Restore chooses a supported writable destination for the actual runtime; it does not write to a supplied read-only file or infer a missing source path from a fingerprint.

This recovery path preserves verified original key bytes for the Server or Worker recovery set. It is not a general secret vault, a merge mechanism for two installations, or a live migration API.

Account and location constraints

A Windows desktop Credential Manager entry does not automatically serve a Windows service. Use the service account’s approved secret-delivery path and verify its ACLs.
A macOS launch daemon normally runs outside a logged-in user’s Keychain session. Treat the daemon account and restart boundary as separate.
On Linux, Secret Service depends on the service account and session. An unavailable D-Bus Secret Service endpoint does not make a desktop unlock prompt a suitable service recovery path.
On WSL, treat LyftData as a Linux process and keep managed private-key files in the Linux filesystem under the Linux runtime account, not under /mnt/c.
In containers, persist state outside the image and provide storage and recovery material for the actual container user. Do not modify a mounted supplied secret in place to force it into a different source contract.

None of these choices protects against an attacker controlling the runtime account or an administrator who can access the running system. Keep state backups, recovery bundles, and recovery secrets under separate access controls.

If normal startup cannot load the expected key, stop unsafe write activity and use the bounded offline recovery path. After a successful restore, start LyftData normally and verify redacted credential metadata, activation state, and a bounded non-destructive workload through the intended Worker.