Secrets

BKE needs credentials for some components — Longhorn’s backup target, netdata’s Cloud claim token. It takes them from a directory on the node and never sends them anywhere.

One directory per Secret, one file per key

/etc/bke/secrets.d/
└── longhorn-backup/
    ├── AWS_ACCESS_KEY_ID
    ├── AWS_SECRET_ACCESS_KEY
    └── AWS_ENDPOINTS

The directory name is the Secret’s name. Each file’s name is a key; each file’s bytes are that key’s value. No quoting rules, no escaping, no format to get wrong.

Make sure the bytes are what you mean. echo appends a newline and most credentials do not want one:

printf '%s' 'AKIAEXAMPLE' > /etc/bke/secrets.d/longhorn-backup/AWS_ACCESS_KEY_ID

Permissions are checked

Each directory under secrets.d/ must be mode 0700 exactly. BKE reads the mode and refuses to run if it is anything else — not merely if it is readable by others.

chmod 0700 /etc/bke/secrets.d
chmod 0700 /etc/bke/secrets.d/*
chmod 0600 /etc/bke/secrets.d/*/*

apply.sh also requires root, and requires /etc/bke/license to be 0600.

This is not cosmetic. It exists because credentials sitting readable in a directory that syncs is a real failure we have found in practice, not a hypothetical one.

This directory is never transmitted

config.yaml is sent to api.maml.uk so your settings can be checked and the component configuration rendered, and /etc/bke/values/ is sent so an upgrade can be merged against what you have. secrets.d/ is not, and nothing sends it. They are separate precisely so that some can leave your cluster and this one cannot.

That separation is only as good as what you put where. values/<component>.yaml is yours to edit and BKE cannot tell a password in it from any other setting, so a credential belongs here, in secrets.d/, and is referred to by the Secret’s name. BKE’s own files name Secrets and never contain their values.

The consequence is that we cannot help you with the contents of this directory, and cannot see whether a value is correct. Which brings us to the part of this page that matters most.

What a missing key actually costs

In BKE as it stands, a missing key is always silent. The component starts, reports healthy, and the feature that needed the key does not work.

There are two ways a Helm chart can consume a Secret, and they behave oppositely. Both were measured on a cluster at this release’s Kubernetes version, with a Secret containing one key and pods asking for another:

Mechanism	A missing key does this	You would see
envFrom.secretRef	pod Running, variable simply unset	nothing at all
env.valueFrom.secretKeyRef	pod Pending, CreateContainerConfigError	a clear error naming the key

Every component BKE installs uses the first mechanism. Longhorn’s manager reads its credential through the Kubernetes API rather than a mount; netdata uses envFrom in all three of its pod specs. So the second row does not happen to you, and the first one does.

Per key

Secret	Key	Absent means
netdata’s claim Secret	NETDATA_CLAIM_TOKEN	netdata runs and never claims. The cluster does not appear in Netdata Cloud. No error anywhere
Longhorn’s backup Secret	AWS_ACCESS_KEY_ID	Longhorn runs; the backup target cannot authenticate, so backups do not happen
Longhorn’s backup Secret	AWS_SECRET_ACCESS_KEY	as above
Longhorn’s backup Secret	AWS_ENDPOINTS	Required only for S3 that is not AWS’s own. Legitimately absent on AWS S3 — which is why this warns rather than refuses

AWS_ENDPOINTS must carry a scheme: https://s3.example.com, not s3.example.com. A bare host is rejected by Longhorn as an invalid URI, and the result is a backup target that never becomes available — with the reason visible only in longhorn-manager’s log. We have had this one ourselves.

So BKE warns, loudly

Because the warning is the only signal you will get, apply.sh names any key it expected and did not find, and says in those words that the component will start anyway. Read those warnings. A clean-looking install with a backup target that has never worked is what those warnings are there to prevent.

It warns rather than refuses because BKE’s list of expected keys is fixed and a legitimate cluster can differ from it — AWS_ENDPOINTS on AWS S3 being exactly that case. Refusing would make BKE wrong about a working configuration.

Values are never read, only presence is checked. Extra keys are never mentioned.

A missing Secret, by contrast, refuses

Every Secret name referenced in config.yaml must resolve — to a directory here, or to a Secret already in the cluster if the name is in secrets.unmanaged. An unresolved name is a pre-flight refusal, before the first chart is fetched. Every unresolved reference is reported, not just the first one.

The distinction is the point: a missing Secret means you have not finished setting up, and stopping is right. A missing key might mean your configuration is legitimately different from the default one.

Keeping your own Secrets

If a Secret is produced by sealed-secrets, the External Secrets Operator, or by hand, list its name under secrets.unmanaged in config.yaml. BKE will verify it exists and create nothing.

Note that components.<name>.secrets and components.<name>.settings.*.credentialSecret are not the same list. secrets names what BKE must create; the settings name what the values reference. A cluster whose Secret comes from elsewhere references the name and lists nothing under secrets.

Rotating a credential

Edit the file and re-run apply.sh. The Secret is updated in place.

Updating a Secret does not by itself change a running pod. envFrom is resolved once, when the container starts, so pods holding the old value keep it. BKE restarts exactly the workloads that use a Secret it changed.

If you rotate a Secret by some other route — your own kubectl apply, or an operator syncing it — nothing restarts, and the old value stays in use until something else happens to restart those pods.

What does leave your cluster

Nothing on this page. secrets.d/ is read on the node, mapped onto Kubernetes Secrets, and never transmitted anywhere.

One thing does leave, on a schedule, and it is a different thing entirely: a daily report of your BKE and component versions, a node count and a cluster identifier. Cluster reporting sets out every field of it, what the job is and is not permitted to read, and how to turn it off.