BKE

Feeding your load balancer

A cluster without a cloud load balancer is usually reached through one you run yourself: HAProxy or another load balancer in front of the nodes, sending traffic to the ports your ingress listens on. Its list of servers has to follow the cluster’s nodes. aybak does that for you. It is optional, and nothing of it is installed until config.yaml asks for it (see the aybak block in config.yaml).

What it does

aybak runs as one pod in the namespace bke-aybak. Every resyncSeconds, 15 by default, it reads the cluster’s nodes and selects:

  • the worker nodes, or every node with nodes: all;
  • of those, the nodes that are Ready and accept new pods. A node you cordon leaves the set, so the load balancer stops sending to a node before you drain it;
  • one address per node: the first InternalIP or ExternalIP in the node’s status, or the IPv4 address Calico has assigned it. On a dual-stack node the first InternalIP may be an IPv6 address; calico always gives IPv4.

When the set differs from the one an output last accepted, aybak sends the new set to that output. An unchanged set is not sent again.

A failed or empty read sends nothing. If the Kubernetes API cannot be read, or no node qualifies, every output keeps the last set it accepted. A load balancer emptied because the API server was briefly unreachable would take your whole cluster offline, and aybak never does that.

The set it sends is the same for every output:

{
    "cluster": "prod",
    "generated": "2026-11-11T10:00:00Z",
    "nodes": [
        {"name": "worker-1", "address": "10.1.0.4"},
        {"name": "worker-2", "address": "10.1.0.6"}
    ]
}

The nodes are sorted by name. cluster is cluster.name from config.yaml.

The three outputs

Use whichever suits your load balancer; more than one may be used at once. An output that fails is tried again on the next pass, and does not hold back the others.

git

aybak commits the set to a file in a git repository, for a pipeline of yours that renders the load balancer’s configuration and deploys it. The file is nodes/<cluster.name>.json unless git.path names another. A commit is made only when the nodes differ from those already in the file, so a restarted aybak does not commit the same nodes again.

aybak reaches the repository over SSH or HTTPS, as git.url says.

Over SSH, with a URL such as ssh://git@git.example.com/lb/nodes.git:

  • The git server’s host key must be in /etc/bke/secrets.d/aybak-git/known_hosts. aybak does not learn a host key on first contact, so a server that presents a different key is refused.
  • The key in ssh-privatekey needs write access to the branch, and nothing more. A deploy key on that one repository is enough.

Over HTTPS, with a URL such as https://git.example.com/lb/nodes.git and git.user:

  • The token, or password, of git.user goes in /etc/bke/secrets.d/aybak-git/token. It needs write access to that one repository and nothing more; where your git server can limit a token to one repository, do so.
  • git asks for the token only when the server does, and aybak hands it over from the file at that moment. It is never in a process argument, the environment or a log line, and nothing stores it.
  • The server’s certificate is checked against the public certificate authorities, or against your own with privateCa: true.

webhook

aybak sends the set to an https:// URL as a POST with a JSON body. The header X-Aybak-Signature carries sha256= followed by the HMAC-SHA256 of the exact body, keyed with the contents of /etc/bke/secrets.d/aybak-webhook/key.

Check the signature before acting on the set. Compute the same value over the body as received and compare the two in constant time; refuse the request if they differ. A receiver that does not check it would accept a node list from anyone who can reach it. Any status from 200 to 299 is taken as accepted.

HAProxy’s Data Plane API

aybak replaces the servers of the backends you name, through the Data Plane API (version 3), in one transaction: HAProxy has either the old servers in every named backend or the new ones in every named backend, never a mixture. Each server is named aybak-<node name>, has health checks on, and listens on the port you give for its backend.

A backend named in dataplane.backends belongs to aybak. Every server in it is replaced, including one you added by hand. Keep any server aybak should not manage in a backend of its own.

Certificates from your own authority

If the webhook, the Data Plane API or the git server over HTTPS presents a certificate issued by your own certificate authority, set privateCa: true and put the authority’s certificate, in PEM form, in /etc/bke/secrets.d/aybak-ca/ca.crt. Only that authority is then trusted for those outputs; the public authorities are not. Git over SSH checks the server’s host key instead, and is not affected.

What leaves your cluster

aybak sends the node set, as above, to the destinations you configure and to nowhere else. Its credentials are mounted into its pod from the Secrets BKE creates from /etc/bke/secrets.d/; they are never sent to us. The pod needs to reach each destination: allow it in any egress rule your network applies.

What it is allowed to do

aybak’s ServiceAccount may get and list nodes, and nothing else. Its pod runs as a fixed non-root user, with a read-only root filesystem, no added capabilities, and the namespace enforcing Kubernetes’ restricted Pod Security level.

Knowing it is working

kubectl -n bke-aybak get pods
kubectl -n bke-aybak logs deployment/aybak

The pod is Ready only while every output holds the current set. It is not Ready while an output is failing, or while no node qualifies, and the log says which: one line per event, in JSON. A pod that is not Ready is not restarted for that reason alone; it keeps trying.

apply.sh --commit waits up to two minutes for the pod to become Ready, and warns if it does not. On a node that has not pulled aybak’s image before, the pull alone can take longer than that; kubectl -n bke-aybak get pods shows which it is.

Changing a setting or a credential

Change config.yaml or the files in /etc/bke/secrets.d/, and run apply.sh --commit. aybak is restarted when either has changed, and not otherwise.

Turning it off

Set enabled: false, or remove the block, and run apply.sh --commit. The namespace bke-aybak, with everything in it, and aybak’s cluster role are removed. Your load balancer keeps the last set it was sent; from then on it is yours to keep up to date.

What it does not do

  • It does not configure the load balancer beyond its server list. Frontends, ports, timeouts and TLS remain yours.
  • It does not check that a node answers on the ports you send traffic to. Your load balancer’s own health checks do that.
  • It sends one address per node, never both of a dual-stack node’s addresses.