BKE

Migrate

From Rancher to BKE.

RKE2 is upstream Kubernetes, so the workloads on a Rancher-managed cluster usually move to BKE unchanged. The work lies in what Rancher added above the cluster: projects, authentication, Fleet, its monitoring and logging applications, and the ingress controller. This guide lists each, and the decision it requires.

This guide assumes the method on the migration overview: an inventory, a classification, a target cluster built beside the source, and workloads moved in waves. It covers what is specific to a downstream cluster managed by Rancher, usually running RKE2. Of the three platforms with guides here, this is normally the shortest migration, because both distributions install upstream Kubernetes and several of the same components.

The inventory

# Namespaces and the Rancher project each belongs to.
kubectl get ns -o custom-columns=NAME:.metadata.name,PROJECT:.metadata.annotations.field\\.cattle\\.io/projectId

# Ingress objects and the controller each is written for.
kubectl get ingress -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name,CLASS:.spec.ingressClassName

# Applications installed from Rancher's catalogue, as Helm releases.
helm list -A

# Persistent volumes and the storage classes behind them.
kubectl get pvc -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name,CLASS:.spec.storageClassName,SIZE:.spec.resources.requests.storage

Add to it the authentication provider Rancher is configured with, every Fleet GitRepo, and any policy written for the cluster’s network plugin rather than as a Kubernetes NetworkPolicy.

Workload objects

Rancher and RKE2 On BKE The work involved
Deployment, StatefulSet, Service, ConfigMap, Secret, Job, CronJob Unchanged Export, remove cluster-assigned fields and cattle.io annotations, apply
Kubernetes NetworkPolicy Unchanged, enforced by Calico None
Calico policy resources, from RKE2’s Canal or Calico Calico policy resources Usually unchanged, since BKE’s network is Calico. Confirm each resource against the Calico version in use
Cilium policy resources Kubernetes NetworkPolicy, or Calico’s own policy resources Rewrite
Project Namespaces, with access granted per namespace See Projects and access below
Ingress for ingress-nginx Ingress with ingressClassName: kong See Ingress below
Pod Security Admission labels Unchanged Rancher’s Pod Security Admission configuration templates become explicit labels on each namespace

Projects and access

A Rancher project groups namespaces so that one role binding applies to all of them. Kubernetes has no such object, and neither does BKE. In config.yaml, auth.rbac.namespaces grants a group admin, edit or view in each namespace named, so a project’s members become a group listed under each of the project’s namespaces. Project resource quotas become a ResourceQuota in each namespace.

Rancher authenticates users itself, against Active Directory, an LDAP directory, an OIDC provider or others, and passes their requests to the cluster. On BKE the API server trusts your identity provider directly, through OIDC, and users log in with kubectl without an intermediary. The provider must be able to emit a groups claim; Cluster login explains why. A directory reached through LDAP is usually reached through an OIDC provider federated with it.

Role bindings to individual Rancher users have no equivalent in config.yaml and should be converted to group membership in the identity provider.

Ingress

RKE2 installs ingress-nginx by default, and the Kubernetes project has retired ingress-nginx, so this change is required whether or not the estate leaves Rancher. Each Ingress needs ingressClassName: kong in place of nginx. The nginx.ingress.kubernetes.io/ annotations each have a different answer:

  • Rate limits, IP allow-lists, basic authentication and CORS become KongPlugin resources attached to the Ingress. See Ingress with Kong.
  • Timeouts and body-size limits become Kong settings on the Service or the route, following Kong’s documentation.
  • configuration-snippet and server-snippet hold raw nginx configuration and have no translation. Each must be read and its purpose re-expressed.

Certificates issued by cert-manager are re-issued on BKE by re-creating the ClusterIssuer and keeping the annotation on each Ingress.

Storage

Rancher clusters frequently run Longhorn, which is also BKE’s storage component. That allows a fourth way to move data, in addition to those on the migration overview: restore from a Longhorn backup.

  1. Point the BKE cluster’s Longhorn at the same backup target as the source cluster, or at a copy of it. See Storage for the backup.target setting.
  2. Take a backup of each volume on the source, with the application stopped or its data quiesced so that the backup is consistent.
  3. Restore each backup as a volume on the BKE cluster, and bind it to a PersistentVolumeClaim of the same name in the target namespace.

Longhorn restores a backup into a version equal to or newer than the one that took it. Compare the two clusters’ Longhorn versions before relying on this method, and test one restore before planning a wave around it. Replication at the application level remains the method with the shortest interruption.

A cluster using another storage system moves its data by the methods on the overview.

Rancher’s applications

Rancher On BKE
Rancher Monitoring BKE’s netdata component, which collects application metrics exposed in the Prometheus format as well as node and container figures. Where the estate depends on ServiceMonitor, PodMonitor or PrometheusRule objects, the upstream kube-prometheus-stack chart, on which Rancher Monitoring is based, accepts them unchanged and is installed and operated by you
Rancher Logging BKE’s Fluent Bit component. Each ClusterOutput and Output becomes a Fluent Bit output. See Logging with Fluent Bit
Rancher Istio BKE’s Kuma component, with its policies rewritten, or upstream Istio installed and operated by you
Fleet A GitOps tool installed and operated by you, such as Argo CD or Flux, reading the same repositories. Customisations in fleet.yaml need translating to that tool’s form
Applications from the catalogue The same Helm charts, installed from their upstream repositories with the values exported by helm get values

Nodes

RKE2 runs on several Linux distributions. BKE runs on Debian 12 or 13 and refuses any other. Where hardware is too limited for two full clusters at once, the source cluster can be reduced in stages: drain and remove a node from RKE2, reinstall it with Debian, and add it to BKE as a worker. See Adding nodes. Each stage reduces the source cluster’s capacity before workloads have left it, so the order of nodes and waves should be planned together.

The Rancher manager

If Rancher continues to manage other clusters, a BKE cluster can be registered in it as an imported cluster, so that it appears in the same console. Rancher then observes the cluster, but BKE continues to install and upgrade it. If the BKE cluster is the last cluster Rancher manages, the Rancher installation can be retired with the source cluster.

The order of work

  1. Move ingress from ingress-nginx and replace snippet annotations while still on RKE2. Kong can run beside ingress-nginx there, and the change is required in any case.
  2. Build the BKE cluster with cluster login enabled and Longhorn configured with a backup target.
  3. Install the services the applications depend on, such as the GitOps tool, and point it at the BKE cluster.
  4. Move the pilot application, including its data.
  5. Move the remaining applications in waves, project by project, and decommission as the overview describes.

The comparison with Rancher sets out why an organisation might make this move.

Begin with one namespace on a trial cluster.

A 14-day trial licence is self-service. It is long enough to build a target cluster and move a pilot application onto it.