BKE

Migrate

From Tanzu to BKE.

Tanzu Kubernetes clusters run close to upstream Kubernetes, so most workload manifests move to BKE unchanged. The work lies in what vSphere and the Tanzu packages provided around them: volumes, load balancers, the network, ingress and identity. This guide lists each, and the decision it requires.

This guide assumes the method on the migration overview: an inventory, a classification, a target cluster built beside the source, and workloads moved in waves. It covers what is specific to Tanzu — workload clusters created by a vSphere Supervisor or by Tanzu Kubernetes Grid. A workload cluster’s own manifests are usually portable; what the inventory must find is every dependency on the estate around it.

The vSphere estate need not move first

BKE requires Debian 12 or 13 servers and is indifferent to what runs them. Debian virtual machines on the existing vSphere estate are suitable nodes, and building the target cluster there separates two decisions: leaving Tanzu, and deciding the future of the hypervisor. The second can be made later, on its own timetable, by adding nodes elsewhere and draining the virtual ones.

The nodes are ordinary virtual machines, created by whatever means the estate already uses. Cluster API does not create them, and BKE does not require it. See Adding nodes.

The inventory

Three groups of objects in a Tanzu workload cluster indicate dependencies on its surroundings:

# Services that vSphere networking, NSX or Avi gives an external address.
kubectl get svc -A --field-selector spec.type=LoadBalancer

# Persistent volumes and the storage classes behind them.
kubectl get pvc -A -o custom-columns=NS:.metadata.namespace,NAME:.metadata.name,CLASS:.spec.storageClassName,SIZE:.spec.resources.requests.storage

# Packages installed through kapp-controller.
kubectl get packageinstalls -A

Add to it every Antrea-specific policy (resources in the crd.antrea.io group), every Contour HTTPProxy, and the way users currently log in.

Workload objects

Tanzu On BKE The work involved
Deployment, StatefulSet, Service, ConfigMap, Secret, Job, CronJob Unchanged Export, remove cluster-assigned fields, apply
Kubernetes NetworkPolicy Unchanged, enforced by Calico None
Antrea ClusterNetworkPolicy and NetworkPolicy (crd.antrea.io) Kubernetes NetworkPolicy, or Calico’s own policy resources Rewrite. Tiers and cluster-wide rules have equivalents in Calico, but not in the same form
Service of type LoadBalancer See Load balancers below A decision for each Service
Contour HTTPProxy Ingress with ingressClassName: kong See Ingress below
Pod Security Admission labels Unchanged None. Both platforms use the upstream mechanism
PersistentVolumeClaim on a vSphere storage class A PersistentVolumeClaim on a Longhorn storage class, with the data copied See Storage below

Load balancers

On a Tanzu cluster, a Service of type LoadBalancer receives an external address from vSphere networking, NSX or Avi. BKE does not include a load-balancer implementation for Services, and a Service of that type on a BKE cluster remains without an address.

Each such Service therefore needs one of three answers:

  • HTTP and HTTPS move behind Kong, as an Ingress. Kong is reached through the load balancer in front of your worker nodes. See Reference architecture.
  • Other TCP services, such as a database exposed to clients outside the cluster, become Services of type NodePort, with your load balancer forwarding a port to every worker.
  • Where many Services need addresses of their own, a load-balancer implementation for bare-metal clusters can be installed and operated alongside BKE. That is a component outside BKE, and outside BKE support.

The existing Avi or NSX load balancer is a suitable device to place in front of the BKE cluster, configured as described in the reference architecture.

Ingress

Contour, installed as a Tanzu package, routes through HTTPProxy resources, or through standard Ingress objects. Standard Ingress objects need only ingressClassName: kong. Each HTTPProxy becomes an Ingress, and the features it used — rate limits, header changes, timeouts, authentication — become KongPlugin resources attached to it. See Ingress with Kong.

Certificates issued by a cert-manager package move to BKE’s cert-manager: re-create the ClusterIssuer, add its annotation to each Ingress, and the certificates are issued afresh. Certificates from an internal certificate authority can be imported as Secrets, or issued through a cert-manager CA issuer holding that authority’s key.

Storage

Volumes on a vSphere storage class are virtual disks provisioned by the vSphere CSI driver, and they do not move between clusters. BKE’s storage component is Longhorn, which replicates volumes across the cluster’s own nodes and backs them up to an S3-compatible store.

Copy the data by one of the methods on the migration overview, in its order of preference: replication at the application level, then dump and restore, then a volume-level copy with a tool such as Velero. Where the estate already uses Velero for Tanzu backups, that experience transfers directly.

When BKE nodes are virtual machines, give each one a dedicated virtual disk for Longhorn’s data, on a datastore whose performance suits the workloads. Longhorn replicates across nodes, so the replica count and the datastore’s own redundancy should be decided together rather than multiplied without intent. See Storage.

Identity and access

Tanzu clusters authenticate users through Pinniped or through vSphere single sign-on. BKE has neither: the API server trusts your identity provider directly, through OIDC. If Pinniped is already federated with an OIDC or LDAP provider, the same provider is usually the right one for BKE, provided it can emit a groups claim. Cluster login explains why that claim is required.

Group role bindings map onto auth.rbac in config.yaml, cluster-wide or per namespace. Access granted through vSphere namespace permissions has no direct equivalent and must be expressed as groups.

Packages

Each package installed through kapp-controller has one of three destinations.

Package On BKE
cert-manager BKE’s cert-manager component
Contour BKE’s Kong component
Fluent Bit BKE’s Fluent Bit component. Re-create the outputs. See Logging with Fluent Bit
Prometheus, Grafana BKE’s netdata component, which collects application metrics exposed in the Prometheus format as well as node and container figures. Prometheus itself can be installed and operated by you where the estate depends on its alerting rules
Harbor Keep the existing registry, which the BKE cluster pulls from as the Tanzu cluster did, or use the registry chart from the Mamluk Enterprise Helm Charts
external-dns, Velero, and others Installed and operated by you, from the upstream charts

Workloads built on Tanzu Application Platform supply chains depend on a platform of their own: build services, conventions and delivery. These become work in a CI system and deployment tooling — your own, or the CI chart from the Mamluk Enterprise Helm Charts — and should be scoped as a separate project from the cluster migration.

The order of work

  1. Create Debian virtual machines on the existing vSphere estate, and build the BKE cluster on them with cluster login enabled and Longhorn configured with a backup target.
  2. Configure the existing load balancer in front of the BKE workers and control plane.
  3. Decide the answer for each LoadBalancer Service, and translate each HTTPProxy.
  4. Move the pilot application, including its data.
  5. Install the services the remaining applications depend on, such as the registry and CI, before the waves that need them.
  6. Move the remaining applications in waves, and decommission as the overview describes.

The comparison with Tanzu sets out why an organisation might make this move.

Begin with one namespace on a trial cluster.

A 14-day trial licence is self-service. It is long enough to build a target cluster and move a pilot application onto it.