BKE

Migrate

From OpenShift to BKE.

OpenShift is upstream Kubernetes with a large set of platform objects and services above it. A migration to BKE is chiefly the work of finding each of those objects in your estate and giving it an upstream equivalent. This guide lists them, and the decision each one requires.

This guide assumes the method on the migration overview: an inventory, a classification, a target cluster built beside the source, and workloads moved in waves. It covers what is specific to OpenShift. Every estate differs in how far it has adopted OpenShift’s own objects, so the inventory below decides the size of the work, and this guide cannot.

The inventory

OpenShift’s own object types live in API groups ending in openshift.io, and operators installed through the Operator Lifecycle Manager in operators.coreos.com. Counting the objects of each type in use, across every project, is the most useful first measurement of a migration:

for r in routes.route.openshift.io \
         deploymentconfigs.apps.openshift.io \
         imagestreams.image.openshift.io \
         buildconfigs.build.openshift.io \
         templates.template.openshift.io \
         subscriptions.operators.coreos.com; do
  printf '%-45s %s\n' "$r" "$(oc get "$r" -A --no-headers 2>/dev/null | wc -l)"
done

Add to it the custom resources of every operator installed, and the security context constraints each service account is granted, which the next sections explain.

Workload objects

OpenShift On BKE The work involved
Project Namespace Create the namespaces. Project-level annotations, quotas and limit ranges are re-created as ordinary ResourceQuota and LimitRange objects
Route Ingress with ingressClassName: kong See Routes below
DeploymentConfig Deployment See DeploymentConfigs below
ImageStream, ImageStreamTag An image reference in a registry See Images and builds below
BuildConfig, Build A CI system See Images and builds below
Template A Helm chart or a Kustomize overlay Parameters become chart values. Templates are a packaging format and need rewriting, not translating object by object
SecurityContextConstraints Pod Security Admission and each pod’s securityContext See Security context constraints below
EgressFirewall, EgressNetworkPolicy NetworkPolicy egress rules, enforced by Calico Rules by address range translate directly. Rules by DNS name need a different approach
Deployment, StatefulSet, Service, ConfigMap, Secret, Job, CronJob, NetworkPolicy Unchanged Export, remove cluster-assigned fields, apply

When exporting objects, remove the fields the source cluster assigned — metadata.uid, resourceVersion, creationTimestamp, status, and a Service’s clusterIP — and any annotation beginning openshift.io/. Exporting from source control is better than exporting from the cluster, where the manifests are kept there.

Routes

An edge-terminated Route is an Ingress rule with a TLS section. The hostname, path and target Service carry over directly:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: shop
  namespace: retail
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  ingressClassName: kong
  tls:
    - hosts: [shop.example.com]
      secretName: shop-tls
  rules:
    - host: shop.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: shop
                port:
                  number: 80

Four kinds of Route need individual attention:

  • Certificates held in the Route. OpenShift allows the certificate and key to be written into the Route itself. On BKE they belong in a Secret, either issued by cert-manager as above or created from the existing certificate with kubectl create secret tls.
  • Passthrough and re-encrypt termination. These mean the application terminates TLS itself. Decide for each one whether TLS can terminate at Kong instead, which is usually simpler, or configure Kong for the application’s own TLS following Kong’s documentation.
  • Router annotations. Timeouts, rate limits, IP allow-lists and sticky sessions set through haproxy.router.openshift.io/ annotations become KongPlugin resources. See Ingress with Kong for rate limiting, caching and access control.
  • Generated hostnames. A Route without a host receives one under the cluster’s wildcard apps domain. On BKE every hostname is named explicitly, and its DNS record points at your ingress load balancer.

Traffic reaches Kong through the load balancer in front of your worker nodes, as described in Reference architecture, and not through dedicated infrastructure nodes.

DeploymentConfigs

Red Hat has deprecated DeploymentConfig in favour of Deployment, so this work is required whether or not the estate leaves OpenShift. The translation is largely mechanical: change the apiVersion and kind, rewrite the selector as matchLabels, and replace the Rolling or Recreate strategy parameters with their Deployment equivalents.

What does not translate is the trigger. A DeploymentConfig with an image change trigger rolls out when an ImageStream tag moves. A Deployment rolls out when its pod template changes, so the step that publishes a new image must also update the image reference in the manifest, which is the responsibility of your CI or deployment tooling. Lifecycle hooks become Jobs, or init containers, run by the same tooling.

Images and builds

BKE itself includes no image registry and no build system. Both are provided by the Mamluk Enterprise Helm Charts, and a registry or CI system the estate already operates is equally suitable.

  1. Choose the registry the BKE cluster will pull from: the registry chart from the Mamluk Enterprise Helm Charts, or one the estate already operates.
  2. Copy the images that running workloads use out of the integrated registry, by digest, with a tool such as skopeo copy. Images that only exist as ImageStream tags have no other copy, and the inventory must find them.
  3. Rewrite image references. A workload that names image-registry.openshift-image-registry.svc:5000/<project>/<name> must name the new registry instead.
  4. Move the builds. Source-to-image and Docker strategy BuildConfigs, and pipelines on OpenShift Pipelines, become jobs in the CI system that build an image, push it to the registry, and update the manifest that deploys it.

If the registry requires credentials, each namespace needs a pull Secret, referenced from the pod or from its service account.

Security context constraints

OpenShift admits each pod against a security context constraint and, under the default restricted constraints, runs it as an arbitrary user ID allocated to the project. Upstream Kubernetes uses Pod Security Admission, enabled per namespace by label, and runs a container as the user its image or its securityContext specifies.

Two consequences follow:

  • An image that has always run under OpenShift will usually run on BKE, because it has been built to work as a non-root user. The reverse is the risk: an image whose Dockerfile sets no USER runs as root on upstream Kubernetes unless the pod says otherwise.
  • Set the policy explicitly. Label each application namespace with the Pod Security Standard it should meet, and set runAsNonRoot, runAsUser and fsGroup in each pod’s securityContext rather than relying on a platform to assign them:
kubectl label namespace retail \
  pod-security.kubernetes.io/enforce=restricted \
  pod-security.kubernetes.io/warn=restricted

A workload that was granted anyuid, privileged or a custom constraint is the one to examine individually. The grant records a requirement, and the target namespace’s policy must allow it.

Platform services

Identity and access

OpenShift’s OAuth server sits between users and the cluster. BKE has no equivalent: the API server trusts your identity provider directly, through OIDC. Users of OpenShift’s htpasswd provider, or of its own LDAP integration, must be represented in an identity provider that can emit a groups claim. Cluster login explains why that claim is required.

Project role bindings map onto auth.rbac in config.yaml: groups granted access to every namespace, and groups granted admin, edit or view in named namespaces only. Role bindings to individual users have no equivalent in that file, and should be converted to group membership in the identity provider.

Storage

BKE’s storage component is Longhorn, which replicates volumes across the cluster’s own nodes and backs them up to an S3-compatible store. OpenShift Data Foundation volumes, or volumes on an external array through a CSI driver, do not move by themselves: the data is copied by one of the methods on the migration overview, in its order of preference.

Map each source storage class to a Longhorn storage class before the first wave, deciding the replica count deliberately. Storage explains the choices, and why a backup target should exist before any real data does.

Monitoring and logging

OpenShift’s monitoring stack — Prometheus, Alertmanager and the console’s dashboards — is replaced by netdata, which monitors nodes and containers and collects application metrics exposed in the Prometheus format. Applications that publish a Prometheus endpoint therefore need no change to be monitored. metrics-server provides the figures that autoscaling reads.

ServiceMonitor and PrometheusRule objects are resources of the Prometheus Operator, and netdata does not read them. Where the estate depends on them, or on alerting rules written in PromQL, either re-create the scrape targets and alerts in netdata, or install the Prometheus Operator from the upstream charts and operate it yourself.

Log forwarding configured through OpenShift Logging’s ClusterLogForwarder is re-created as a Fluent Bit output. See Logging with Fluent Bit.

Operators

Each operator installed through the Operator Lifecycle Manager must be replaced by an installation that does not depend on it, usually the upstream project’s Helm chart. Its custom resources then move with the workloads that use them, provided the replacement accepts the same resource definitions. Confirm that before relying on it, because a vendor’s certified operator and the upstream project do not always agree.

For databases, caches and message brokers, the Mamluk Enterprise Helm Charts replace the operator with a chart whose replication, backups and upgrade path are already decided.

Service mesh

OpenShift Service Mesh is based on Istio. BKE’s mesh component is Kuma, which is a different product: its policies are not Istio’s, and translating them is a piece of work in its own right. An estate with heavy use of the mesh may prefer to install upstream Istio on BKE and keep its policies. Kuma is opt-in and changes nothing until a namespace is enrolled, so the two do not conflict.

Nodes

OpenShift nodes run Red Hat Enterprise Linux CoreOS, and the Machine API can create and remove them. BKE nodes are Debian 12 or 13 servers that you provision by whatever means the estate already uses. BKE prepares each one, and kubeadm joins it to the cluster. See Adding nodes. Pod-to-pod traffic between nodes is encrypted with WireGuard by Calico, which replaces OVN-Kubernetes IPsec where that was enabled.

The order of work

  1. Translate DeploymentConfigs to Deployments, and move image builds to an external registry, while still on OpenShift. Both changes work there, and making them first reduces the cutover to a change of cluster.
  2. Build the BKE cluster with cluster login enabled, Longhorn configured with a backup target, and Kong behind your load balancer.
  3. Move the pilot application, including its Routes, Secrets and data.
  4. Replace the platform services the remaining applications depend on — monitoring, log forwarding, operators — before the waves that need them.
  5. Move the remaining applications in waves, and decommission as the overview describes.

The comparison with OpenShift and the cost-of-ownership example set out why an organisation might make this move.

Begin with one namespace on a trial cluster.

A 14-day trial licence is self-service. It is long enough to build a target cluster and move a pilot application onto it.