Skip to content

Preparing the host cluster for tenant exposure

The host cluster must be prepared before tenants can be reached from outside. A bare cluster + helm install kubespaces gives you the Tier 1 demo (port-forward access, see prerequisites.md); public tenant endpoints additionally need the pieces on this page, installed once per host cluster. Every step here is cluster-admin, day-0 platform work — after it, tenant exposure is fully automated by the operator.

The reference implementation below uses Envoy Gateway, but nothing in KubeSpaces depends on it: any Gateway API implementation that supports TLSRoute works (the pre-OSS KubeSpaces production setup ran the exact same Gateway shape on Istio for years). Contour, Istio, Envoy Gateway are all known-good TLSRoute implementations; check your implementation's conformance docs before picking something else.

1. Gateway API implementation (with TLSRoute)

TLSRoute is GA (v1, Standard channel) since Gateway API v1.5 (February 2026). The operator creates TLSRoutes using the v1 API (gateway.networking.k8s.io/v1), so Gateway API >= 1.5 CRDs are required — a pre-1.5 install only served TLSRoute in the experimental channel as v1alpha2, which the operator no longer uses. (v1alpha2 is still served by 1.5.x for backward compatibility but is removed in Gateway API 1.6.)

Kubernetes >= 1.31 is also required: the TLSRoute v1 CRD ships CEL validation rules (x-kubernetes-validations), and CEL validation for custom resources only became generally available in Kubernetes 1.31. On older clusters the CRD applies but its validation rules are ignored.

Envoy Gateway bundles the CRDs it needs (TLSRoute v1 included), which makes this a one-liner:

helm install eg oci://docker.io/envoyproxy/gateway-helm \
  -n envoy-gateway-system --create-namespace --wait

If you bring another implementation, make sure kubectl get crd tlsroutes.gateway.networking.k8s.io succeeds afterwards and that it serves v1 (kubectl get crd tlsroutes.gateway.networking.k8s.io -o jsonpath='{.spec.versions[*].name}' should list v1); if it does not, install the standard channel CRDs from gateway-api releases (v1.5.0 or newer).

2. The tenant API Gateway (TLS passthrough)

One shared Gateway carries every tenant's API server traffic. The listener is TLS Passthrough: the gateway routes on SNI and never terminates TLS, so clients verify vCluster's own serving certificate end-to-end and client-cert kubeconfigs keep working. No cert-manager involvement on this path.

kubectl apply -f examples/host/gateway-api.yaml

That manifest (adjust <domain>) is:

apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoy
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: kubespaces-api
  namespace: kubespaces-system
spec:
  gatewayClassName: envoy
  listeners:
    - name: apipassthrough          # referenced by operator.tenantApi.gateway.sectionName
      hostname: "*.api.<domain>"
      port: 443
      protocol: TLS
      tls:
        mode: Passthrough
      allowedRoutes:
        namespaces:
          from: Same                # operator-created TLSRoutes live in this
                                    # namespace; Same (not All) means tenant
                                    # namespaces can never attach routes here

Wait for an address:

kubectl get gateway kubespaces-api -n kubespaces-system
# PROGRAMMED=True and an ADDRESS

3. DNS

Point tenant API hostnames at the Gateway's address. Two options:

  • Wildcard record (simplest): *.api.<domain> → the Gateway's LoadBalancer IP. One record, covers every tenant forever.
  • external-dns: the operator annotates every TLSRoute with external-dns.alpha.kubernetes.io/hostname (and honors an optional target override for on-prem appliances via operator.tenantApi.externalDnsTarget), so external-dns watching tlsroute sources creates per-tenant records automatically.

4. Tell KubeSpaces about it

# values.yaml
operator:
  enabled: true
  tenantApi:
    domain: <domain>                 # tenants get <tenant>.api.<domain>
    gateway:
      name: kubespaces-api
      namespace: kubespaces-system
      sectionName: apipassthrough

All three of domain, gateway.name and gateway.namespace must be set; leave them empty and the operator skips exposure entirely (Tier 1 behavior).

From here the operator automates everything per tenant: TLSRoute + ReferenceGrant, vCluster cert SANs, kubeconfig server URL, and status.apiServerUrl on the Tenant.

5. Verify

kubectl apply -f - <<'EOF'
apiVersion: kubespaces.io/v1alpha1
kind: Tenant
metadata: {name: smoke}
spec: {owner: you@example.com}
EOF
kubectl wait tenant smoke --for=jsonpath='{.status.phase}'=Ready --timeout=5m
kubectl get secret vc-smoke -n kubespaces-tenant-smoke \
  -o jsonpath='{.data.config}' | base64 -d > /tmp/smoke.kubeconfig
kubectl --kubeconfig /tmp/smoke.kubeconfig get nodes   # over the public endpoint
kubectl delete tenant smoke

Tenant app exposure (the apps Gateway)

The second half of the networking story: workloads inside tenants exposed at https://<name>.<tenant>.apps.<domain>. Additional host prerequisites:

  1. cert-manager — the operator creates a per-tenant wildcard Certificate (*.<tenant>.apps.<domain>); a wildcard cert can only be issued via DNS-01, so production needs a DNS-01 ClusterIssuer (evaluation can use a self-signed one):
helm install cert-manager oci://quay.io/jetstack/charts/cert-manager \
  -n cert-manager --create-namespace --set crds.enabled=true --wait
  1. The apps Gateway — TLS-terminating, one static base listener; the operator manages one listener per tenant on it:
kubectl apply -f examples/host/gateway-apps.yaml   # Gateway + selfsigned issuer

  1. DNS — one record covers everything: *.apps.<domain> → the apps Gateway address (DNS wildcards match multiple labels, so it also covers web.tenant.apps.<domain>).

  2. Chart values:

operator:
  tenantApps:
    domain: <domain>
    gateway: {name: kubespaces-apps, namespace: kubespaces-system}
    clusterIssuer: selfsigned          # or your DNS-01 issuer

Per tenant, the operator then: issues the wildcard Certificate, adds listener t-<tenant> (hostname *.<tenant>.apps.<domain>, TLS terminate, allowedRoutes restricted to that tenant's namespace), enables vCluster's native sync.toHost.gatewayApi.httpRoutes, projects the apps Gateway into the tenant's virtual default namespace, and reports status.appsDomain. Tenant users expose an app by creating a plain HTTPRoute in their default namespace, referencing the Gateway they can see right next to it:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: web}
spec:
  parentRefs:
    - name: kubespaces-apps              # projected into default by vCluster
  hostnames: ["web.<tenant>.apps.<domain>"]
  rules:
    - backendRefs: [{name: web, port: 80}]

Routes must live in the virtual default namespace — vCluster projects the Gateway into exactly one namespace, and its route authorization requires the route to sit next to a Gateway it can see. On sync, vCluster translates the parentRef back to the real host Gateway.

Isolation is structural: the synced route lands in the tenant's host namespace, and the only listener that admits routes from that namespace is the tenant's own — whose hostname is *.<tenant>.apps.<domain>. A route claiming another tenant's hostname matches no listener it is allowed to attach to, so hostname theft is impossible without any admission policy. TLSRoute sync stays disabled by design: tenant-authored passthrough routes must never reach the API gateway (which additionally only admits routes from its own namespace).

GitOps note: the operator writes spec.listeners on this one Gateway. Configure your GitOps tool to ignore listener drift on it (Flux spec.ignore, Argo CD ignoreDifferences) — everything else on this page stays fully declarative. Gateway API caps a Gateway at 64 listeners; beyond ~60 tenants, shard across a second apps Gateway (tracked upstream in #16 follow-ups).

GitOps

All of the above is declarative and belongs in your platform GitOps repository (Flux/Argo) rather than in shell history — the pre-OSS KubeSpaces ran exactly this layout (gateway-api → implementation → gateways → DNS) as Flux Kustomizations. A reference layout ships in examples/host/; reconcile that directory and the host is ready.

  • The apps gateway (wildcard TLS termination + cert-manager, for tenant workloads and the portal) is the second half of the networking story — tracked in #16 and #17.
  • prerequisites.md — the tier model and why the API path needs no certificates.