Install Cilium Gateway

Cilium Gateway API serves as a single entry point to the cluster for handling external traffic to BRIX. Gateway API provides a unified interface for managing access to BRIX and uses Cilium to pass data to eBPF programs.

Cilium Gateway allows you to:

  • Control access to services from external sources.
  • Automatically configure L7 routing (HTTP/gRPC).
  • Provide TLS decryption.
  • Provide load balancing.

Installing and configuring Cilium Gateway consists of several steps:

  1. Download the Helm chart and configuration file.
  2. Fill in the configuration file.
  3. Install the chart with Helm in the Kubernetes cluster.
  4. Create a TLS certificate.
  5. Create a Gateway.
  6. Configure the external IP address with MetalLB.
  7. Configure BRIX to work with Gateway API.

Important: we recommend using only one Gateway controller in the cluster to avoid conflicts between HTTPRoute and Gateway, unpredictable routing, and increased load. Two controllers may be deployed only temporarily, for example during a migration, or in strictly isolated namespaces with different GatewayClass.

Step 1: Download the Helm chart and configuration file

To install over the Internet, obtain the values-cilium.yaml configuration file, by running the command:

helm repo add brix365 https://charts.brix365.com
helm repo update
helm show values brix365/cilium > values-cilium.yaml

Get the configuration file for installation in an offline environment without Internet access

  1. On a computer with Internet access, download the archive of the latest version of the cilium chart using the command:

helm repo add brix365 https://charts.brix365.com
helm repo update
helm pull brix365/cilium

  1. Copy the downloaded cilium-X.Y.Z.tgz chart archive  to the server where the installation will be performed.
  2. Extract the cilium-X.Y.Z.tgz chart and create a copy of the values.yaml configuration file named values-cilium.yaml:

tar -xf cilium-X.Y.Z.tgz
cp cilium/values.yaml values-cilium.yaml

Step 2: Fill in the configuration file

Specify the Cilium installation settings in the values-cilium.yaml configuration file.

Important: make sure that the url: http://kube-system-kube-prometheus-prometheus.kube-system.svc parameter contains the path to your Prometheus service.

Example values-cilium.yaml file contents

## Cilium settings
cilium:
 gatewayAPI:
   enabled: true
## To use hostNetwork for Gateway (otherwise CNI), uncomment envoy.securityContext.capabilities
   hostNetwork:
     enabled: false
 
## Enable Envoy inside Cilium
 envoy:
   enabled: true
##  When hostNetwork.enabled=true, uncomment securityContext
   securityContext:
     capabilities:
       keepCapNetBindService: true
       envoy: [NET_BIND_SERVICE,BPF,SYS_ADMIN,NET_ADMIN,NET_RAW]

Fill in the private registry connection parameters for installation in an offline environment without Internet access

 
To connect to a private registry:

  1. Download the BRIX images and upload them to the local image registry. For details, see the article Download BRIX images.
  2. In the values-cilium.yaml:
    • in the image.repository specify the path to the private registry in the format registry.example.com.
    • in the imagePullSecrets specify the name of the secret that grants access to the private registry. The secret must be created manually and encoded in Base64.

cilium:
## private registry connection parameters
  image:
    repository: "registry.example.com/docker/addons/cilium/cilium"
    tag: "v1.19.3"
    useDigest: false
## the secret with access to the private registry must be created manually and encoded in Base64
  imagePullSecrets: []
## Nodes on which to run Cilium
  nodeSelector:
    kubernetes.io/os: linux
  tolerations:
    - operator: Exists
 
 
  envoy:
    image:
## private registry connection parameters
      repository: "registry.example.com/docker/addons/cilium/cilium-envoy"
      tag: "v1.36.6-1776000132-2437d2edeaf4d9b56ef279bd0d71127440c067aa"
## Nodes on which to run Cilium
    nodeSelector:
      kubernetes.io/os: linux
    tolerations:
      - operator: Exists

Step 3: Install the cilium chart in the Kubernetes cluster

  1. Install the CRDs from Gateway API version 1.5.1 according to the instructions in the Cilium Gateway API documentation.
    You can also install them using the following commands:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_gatewayclasses.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_gateways.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_httproutes.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_referencegrants.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_grpcroutes.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.1/config/crd/standard/gateway.networking.k8s.io_backendtlspolicies.yaml 
kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.5.0/config/crd/experimental/gateway.networking.k8s.io_tlsroutes.yaml

  1. Use Helm to install the cilium chart in the kube-system:
    1. To install over the Internet, run the following command:

helm upgrade --install -n kube-system cilium brix365/cilium -f values-cilium.yaml

    1. To install without Internet access, go to the directory containing the downloaded chart and run the following command:

helm upgrade --install cilium ./cilium -f values-cilium.yaml -n kube-system

Step 4: Create a TLS certificate

To enable access over HTTPS, create a TLS certificate and add it to Kubernetes as a secret.

You can use:

Add the certificate you created to Kubernetes. To do this, in the kube-system namespace, create a TLS secret with the certificate file in .crt and the key in .key:

kubectl create secret tls brix365-tls -n kube-system \
--cert=/etc/ssl/certs/selfsigned.crt \
--key=/etc/ssl/private/selfsigned.key

where:

  • brix365-tls is the name of the secret that will be used in the Gateway.
  • --cert is the path to the certificate.
  • --key is the path to the private key.

Step 5: Create a Gateway

The Gateway resource defines the rules for processing incoming traffic in the cluster. It specifies which ports and protocols are used and which certificates are applied for HTTPS.

To create the Gateway resource and associate it with the GatewayClass:

kubectl apply -n kube-system -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
   name: cilium-gateway
spec:
   gatewayClassName: cilium
   listeners:
   - name: http
     port: 80
     protocol: HTTP
     allowedRoutes:
       namespaces:
        from: All
   - name: https
     port: 443
     protocol: HTTPS
     tls:
       mode: Terminate
       certificateRefs:
       - name: brix365-tls
     allowedRoutes:
       namespaces:
         from: All
EOF

where:

  • gatewayClassName is the controller that will process the Gateway.
  • listeners is a list of entry points:
    • port is the port.
    • protocol is the protocol: HTTP or HTTPS.
    • tls.mode: Terminate is TLS connection termination on the Gateway.
    • certificateRefs is a reference to the previously created TLS secret. The value in the name field must match the name of the certificate in Kubernetes.
  • allowedRoutes specifies the namespaces from which routes can be attached.

After creating the Gateway resource, you can attach routes to publish services using the HTTPRoute.

Step 6: Configure the external IP address with MetalLB (optional)

For Gateway API to work correctly, the cluster node must have direct Internet access with a public IP address using a NodePort. After creating the Gateway service of the LoadBalancer type, which receives incoming traffic.

You can install MetalLB, which works similarly to cloud load balancers. It uses L2 mechanisms (ARP/NDP) or the BGP protocol to provide IP addresses to LoadBalancer, which are created by Gateway.

When to use MetalLB

 
The MetalLB load balancer is needed when:

  1. The node does not have a public IP address. MetalLB provides a virtual IP address (VIP) from a preconfigured pool. In L2 mode, the active node receives traffic at this VIP, providing keepalive-like behavior.
  2. A highly available Gateway is required. If the current active node fails in L2 mode, another node automatically takes over the VIP after ARP/NDP is updated. BGP mode provides faster failover through the routing protocol.
  3. Gateway API services must be accessible from external sources. Services of the LoadBalancer type, referenced by Gateway, must have a real IP address accessible outside the cluster. Direct access is also required if specific IP addresses are configured for Gateway in the addresses.

MetalLB is not required if:

  1. A public IP address already exists on the node. It is sufficient to use NodePort with direct access to the node, for example, for Gateway with the hostNetwork: true parameter or when specifying NodePort.
  2. The cluster is deployed in a cloud, such as Yandex Cloud. The cloud provider has its own load balancer (NLB, ALB, L7 Load Balancer) that can assign an IP address to a service of the LoadBalancer type, used by the Gateway.
  3. The Gateway controller accepts connections directly on host ports. If the hostNetwork parameter is set to true, the Gateway controller, for example envoy-gateway or istio, accepts connections directly. An additional load balancer is not required.
  4. In a test environment without access requirements. A simpler and faster solution is to use Gateway with a service of the NodePort type  and the node IP address.
  5. OpenELB or another load balancer is already installed. MetalLB will conflict with the existing component if it attempts to manage the same IP pools.

To check whether an external IP address is available for Gateway, view the service settings using the command:

kubectl get svc -n kube-system

Check the value in the EXTERNAL-IP:

  • If an IP address is specified, external access is already configured.
  • If the <pending> or <none> value is set, there is no external load balancer in the cluster:
    • In cloud Kubernetes clusters, the external load balancer is created automatically, so no additional action is required.
    • In local or test environments, you can use MetalLB.

Install MetalLB

To use MetalLB to configure external access to Gateway:

  1. Install MetalLB:

kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.3/config/manifests/metallb-native.yaml

  1. Run the command to monitor pod status and wait for the components to start:

kubectl get pods -n metallb-system -w

  1. Configure the IP address pool:

kubectl apply -f - <<EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: host-ip-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.29.176-192.168.29.176
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: host-ip-adv
  namespace: metallb-system
spec:
  ipAddressPools:
  - host-ip-pool
EOF

where 192.168.29.176 is an example of the host IP address where Kubernetes is installed. Use the IP address of a node in your cluster that is accessible to external traffic.

  1. Check the settings of the Gateway:

kubectl get svc -n kube-system

Gateway is ready to route BRIX traffic.

Step 7: Configure BRIX to work with Gateway API

After installing Gateway enable Gateway API support in the BRIX configuration.

To do this, in the values-brix365.yaml configuration file, add or modify the following parameters:

global:
 …
  gatewayAPI:
  ## enable Gateway API (HTTPRoute)
    enabled: true
    parentRefs:
    - name: cilium-gateway
      namespace: kube-system

where:

  • enabled enables the use of Gateway API instead of Ingress.
  • parentRefs specifies the Gateway, through which BRIX is published.

The values must match the parameters of the previously created Gateway.

For more information about editing the configuration file, see the article about changing BRIX parameters.

cilium-ebpf.html envoy-gateway.html