Kubernetes cluster

BRIX is installed in a Kubernetes cluster. To manage it, we recommend using the Deckhouse platform built on Open Source components.

Deckhouse is certified by CNCF. Besides Kubernetes, this platform incorporates additional modules for monitoring, traffic balancing, autoscaling, secure access, and more. The modules are pre‑configured, integrated with each other, and ready to use. The management of all cluster components and the platform, as well as their updates, are fully automated.

You can check the supported Kubernetes versions for the platform in the official Deckhouse documentation.

This article outlines the deployment of a Kubernetes cluster consisting of a single master node.

The installation consists of five steps:

1. Prepare infrastructure.
2. Prepare configuration file.
3. Install the Kubernetes cluster based on Deckhouse.
4. Set up Deckhouse.
5. Install Helm.

Step 1: Prepare infrastructure

To deploy a Kubernetes cluster based on the Deckhouse platform, you will need:

1. A personal computer.

A computer from which the installation will be carried out. It is only needed to launch the Deckhouse installer and will not be part of the cluster.

System requirements:

• OS: Windows 10+, macOS 10.15+, Linux (Ubuntu 18.04+, Fedora 35+).
• Installed Docker for running the Deckhouse installer.
• Access to a proxying registry or to a private container image storage with Deckhouse container images.
• SSH access by key to the node that will be the master node of the future cluster.

2. Master node.

A server (physical server or virtual machine) that will be the master node (master node) of the future cluster.

During the installation process, the Deckhouse installer, launched on a personal computer, will connect to the master node via SSH, install the necessary packages, configure the Kubernetes control plane, and deploy Deckhouse.

System requirements:

• At least 12 CPU cores.
• At least 16 GB of RAM.
• At least 200 GB of disk space.
• A supported OS.
• Access to a proxying registry or to a private container image storage with Deckhouse container images.

Начало внимание

Deckhouse only supports Bearer token authentication scheme in the registry.

Конец внимание

• Access to a proxy server for downloading OS deb/rpm packages (if necessary).
• SSH access from a personal computer using key.
• The node should not have container runtime packages installed, such as containerd or Docker.

Important: Installation directly from the master node is currently not supported. The installer in the form of a Docker image cannot be run on the same node where the master node is planned to be deployed, as the node should not have container runtime packages installed, such as containerd or Docker. In the absence of management nodes, install Docker on any other node of the future cluster, run the Docker image of the installer, install Deckhouse, and then remove the Docker image of the installer from the node along with Docker.

3. Downloading Deckhouse images to the local image registry.

You can deploy a Kubernetes cluster using Deckhouse in a closed environment without internet access. For this, first download the Deckhouse platform images on a computer with internet access and upload them to the local image registry. Read more in Download Deckhouse Images.

Step 2: Prepare configuration file

To install Deckhouse, prepare a YAML configuration file for installation. To obtain the YAML configuration file, use the Getting started service on the Deckhouse website. The service will generate an up-to-date YAML file for the current version of the platform.

1. Generate the YAML file using the Getting started service by following these steps:
2. Choose the infrastructure: Bare Metal.
3. Read the installation information.
4. Specify the template for the cluster’s DNS names. In our case it is %s.example.com.
5. Save config.yml.

2. Make the necessary changes in config.yml by doing the following:

1. Set the cluster’s pod address space in podSubnetCIDR.
2. Set the cluster’s Service address space in serviceSubnetCIDR.
3. Specify the required Kubernetes version in kubernetesVersion.
4. Check the update channel in releaseChannel (EarlyAccess).
5. Check the domain name template in publicDomainTemplate (%s.example.com).
   It is used to form domains for system applications in the cluster. For example, Grafana for the template %s.example.com will be accessible as grafana.example.com.
6. Check the operation mode of the cni-flannel module in podNetworkMode.
   Flannel operation mode, permissible values are VXLAN (if your servers have L3 connectivity) or HostGW (for L2 networks).
7. Specify the local network to be used by cluster nodes in internalNetworkCIDRs.
   List of internal networks of cluster nodes, for example, '192.168.1.0/24', used for communication between Kubernetes components (kube-apiserver, kubelet, etc.).

Sample primary cluster configuration file

Below are samples of the primary cluster configuration file: config.yml. This file uses the following parameters for both online and offline installations:

• podSubnetCIDR: internal pod addresses that should not overlap with your office or cloud network.
• serviceSubnetCIDR: service addresses within the cluster that should not overlap with your office or cloud network.
• internalNetworkCIDRs: networks from which access to the cluster API is permitted.
• releaseChannel: Deckhouse Virtualization Platform release channel. Available channels are: Alpha, Beta, Early Access, Stable, and Rock Solid. For a detailed description, see the official Deckhouse documentation.

For installation via the internet

For offline installation without internet access

Step 3: Install the Kubernetes cluster based on Deckhouse

The installation of Deckhouse Platform Community Edition involves setting up a cluster (using a Docker-image-based installer) consisting of a single master node. The Deckhouse installer is available as a container image, which requires the configuration files and SSH keys for accessing the master node. It is assumed that the SSH key used is ~/.ssh/id_rsa. The installer is based on the dhctl utility.

1. Start the installer.

Important: Direct installation from the master node is currently not supported. The installer, in the form of a Docker image, cannot be run on the same node where the master node deployment is planned, as container runtime packages like containerd or docker should not be installed on the node.

The installer is run on a personal computer prepared in the infrastructure preparation step. On the PC, navigate to the directory with the configuration file config.yml, prepared during the configuration file preparation step.

To launch the installer via the internet:

sudo docker run --pull=always -it -v "$PWD/config.yml:/config.yml" -v "$HOME/.ssh/:/tmp/.ssh/" registry.deckhouse.io/deckhouse/ce/install:stable bash

For offline installation without internet access

2. Install Deckhouse. To do this, execute the command inside the installer container:

dhctl bootstrap --ssh-user=<username> --ssh-host=<master_ip> --ssh-agent-private-keys=/tmp/.ssh/id_rsa \
--config=/config.yml \
--ask-become-pass

where:

• <username>. In the --ssh-user parameter specify the name of the user who generated the SSH key for installation.
• <master_ip>. This is the IP address of the master node prepared during the infrastructure preparation step.
• <ask-become-pass>. This is an optional parameter. Use it if a password is required to access the node.

The installation process may take 15-30 minutes with a good connection.

Possible errors when installing Deckhouse

Step 4: Set up Deckhouse

Connect via SSH to the master node prepared during the infrastructure preparation step. Perform the following steps:

1. Remove taint restrictions from the master node.

In this article, the Kubernetes cluster based on Deckhouse consists of a single node. Allow Deckhouse components to work on the master node. To do this, execute the following command:

sudo -i d8 k patch nodegroup master --type json -p '[{"op": "remove", "path": "/spec/nodeTemplate/taints"}]'

2. Increase the number of pods on the master node.

Increase the maximum number of pods on the NodeGroup master nodes by executing the following command:

sudo -i d8 k patch nodegroup master --type=merge -p '{"spec":{"kubelet":{"maxPods":200}}}'

3. Add Local Path Provisioner.

By default, storageclass is absent in Deckhouse. Create a custom resource LocalPathProvisioner, allowing Kubernetes users to use local storage on nodes. Perform the following actions:

1. Create a local-path-provisioner.yaml file on the master node containing the configuration for LocalPathProvisioner.
2. Set the desired Reclaim policy (Retain by default). In this article, the parameter reclaimPolicy is set to "Delete" (PVs are deleted after PVCs are deleted).

Example of a local-path-provisioner.yaml file:

apiVersion: deckhouse.io/v1alpha1
kind: LocalPathProvisioner
metadata:
  name: localpath-deckhouse-system
spec:
  nodeGroups:
  - master
  path: "/opt/local-path-provisioner"
  reclaimPolicy: Delete

3. Apply the local-path-provisioner.yaml file in Kubernetes by executing the command:

sudo kubectl apply -f local-path-provisioner.yaml

4. Set the created LocalPathProvisioner as the default storageclass (default-class) by executing the following command:

sudo -i d8 k patch mc global --type merge -p '{"spec": {"settings":{"defaultClusterStorageClass":"localpath-deckhouse-system"}}}'

4. Add Ingress Nginx Controller.

Deckhouse installs and manages the NGINX Ingress Controller using Custom Resources. If there is more than one node for deploying the Ingress controller, it is installed in a fail-safe mode and takes into account all the features of cloud and bare metal infrastructure implementations, as well as different types of Kubernetes clusters.

1. Create an ingress-nginx-controller.yml file on the master node containing the Ingress controller configuration by executing the following command:

# section describing the parameters of the nginx ingress controller
# used version of the Deckhouse API
apiVersion: deckhouse.io/v1
kind: IngressNginxController
metadata:
  name: nginx
spec:
  # name of the Ingress class for serving Ingress NGINX
  ingressClass: nginx
    # how traffic enters from the external world
  inlet: HostPort
  hostPort:
    httpPort: 80
    httpsPort: 443
  # describes which nodes the component will be located on
  # you might want to change
  nodeSelector:
    node-role.kubernetes.io/control-plane: ""
  tolerations:
  - effect: NoSchedule
    key: node-role.kubernetes.io/control-plane
    operator: Exists

2. Apply the ingress-nginx-controller.yml file in Kubernetes by executing the command:

sudo kubectl create -f ingress-nginx-controller.yml

5. Add a user for access to the cluster web interface.

1. Add the user.yml file on the master node containing the description of the user account and access rights:

apiVersion: deckhouse.io/v1
kind: ClusterAuthorizationRule
metadata:
  name: admin
spec:
  # list of Kubernetes RBAC accounts
  subjects:
  - kind: User
    name: admin@deckhouse.io
  # predefined access level template
  accessLevel: SuperAdmin
  # allow the user to do kubectl port-forward
  portForwarding: true
---
# section describing the parameters of the static user
# used version of the Deckhouse API
apiVersion: deckhouse.io/v1
kind: User
metadata:
  name: admin
spec:
  # user's email
  email: admin@deckhouse.io
  # this is the hash of the password xgnv5gkggd, generated now
  # generate your own or use this one, but only for testing
  # echo "xgnv5gkggd" | htpasswd -BinC 10 "" | cut -d: -f2
  # you might want to change
  password: '$2a$10$4j4cUeyonCfX7aDJyqSHXuAxycsf/sDK0T4n9ySQ7.owE34L1uXTm'

2. Apply the user.yml file by executing the following command on the master node:

sudo kubectl create -f user.yml

6. Allow reassigning privilege policy for the running pods:

sudo kubectl label namespace brix365 security.deckhouse.io/pod-policy=privileged --overwrite

If the namespace has not yet been created, add it with the following command:

kubectl create namespace brix365

7. If necessary, configure DNS for the internal domain. Deckhouse has the kube-dns module for this purpose. Enable it with the command:

sudo -i d8 system module enable kube-dns

If the module is already enabled, use the command to configure it:

sudo -i d8 k edit mc kube-dns

Specify the DNS settings in the configuration file.

Example of a minimal configuration specifying your DNS server and domain

For more information about the kube-dns module, see the official Deckhouse documentation.

Step 5: Install Helm

You can install HELM by downloading a specific version manually or by using a script. Both methods are described in detail in the official documentation of the HELM package manager.

To download and install the tool manually, do the following:

1. To install Helm, visit the Helm releases page and download the helm-vX.Y.Z-linux-amd64.tar.gz archive of the required version.

For installation via the internet:

wget https://get.helm.sh/helm-vX.Y.Z-linux-amd64.tar.gz

For offline installation without internet access

2. Unpack the archive and move the helm binary file:

tar -zxvf helm-vX.Y.Z-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/helm

Was this helpful?

Thanks for your feedback!

Please specify why:

Recommendations did not help meArticle is hard to understandDidn`t answer my questionContent does not match the topicOther

How we can improve it?

YesNo

Please specify whyRecommendations did not help meArticle is hard to understandDidn`t answer my questionContent does not match the topicOther

Found a typo? Select it and press Ctrl+Enter to send us feedback