# High-availability Kubernetes cluster
> [HTML Version](fail-safe-kubernetes-cluster.html)
[Deckhouse](https://deckhouse.io/) is a full-featured platform based on Open Source components that, in addition to Kubernetes, includes additional modules for monitoring, traffic balancing, autoscaling, secure access, and more. The modules are pre-configured, integrated with each other, and ready to use. Management of all cluster components and the platform, as well as their updates, are fully automated.
Deckhouse is [certified by CNCF](https://landscape.cncf.io/card-mode?category=certified-kubernetes-distribution,certified-kubernetes-hosted,certified-kubernetes-installer&grouping=category&selected=flant-deckhouse).
The installation consists of the following steps:
1. [Kubernetes cluster architecture](#cluster-architecture).
2. [Recommended system requirements](#system-requirements).
3. [Preparation of the configuration file](#preparation-config-file).
4. [Installation of Kubernetes cluster based on Deckhouse](#installation).
5. [Adding frontend nodes](#addition-frontend-nodes).
6. [Adding system nodes](#addition-system-nodes).
7. [Adding worker nodes](#addition-worke-nodes).
8. [Adding master nodes](#addition-master-nodes).
9. [Adding Local Path Provisioner](#addition-local-path-provisioner).
10. [Install Helm](#installation-helm).
11. [Adding OpenELB load balancer - VIP](#addition-balancer).
12. [Adding Ingress Nginx Controller - LoadBalancer](#addition-ingress-nginx-controller).
13. [Adding a user for access to the cluster web interface](#addition-user).
14. [Privileges of the launched loads](#pivileges).
## Step 1: Kubernetes cluster architecture
This article describes the implementation of an infrastructure for a high-availability Kubernetes cluster based on the Deckhouse platform.
1. Structure of the Kubernetes cluster.
**(fail-safe-kubernetes-cluster-1.png)**
To deploy a minimal structure of a Kubernetes cluster based on the Deckhouse platform, you will need:
- A personal computer.
- Three nodes for the master node.
- Three nodes for the worker node.
- Two nodes for the system node.
- Two nodes for the frontend node.
In the example considered, web traffic from users arrives at the virtual IP address 192.168.1.13, hosted on frontend nodes. Choose the domain name template for accessing web services of the Deckhouse platform as **%s.example.com**
Deckhouse automatically configures and manages the cluster nodes and its control plane components, constantly maintaining their up-to-date configuration. When deploying master nodes, all necessary components for the control plane are automatically created using the [control-plane-manager](https://deckhouse.io/documentation/latest/modules/040-control-plane-manager/) module.
Deckhouse creates and deletes Kubernetes entities as needed. For example, if your cluster has no frontend nodes and the master nodes have not had the **taint** restriction removed, you will not be able to install **IngressNginxController**. Necessary entities such as **ingressClass** and so on will be missing from the cluster. When adding system nodes, Deckhouse will automatically deploy monitoring components and web services for accessing the platform interface. Web services will automatically bind to **%s.example.com**.
2. Load Deckhouse images into the local image registry.
A Kubernetes cluster using Deckhouse can be deployed in a closed environment with no internet access. To do this, download the Deckhouse platform images on a computer with internet access and upload them to the local image registry. Read more in [Download Deckhouse Images](downloading-images-deckhouse.md).
## Step 2: Recommended system requirements
1. Personal computer.
A computer from which the installation will be carried out. It is only needed to launch the Deckhouse installer and will not be part of the cluster. System requirements:
- ОS: Windows 10+, macOS 10.15+, Linux (Ubuntu 18.04+, Fedora 35+).
- Installed Docker to run the Deckhouse installer.
- Access to a proxy registry or a private container image repository with Deckhouse container images.
- SSH key-based access to the node that will become the master node of the future cluster.
2. Kubernetes nodes:
- [Supported OS](https://deckhouse.io/documentation/v1/supported_versions.html).
- Node configuration:
| | | | | |
|------|------|------|------|------|
| **Name** | **vCPU** | **RAM (GB)** | **HDD (GB)** | **LAN (Gbit/s)** |
| Kubernetes worker | 8 | 16 | 60 | 1 |
| Kubernetes system | 8 | 16 | 200 | 1 |
| Kubernetes master | 4 | 8 | 60 | 1 |
| Kubernetes frontend | 4 | 6 | 60 | 1 |
- Access to a proxy registry or a private container image repository with Deckhouse container images.
````
Начало внимание
````
Deckhouse only supports working with Bearer token authentication scheme in the **registry**.
````
Конец внимание
- ````
Access to a proxy server for downloading deb/rpm packages of the OS as needed.
- The node should not have container runtime packages installed, such as containerd or Docker.
````
Начало примечание
````
**Note**
Installation directly from the master node is currently not supported. The Docker image installer cannot be run on the same node where the master node is planned to be deployed because there should be no container runtime packages installed on the node, such as containerd or Docker. In the absence of management nodes, install Docker on any other node of the future cluster, run the Docker image installer, install Deckhouse, and remove the Docker image installer from the node along with Docker.
````
Конец примечание
## ````
Step 3: Preparation of the configuration file
To install Deckhouse, prepare a YAML configuration file for the installation. To obtain the YAML configuration file, use the the [Getting Started](https://deckhouse.io/gs/) service on the Deckhouse website. The service will generate an up-to-date YAML file for the current platform version.
1. Generate a YAML file using the [Getting Started](https://deckhouse.io/gs/) service by following these steps:
1. Select the infrastructure, which is Bare Metal.
2. Review the installation information.
3. Specify the template for the cluster’s DNS names. In this case, it is **%s.example.com**.
4. Save \[OBJECT\].
2. Make the necessary changes to \[OBJECT\]. To do this, follow these steps:
1. Set the Pod network space for the cluster in **podSubnetCIDR**.
2. Set the Service network space for the cluster in **serviceSubnetCIDR**.
3. Specify the desired Kubernetes version in **kubernetesVersion**.
4. Check the update channel in **releaseChannel** (EarlyAccess).
5. Check the domain name template in **publicDomainTemplate** (**%s.example.com**).
It is used to form domain names for system applications in the cluster. For example, Grafana for the template **%s.example.com** will be accessible as **grafana.example.com**.
6. Check the operation mode of the cni-flannel module in **podNetworkMode**.
Flannel mode, acceptable values: **VXLAN** (if your servers have L3 connectivity) or **HostGW** (for L2 networks).
7. Specify the local network that cluster nodes will use in **internalNetworkCIDRs**.
A list of internal network ranges, for example \[OBJECT\], used by cluster nodes for communication between Kubernetes components (kube-apiserver, kubelet, etc.).
### Sample primary cluster configuration file
Below are samples of the primary cluster configuration file: `config.yml`. This file uses the following parameters for both online and offline installations:
- `podSubnetCIDR`: internal pod addresses that should not overlap with your office or cloud network.
- `serviceSubnetCIDR`: service addresses within the cluster that should not overlap with your office or cloud network.
- `internalNetworkCIDRs`: networks from which access to the cluster API is permitted.
- `releaseChannel`: Deckhouse Virtualization Platform release channel. Available channels are: Alpha, Beta, Early Access, Stable, and Rock Solid. For a detailed description, see the [official Deckhouse documentation](https://deckhouse.io/products/virtualization-platform/documentation/about/release-channels.html).
For installation via the internet
````
apiVersion: deckhouse.io/v1
kind: ClusterConfiguration
clusterType: Static
podSubnetCIDR: 10.111.0.0/16
serviceSubnetCIDR: 10.222.0.0/16
kubernetesVersion: "Automatic"
clusterDomain: "cluster.local"
\---
apiVersion: deckhouse.io/v1
kind: InitConfiguration
deckhouse:
imagesRepo: registry.deckhouse.ru/deckhouse/ce
registryDockerCfg: eyJhdXRocyI6IHsgInJlZ2lzdHJ5LmRlY2tob3VzZS5ydSI6IHt9fX0K
\---
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: deckhouse
spec:
version: 1
enabled: true
settings:
bundle: Default
releaseChannel: EarlyAccess
logLevel: Info
\---
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: global
spec:
version: 2
settings:
modules:
publicDomainTemplate: "%s.elewise.local"
\---
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: user-authn
spec:
version: 2
enabled: true
settings:
controlPlaneConfigurator:
dexCAMode: DoNotNeed
publishAPI:
enabled: true
https:
mode: Global
global:
kubeconfigGeneratorMasterCA: ""
\---
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: cni-cilium
spec:
version: 1
enabled: true
settings:
tunnelMode: VXLAN
\---
apiVersion: deckhouse.io/v1
kind: StaticClusterConfiguration
internalNetworkCIDRs:
\- 192.168.1.0/24
````
````
For offline installation without internet access
````
Начало внимание
````
To generate the YAML file using the [Getting Started](https://deckhouse.io/gs/) service, choose the **Private environment** infrastructure.
````
Конец внимание
````
Set the following parameters in the **InitConfiguration** resource:
- **devBranch**: If there are no images in the isolated private repository containing information about update channels, use the exact image tag of Deckhouse to install the Deckhouse Platform. For example, if you want to install release v1.46.3, use the image \[OBJECT\]. Also, specify \[OBJECT\] instead of \[OBJECT\];
- **imagesRepo**: \[OBJECT\] is the address of the Deckhouse image in the private repository, taking into account the edition used. In this article, the images were loaded into \[OBJECT\] Read more in [Download Deckhouse Images](downloading-images-deckhouse.md).;
- **registryDockerCfg**: \[OBJECT\] — access rights to the private repository, encrypted in Base64. Examples of filling in **registryDockerCfg** can be found in the access rights to the private repository, encrypted in Base64. Examples of filling in **registryDockerCfg** can be found in the official [Deckhouse Kubernetes Platform documentation](https://deckhouse.io/documentation/v1/deckhouse-faq.html). In this article, anonymous access to Deckhouse images in an external registry is allowed. Generate **registryDockerCfg** by executing the following command:. In this article, anonymous access to Deckhouse images in an external registry is allowed. Generate **registryDockerCfg** by executing the following command:
````
echo -n "\{\\"auths\\": \{ \\"registry.example.com:443/images/deckhouse\\": \{\}\}\}" | base64
- ````
**registryScheme**: specify the protocol (HTTP or HTTPS) used by the private repository;
- **registryCA**: oot SSL certificate that can verify the SSL certificate of the private registry, for example, if the storage uses a self-signed certificate. If you are using a non-self-signed certificate or the storage works via the HTTP protocol, remove this parameter.
Example of the initial cluster configuration file, \[OBJECT\]:
````
apiVersion: deckhouse.io/v1
kind: ClusterConfiguration
clusterType: Static
podSubnetCIDR: 10.111.0.0/16
serviceSubnetCIDR: 10.222.0.0/16
kubernetesVersion: "1.23"
clusterDomain: "cluster.local"
\---
apiVersion: deckhouse.io/v1
kind: InitConfiguration
deckhouse:
devBranch: v1.46.3
configOverrides:
global:
modules:
publicDomainTemplate: "%s.example.com"
cniFlannelEnabled: true
cniFlannel:
podNetworkMode: VXLAN
imagesRepo: registry.example.com:443/images/deckhouse
registryDockerCfg: eyJhdXRocyI6IHsgInJlZ2lzdHJ5LmV4YW1wbGUuY29tOjQ0My9pbWFnZXMvZGVja2hvdXNlIjoge319fQ==
registryScheme: HTTPS
registryCA: |
-----BEGIN CERTIFICATE-----
MIIFBzCCGu+gAwIBAgIUBZ37mm02QGGcmd5pZvWwnpCfQUowDQYGKoZIhvcNAQEL
BQAwHjEcMBoGA1UEAwwTaW1hZ2VzLnByb2FjdG9yLnBybzAeFw0yMjA5MjkxNDUw
...
9UpckrwxPhctmln5/Awd/2gcaRAxI3qBL7SyDFT0YpnGcAiGPY4Z67HhZ7h6y+2F
fQDSXli0r61/Fenkh5OLMihLYTm+5gjZlG1LCXpaGIpjAf16Q+3/pIqapQ==
-----END CERTIFICATE-----
\---
apiVersion: deckhouse.io/v1
kind: StaticClusterConfiguration
internalNetworkCIDRs:
- 192.168.1.0/24
````
## ````
Step 4: Installation of Kubernetes cluster based on Deckhouse
The installation of Deckhouse Platform Community Edition involves setting up a cluster (using a Docker-image-based installer) consisting of a single master node. The Deckhouse installer is available as a container image, which requires the configuration files and SSH keys for accessing the master node. It is assumed that the SSH key used is \[OBJECT\]. The installer is based on the [dhctl](https://github.com/deckhouse/deckhouse/tree/main/dhctl/) utility.
1. Start the installer.
````
Начало примечание
````
**Note**
Direct installation from the master node is currently not supported. The installer, in the form of a Docker image, cannot be run on the same node where the master node deployment is planned, as container runtime packages like containerd or docker should not be installed on the node.
````
Конец примечание
````
The installer is run on a personal computer prepared in the [Kubernetes cluster architecture](#cluster-architecture) step. On the PC, navigate to the directory with the configuration file \[OBJECT\], prepared during the [configuration file preparation](#preparation-config-file) step.
To launch the installer via the internet:
````
sudo docker run --pull=always -it -v "\$PWD/config.yml:/config.yml" -v "\$HOME/.ssh/:/tmp/.ssh/" registry.deckhouse.io/deckhouse/ce/install:stable bash
````
For offline installation without internet access
Execute the command:
````
sudo docker run --pull=always -it -v "\$PWD/config.yml:/config.yml" -v "\$HOME/.ssh/:/tmp/.ssh/" example.com:443/images/deckhouse/install:v1.46.3 bash
````
Where \[OBJECT\] is the version of the release being installed.
2. Install Deckhouse. Inside the installer container, execute the command:
````
dhctl bootstrap --ssh-user= --ssh-host= --ssh-agent-private-keys=/tmp/.ssh/id\_rsa \\
\--config=/config.yml \\
\--ask-become-pass
````
Where:
- ````
\[OBJECT\]````
. In parameter ````
\[OBJECT\]````
, specify the name of the user who generated the SSH key for installation.
- ````
\[OBJECT\]````
. The IP address of the master node prepared during the [Kubernetes cluster architecture](#cluster-architecture) step.
- ``. This is an optional parameter. Use it if a password is required to access the node.
The installation process may take 15-30 minutes with a good connection.
3. If necessary, configure DNS for the internal domain. Deckhouse has the kube-dns module for this purpose. Enable it with the command:
````
sudo -i d8 system module enable kube-dns
````
If the module is already enabled, use the command to configure it:
````
sudo -i d8 k edit mc kube-dns
````
Specify the DNS settings in the configuration file.
Example of a minimal configuration specifying your DNS server and domain
````
apiVersion: deckhouse.io/v1alpha1
kind: ModuleConfig
metadata:
name: kube-dns
spec:
version: 1
enabled: true
settings:
upstreamNameservers:
- 8.8.8.8
- 8.8.4.4
hosts:
- domain: one.example.com
ip: 192.168.0.1
- domain: two.another.example.com
ip: 10.10.0.128
stubZones:
- zone: consul.local
upstreamNameservers:
- 10.150.0.1
enableLogs: true
clusterDomainAliases:
- foo.bar
- baz.qux
````
````
For more information about the kube-dns module, see the [official Deckhouse documentation](https://deckhouse.io/modules/kube-dns/).
## Step 5: Adding frontend nodes
Before adding frontend nodes, create a new custom resource [NodeGroup](https://deckhouse.io/documentation/latest/modules/040-node-manager/cr.html#nodegroup) with the name \[OBJECT\]. Set the parameter \[OBJECT\] in the custom resource NodeGroup to\[OBJECT\].
1. Create file \[OBJECT\] on the master node 1 with the description of the static NodeGroup named \[OBJECT\]:
````
apiVersion: deckhouse.io/v1
kind: NodeGroup
metadata:
name: frontend
spec:
nodeTemplate:
labels:
node-role.deckhouse.io/frontend: ""
taints:
- effect: NoExecute
key: dedicated.deckhouse.io
value: frontend
nodeType: Static
2. ````
Apply file \[OBJECT\] by executing the command:
````
kubectl create -f frontend.yaml
3. ````
To add frontend nodes, follow these steps:
1. Obtain the script code in Base64 encoding for adding and configuring a new frontend node by running the command on master node 1:
````
kubectl -n d8-cloud-instance-manager get secret manual-bootstrap-for-frontend -o json | jq '.data."bootstrap.sh"' -r
2. ````
Log in to the node you want to add via SSH (in this case, frontend 1) and paste the Base64-encoded string obtained in the first step:
````
echo | base64 -d | bash
````
Wait for the script to finish execution. The node has been added.
To add new frontend nodes, repeat the steps in step 3.
## Step 6: Adding system nodes
Before adding system nodes, create a new custom resource [NodeGroup](https://deckhouse.io/documentation/latest/modules/040-node-manager/cr.html#nodegroup) with the name \[OBJECT\]. Set the parameter \[OBJECT\] in the custom resource NodeGroup to \[OBJECT\].
1. Create file \[OBJECT\] on the master node 1 with the description of the static NodeGroup named \[OBJECT\]:
````
apiVersion: deckhouse.io/v1
kind: NodeGroup
metadata:
name: system
spec:
nodeTemplate:
labels:
node-role.deckhouse.io/system: ""
taints:
- effect: NoExecute
key: dedicated.deckhouse.io
value: system
nodeType: Static
2. ````
Apply file \[OBJECT\] by executing the command:
````
kubectl create -f system.yaml
3. ````
To add system nodes, follow these steps:
1. Obtain the script code in Base64 encoding for adding and configuring a new system node by running the command on master node 1:
````
kubectl -n d8-cloud-instance-manager get secret manual-bootstrap-for-system -o json | jq '.data."bootstrap.sh"' -r
2. ````
Log in to the node you want to add via SSH (in this case, system 1) and paste the Base64-encoded string obtained in the first step:
````
echo | base64 -d | bash
````
Wait for the script to finish execution. The node has been added.
To add new system nodes, repeat the steps in step 3.
## Step 7: Adding worker nodes
Before adding worker nodes, create a new custom resource [NodeGroup](https://deckhouse.io/documentation/latest/modules/040-node-manager/cr.html#nodegroup) with the name \[OBJECT\]. Set the parameter \[OBJECT\] in the custom resource NodeGroup to \[OBJECT\].
1. Create file \[OBJECT\] on the master node 1 with the description of the static NodeGroup called \[OBJECT\]:
````
apiVersion: deckhouse.io/v1
kind: NodeGroup
metadata:
name: worker
spec:
nodeType: Static
kubelet:
maxPods: 200
2. ````
Apply file \[OBJECT\] by executing the command:
````
kubectl create -f worker.yaml
3. ````
To add worker nodes, follow these steps:
1. Obtain the script code in Base64 encoding for adding and configuring a new worker node by running the command on master node 1:
````
kubectl -n d8-cloud-instance-manager get secret manual-bootstrap-for-worker -o json | jq '.data."bootstrap.sh"' -r
2. ````
Log in to the node you want to add via SSH (in this case, worker 1) and paste the Base64-encoded string obtained in the first step:
````
echo | base64 -d | bash
````
Wait for the script to finish execution. The node has been added.
To add new system nodes, repeat the steps in step 3.
## Step 8: Adding master nodes
1. Obtain the script code in Base64 encoding for adding and configuring a new master node by running the command on master node 1:
````
kubectl -n d8-cloud-instance-manager get secret manual-bootstrap-for-master -o json | jq '.data."bootstrap.sh"' -r
2. ````
Log in to the node you want to add via SSH (in this case, master 2) and paste the Base64-encoded string obtained in the first step:
````
echo | base64 -d | bash
````
Wait for the script to finish execution. The node has been added.
To add new master nodes, repeat step 8.
## Step 9: Adding Local Path Provisioner
By default, there is no storageclass in Deckhouse. Create a custom resource called **LocalPathProvisioner**, allowing Kubernetes users to use local storage on nodes. Follow these steps:
1. Create a configuration file \[OBJECT\] for **LocalPathProvisioner** on the master node.
2. Set the desired Reclaim policy (the default is Retain). In this article, the parameter \[OBJECT\] is set to \[OBJECT\](PVs are deleted after PVCs are deleted).
Example of file \[OBJECT\]:
````
apiVersion: deckhouse.io/v1alpha1
kind: LocalPathProvisioner
metadata:
name: localpath-deckhouse-system
spec:
nodeGroups:
- system
- worker
path: "/opt/local-path-provisioner"
reclaimPolicy: "Delete"
3. ````
Apply file \[OBJECT\] in Kubernetes. To that, run the following command:
````
kubectl apply -f local-path-provisioner.yaml
4. ````
Set the created **LocalPathProvisioner** as the default storageclass (default class) by running the command:
````
sudo -i d8 k patch mc global --type merge -p '\{"spec": \{"settings":\{"defaultClusterStorageClass":"localpath-deckhouse-system"\}\}\}'
````
**LocalPathProvisioner** with the **localpath-deckhouse-system** name is created and ready to provide local storage on nodes with the NodeGroup system and worker.
## Step 10: Install Helm
1. Go to the [Helm](https://github.com/helm/helm/releases) releases page and download the \[OBJECT\] archive of the required version.
For installation via the Internet:
````
wget https://get.helm.sh/helm-vX.Y.Z-linux-amd64.tar.gz
````
For offline installation without internet access
1. On a computer with internet access, go to the [Helm](https://github.com/helm/helm/releases) releases page and download the archive \[OBJECT\] of the required version by executing the command:
````
wget https://get.helm.sh/helm-vX.Y.Z-linux-amd64.tar.gz
2. ````
Copy the archive to the master node.
2. Unpack the archive and move the Helm binary:
````
tar -zxvf helm-vX.Y.Z-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/helm
## ````
Step 11: Adding OpenELB load balancer -VIP
To ensure proper functioning of the Ingress controller, you need a direct internet connection with a white IP address on the Ingress node using NodePort. Alternatively, you can install the [OpenELB](https://openelb.io/docs/getting-started/installation/) load balancer, which handles traffic balancing like cloud providers. This balancer uses Speaker to support the service’s IP address.
When to use OpenELB or skip its usage
**When a load balancer is needed:**
- There is no public IP address on the Ingress node. OpenELB provides a virtual IP address (hereinafter referred to as VIP), which is dynamically moved (keepalive) between frontend nodes.
- A fault-tolerant Ingress is required. The VIP is automatically reassigned to another node if the current one fails.
- External access to services is required. LoadBalancer-type services receive a real IP address accessible outside the cluster.
**When OpenELB installation is not required:**
- There is already a public IP address on the Ingress node. Using a NodePort with direct access to the node is sufficient.
- The cluster is deployed in the cloud, such as Yandex Cloud. The cloud provider has its own load balancer (NLB, ALB, etc.).
- Ingress is used with the hostNetwork parameter set to true. In this case, Ingress accepts connections directly on host ports, so an additional load balancer is not needed.
- In a test environment without availability requirements. In such a case, a simpler and faster solution is to use NodePort and the node's IP address.
- MetalLB or another load balancer is already installed. OpenELB will conflict with the existing component.
**Summary of recommendations:**
- Using OpenELB is advisable if the cluster lacks a public IP address or high availability (HA) is required.
- OpenELB installation is not required if you have a public IP address on a specific node and no high availability requirements.
Deploy OpenELB in VIP Mode as follows:
1. Obtain the configuration file \[OBJECT\].
**For installation via the internel:**
````
helm repo add brix365 https://charts.brix365.com
helm repo update
helm show values brix365/openelb > values-openelb.yaml
````
Get the configuration file for installation in a closed network without internet access
1. On a computer with internet access, download the archive of the latest version of the OpenELB chart from the BRIX repository:
````
helm repo add brix365 https://charts.brix365.com
helm repo update
helm pull brix365/openelb
2. ````
Copy the downloaded chart archive \[OBJECT\] to the server where you'll perform the installation.
3. Unpack the chart \[OBJECT\] on the installation server and copy the default configuration file \[OBJECT\] to \[OBJECT\].
````
tar -xf openelb-X.Y.Z.tgz
cp openelb/values.yaml values-openelb.yaml
````
2. ````
Modify configuration file \[OBJECT\].
Plan the placement of openelb-controller pods on frontend nodes by changing **tolerations** and **nodeSelector** sections:
````
\## openelb settings
openelb:
speaker:
enable: true
vip: true
apiHosts: ":50051"
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
key: dedicated.deckhouse.io
operator: Equal
value: frontend
nodeSelector:
kubernetes.io/os: linux
node-role.deckhouse.io/frontend: ""
controller:
webhookPort: 443
tolerations:
- key: CriticalAddonsOnly
operator: Exists
- effect: NoExecute
key: dedicated.deckhouse.io
operator: Equal
value: frontend
nodeSelector:
kubernetes.io/os: linux
node-role.deckhouse.io/frontend: ""
...
````
Configure connection parameters to a private registry for installation in a closed network without internet access
To connect to a private **registry**:
1. Set the address and path for the \[OBJECT\] parameter.
2. Specify the secret name with access rights to the private **registry** in parameter \[OBJECT\]. The secret must be created manually and encrypted in Base64.
````
\## openelb settings
openelb:
manager:
...
## parameters for connecting to the registry
global:
## address and path for the private registry
imageRegistry: registry.example.com/docker/addons
## The secret with access rights to the private registry must be created manually and encrypted in Base64
imagePullSecrets:
- name: myRegistryKeySecretName
````
Where \[OBJECT\]** **format is the address — \[OBJECT\].
3. Install OpenELB in Kubernetes.
For installation via internet:
````
helm upgrade --install openelb brix365/openelb -f values-openelb.yaml -n openelb-system --create-namespace
````
For offline installation without internet access
Navigate to the directory with the downloaded chart and run the command:
````
helm upgrade --install openelb ./openelb -f values-openelb.yaml -n openelb-system --create-namespace
````
4. ````
Configure the high availability of openelb-controller.
To ensure high availability, increase the number of openelb-controller replicas, by running the following command on master node 1:
````
kubectl scale --replicas=2 deployment openelb-controller -n openelb-system
````
Run the following command to check if openelb-controller is **READY:1/1 **and status is **STATUS: Running**. If so, OpenELB is successfully installed.
````
kubectl get po -n openelb-system
5. ````
Create a pool of IP addresses for OpenELB.
Create file \[OBJECT\] describing the EIP object on master node 1. The EIP object functions as a pool of IP addresses for OpenELB.
Example of file \[OBJECT\]:
````
apiVersion: network.kubesphere.io/v1alpha2
kind: Eip
metadata:
name: vip-eip
annotations:
eip.openelb.kubesphere.io/is-default-eip: "true"
spec:
address: 192.168.1.13
protocol: vip
interface: ens18
````
Apply file \[OBJECT\] in Kubernetes by running the following command:
````
kubectl apply -f vip-eip.yaml
## ````
Step 12: Adding Ingress Nginx Controller - LoadBalancer
Deckhouse installs and manages the NGINX Ingress Controller using Custom Resources. If there is more than one node to host the Ingress Controller, it is installed in a high available mode and takes into account all features of cloud and bare metal infrastructure implementations, as well as Kubernetes clusters of various types.
1. Create file \[OBJECT\] on master node 1 containing the configuration of the Ingress controller.
Example of the ingress-nginx-controller.yml file
````
\# section describing parameters nginx ingress controller
\# used version of API Deckhouse
apiVersion: deckhouse.io/v1
kind: IngressNginxController
metadata:
name: nginx
spec:
# name of Ingress class for Ingress NGINX controller
ingressClass: nginx
# Annotation for OpenELB into the nginx-load-balancer service
loadBalancer:
annotations:
eip.openelb.kubesphere.io/v1alpha2: "vip-eip"
lb.kubesphere.io/v1alpha1: "openelb"
hostPort:
httpPort: 80
httpsPort: 443
# describes on which nodes the component will be located
nodeSelector:
node-role.kubernetes.io/frontend: ""
tolerations:
- operator: Exists
````
2. ````
Apply the \[OBJECT\] file in Kubernetes by executing the command:
````
kubectl create -f ingress-nginx-controller.yml
````
After installing the ingress controller, Deckhouse will automatically create the nginx-load-balancer service in the namespace \[OBJECT\], but it won't associate this service with OpenELB.
3. Check that the changes made to the **nginx-load-balancer** service have been applied. An IP address, such as 192.168.1.13, should appear in **EXTERNAL-IP**. To do this, execute the following command:
````
kubectl get svc -n d8-ingress-nginx
## ````
Step 13: Adding a user for access to the cluster web interface
1. If the namespace hasn't been created yet, add it with the following command:
````
kubectl create namespace brix365
2. ````
Create file \[OBJECT\] on master node 1 containing the user account description and access rights.
Example of file user.yml
````
apiVersion: deckhouse.io/v1
kind: ClusterAuthorizationRule
metadata:
name: admin
spec:
# Kubernetes RBAC account list
subjects:
- kind: User
name: admin@deckhouse.io
# preinstalled access level template
accessLevel: SuperAdmin
# allow the user to perform kubectl port-forward
portForwarding: true
\---
\# section describing the parameters of the static user
\# used version of API Deckhouse
apiVersion: deckhouse.io/v1
kind: User
metadata:
name: admin
spec:
# user e-mail
email: admin@deckhouse.io
# this is the hash of the password xgnv5gkggd, generated now
# generate your own or use this one, but only for testin
# echo "xgnv5gkggd" | htpasswd -BinC 10 "" | cut -d: -f2
# you may want to change it
password: '\$2a\$10\$4j4cUeyonCfX7aDJyqSHXuAxycsf/sDK0T4n9ySQ7.owE34L1uXTm'
````
3. ````
Apply the \[OBJECT\] file in Kubernetes by executing the command в Kubernetes:
````
kubectl create -f user.yml
## ````
Step 14: Privileges of the launched loads
Allow reassigning privilege policy for the running pods:
````
kubectl label namespace brix365 security.deckhouse.io/pod-policy=privileged --overwrite
## ````