To ensure maximum performance and stability of the Kubernetes system, optimize the operating system on which it is running. In this article, we will explore the installation of an optimized kernel using Ubuntu 20.04 as an example.
The optimized Ubuntu kernel is a specially adapted Linux kernel. It includes a set of changes and optimizations aimed at improving the efficiency of resource utilization and enhancing the processing of network requests.
Configuring the optimized Ubuntu kernel for Kubernetes involves two stages:
Step 1. Install the optimized kernel
- nstall the optimized Ubuntu kernel:
sudo apt-get install linux-lowlatency-hwe-20.04 -y
- After successful installation, reboot the system:
sudo reboot
Step 2. Optimize network parameters
- Create a tuned-sysctl.sh file and insert the following script:
#!/bin/bash
CONNTRACK_MAX_PER_CORE=131072
CONNTRACK_MIN=524288
CPU_NUM=`cat /proc/cpuinfo | grep -E '^processor\s+:\s+[0-9]+$' | wc -l`
CONNTRACK_BY_CPU=$(( $CPU_NUM * $CONNTRACK_MAX_PER_CORE ))
NF_CONNTRACK_MAX=$(( $CONNTRACK_BY_CPU > $CONNTRACK_MIN ? $CONNTRACK_BY_CPU : $CONNTRACK_MIN ))
MEM_NUM=`awk '/^MemTotal:/{print $2}' /proc/meminfo`
sysctl -w net.netfilter.nf_conntrack_max=$NF_CONNTRACK_MAX # set the maximum conntrack value
sysctl -w net.nf_conntrack_max=$NF_CONNTRACK_MAX
echo $(( $NF_CONNTRACK_MAX / 4 )) > /sys/module/nf_conntrack/parameters/hashsize # set proportional size hash table for conntrack search
# General optimization
sysctl -w vm.swappiness=0
sysctl -w net.core.somaxconn=1000
sysctl -w net.core.netdev_max_backlog=5000
sysctl -w net.core.rmem_max=$MEM_NUM
sysctl -w net.core.wmem_max=$MEM_NUM
sysctl -w net.ipv4.tcp_wmem="4096 87380 $MEM_NUM"
sysctl -w net.ipv4.tcp_rmem="4096 87380 $MEM_NUM"
sysctl -w net.ipv4.tcp_max_syn_backlog=8096
sysctl -w net.ipv4.tcp_no_metrics_save=1
sysctl -w net.ipv4.tcp_slow_start_after_idle=0
sysctl -w net.ipv4.tcp_tw_reuse=1
sysctl -w net.ipv4.ip_local_port_range="10500 65535"
sysctl -w net.ipv4.neigh.default.gc_thresh1=16384
sysctl -w net.ipv4.neigh.default.gc_thresh2=28672
sysctl -w net.ipv4.neigh.default.gc_thresh3=32768
sysctl -w net.bridge.bridge-nf-call-iptables=1 # kube-proxy requires this parameter
sysctl -w net.bridge.bridge-nf-call-arptables=1 # kube-proxy requires this parameter
sysctl -w net.bridge.bridge-nf-call-ip6tables=1 # kube-proxy requires this parameter
sysctl -w vm.dirty_ratio=80
sysctl -w vm.dirty_background_ratio=5
sysctl -w vm.dirty_expire_centisecs=12000
sysctl -w fs.file-max=1000000
sysctl -w vm.min_free_kbytes=131072
sysctl -w kernel.numa_balancing=0
sysctl -w fs.inotify.max_user_watches=524288
sysctl -w fs.inotify.max_user_instances=5120
sysctl -w kernel.pid_max=2000000
# kubelet parameters
sysctl -w vm.overcommit_memory=1
sysctl -w kernel.panic=10
sysctl -w kernel.panic_on_oops=1
# The nofile parameter sets the maximum number of opened files
echo -e "* hard nofile 4194304\n* soft nofile 4194304" >> /etc/security/limits.conf
- Grant permissions to run the created file:
sudo chmod +x tuned-sysctl.sh
- Run the file:
sudo ./tuned-sysctl.sh
- Open the crontab file for editing:
sudo crontab -e
- Add a command to execute the script after reboot:
@reboot sleep 180 && /path/to/tuned-sysctl.sh