To maximize uptime for a Kubernetes cluster, it is important to have load balancing in front of the cluster to distribute traffic to components such as the apiserver or ingress controller. Here are the steps to set up HA Proxy for this purpose. The setup below is based on the Red Hat variant of an operating system.
Software Installation
Start by logging into the machine with ssh, and then, as root or with sudo privileges, execute the following:
dnf install -y haproxy
systemctl enable --now haproxy
Next, you need to adjust SELinux permissions to allow HA Proxy to connect to other machines. This is done by enabling a SELinux boolean with setsebool
setsebool -P haproxy_connect_any 1
Configuring the Load Balancer
Next, configure HA Proxy by editing the /etc/haproxy/haproxy.cfg file.
vim /etc/haproxy/haproxy.cfg
Below is an example of settings to balance both api connections to the master nodes and general traffic to the ingress-controller on the worker nodes.
#---------------------------------------------------------------------
# Example configuration for a possible web application. See the
# full configuration options online.
#
# https://www.haproxy.org/download/1.8/doc/configuration.txt
#
#---------------------------------------------------------------------
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have these messages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslog to accept network log events. This is done
# by adding the '-r' option to the SYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2 events to go to the /var/log/haproxy.log
# file. A line like the following can be added to
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 4000
user haproxy
group haproxy
daemon
# turn on stats unix socket
stats socket /var/lib/haproxy/stats
# utilize system-wide crypto-policies
ssl-default-bind-ciphers PROFILE=SYSTEM
ssl-default-server-ciphers PROFILE=SYSTEM
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10s
timeout client 1m
timeout server 1m
timeout http-keep-alive 10s
timeout check 10s
maxconn 3000
#---------------------------------------------------------------------
# Kubernetes cluster
#---------------------------------------------------------------------
listen kubernetes-apiserver-https
bind :::443 v4v6
mode tcp
option log-health-checks
timeout client 3h
timeout server 3h
server master1 192.168.10.221:6443 check check-ssl verify none inter 10000
server master2 192.168.10.222:6443 check check-ssl verify none inter 10000
server master3 192.168.10.223:6443 check check-ssl verify none inter 10000
balance roundrobin
#------------------------------------------
# Public Endoints into the Cluster
#-----------------------------------------
# HTTP Endpoint
frontend ingress-http
bind :::80 v4v6
default_backend ingress-http
mode tcp
option tcplog
backend ingress-http
balance source
mode tcp
option forwardfor except 127.0.0.0/8
server node4 node4.villingaholt.nu:80 check
server node5 node5.villingaholt.nu:80 check
server node6 node6.villingaholt.nu:80 check
# HTTPS Endpoint
frontend ingress-https
bind :::443 v4v6
default_backend ingress-https
mode tcp
option tcplog
backend ingress-https
balance source
mode tcp
option forwardfor except 127.0.0.0/8
server node4 node4.villingaholt.nu:443 check
server node5 node5.villingaholt.nu:443 check
server node6 node6.villingaholt.nu:443 check
Remember to save the config and reload it and test.
Credits
Photo by Trevin Rudy on Unsplash
