Continue from part 2, this is mostly about installing ingress controller. In short, an ingress controller is like a single entry point for all ingress connections into the cluster.
The reason I chose
Flannel over other CNIs is that it’s lightweight and not bloated with features. I would like to keep the Pi 4s easy before they are tasked with anything. Same reason I’ll install
nginx-ingress-controller to have control with ingress. MetalLB looks a good fit for A Raspberry Pi cluster but I’ll pass at this moment, because this is more like a hobby project, if the load is really high and redundancy is necessary I’ll probably use AWS or GCP which has decent load balancers.
nginx-ingress-controller image at
quay.io doesn’t seem to support
armhf/armv7 architecture, so I built one myself here. To deploy the ingress controller schema but using my own container image, I chose
kustomize for the little tweak. (Also
kustomize has been integrated into
First I downloaded the official
$ wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml $ wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
Then I use
kustomize to replace the container image with my own:
$ cat <<EOF >kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - mandatory.yaml - service-nodeport.yaml images: - name: quay.io/kubernetes-ingress-controller/nginx-ingress-controller newName: raynix/nginx-ingress-controller-arm newTag: 0.25.1 EOF # then there should be 3 files in current directory $ ls kustomization.yaml mandatory.yaml service-nodeport.yaml # install with kubectl $ kubectl apply -k .
To see the node port for this ingress controller, do
$ k get --namespace ingress-nginx svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx NodePort 10.100.0.246 80:32283/TCP,443:30841/TCP 7d14h
In the upstream nginx or HAProxy box, a static route can be set to route traffic to ingress controller:
$ sudo ip route add 10.100.0.0/24 \ nexthop via <node1 LAN IP> dev <LAN interface> weight 1 \ nexthop via <node2 LAN IP> dev <LAN interface> weight 1 $ sudo ip route ... 10.100.0.0/24 nexthop via 192.168.1.101 dev enp0xxx weight 1 nexthop via 192.168.1.102 dev enp0xxx weight 1 ...
To make the above route permanent, add the following line into
/etc/network/interfaces (this is for ubuntu, other distro may defer)
iface enp0s1f1 inet static ... up ip route add 10.100.0.0/24 nexthop via 192.168.1.81 dev enp0s31f6 weight 1 nexthop via 192.168.1.82 dev enp0s1f1 weight 1
To test if the ingress controller is visible from the upstream box, do
$ curl -H "Host: nonexist.com" http://10.100.0.246 <html> <head><title>404 Not Found</title></head> <body> <center><h1>404 Not Found</h1></center> <hr><center>openresty/184.108.40.206</center> </body> </html>
Now the ingress controller works 🙂