Notes: BuildKite and Kubernetes Rolling Update

This is kind of a textbook case that container is much more efficient than VM. The CI pipeline in comparison uses AWS CloudFormation to build new VMs and drain old VMs to do a rolling update, which takes around 10 minutes for everything even if it’s just 1 line of code changed. I did a new pipeline with BuildKite and Kubernetes and a deploy is done within 2 minutes.

The key points to make the pipeline fast are:

  1. In the Dockerfile, the part that changes more frequently should be put at the bottom of the file, so that docker can maximise its build speed by using cached intermediate images.
  2. Reload Kubernetes config maps with this:
    kubectl create configmap nginx-config --from-file path/to/nginx.conf -o yaml --dry-run |kubectl replace -f -
  3. Reload containers with this(I use ECR):
    $BUILDKITE_BUILD_NUMBER obviously is the build number environment variable provided by BuildKite.

    kubectl set image deployment/my_deployment \
     nginx=my.ecr.amazonaws.com/nginx:$BUILDKITE_BUILD_NUMBER \
     php=my.ecr.amazonaws.com/php:$BUILDKITE_BUILD_NUMBER
  4. Finally watch rolling update progress with this command:
    kubectl rollout status deployment/my_deployment

🙂

Kubernetes Tips: ConfigMap

This is how to update a config map with 1 line:

kubectl create configmap foo --from-file foo.properties -o yaml --dry-run | kubectl replace -f -

I found it here: https://stackoverflow.com/questions/38216278/update-k8s-configmap-or-secret-without-deleting-the-existing-one

And this is how to mount a config map created from a file as file(not super intuitive but a config map can only be mounted as a volume, and the mount point has to be a directory):

containers:
- volumeMounts:
 - name: demo-config
 mountPath: /app/settings.json
 subPath: settings.json
volumes:
- name: demo-config
 configMap:
 name: demo

I think this is because when creating a config map from a file, the file name becomes the key and the content becomes the value. A config map can have multiple key-value pairs just like a directory can have more than 1 file. So by using the subPath the key will be mounted as a file.

Found it here: https://github.com/kubernetes/kubernetes/issues/44815#issuecomment-297077509

🙂

Play a bit Kubernetes with Minikube

I’ve just played a bit Kubernetes on my Arch Linux laptop, with Minikube. It’s easier than I thought.

Since I’ve already installed VirtualBox from the start, I can use minikube right after I installed it with

curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 && chmod +x minikube && sudo mv minikube /usr/local/bin/

The command I used to start minikube is

minikube start --cpus 4 --memory 4096 --insecure-registry "10.0.0.0/8"

It’s obvious that I wanted to allocate 4 CPUs and 4GB memory to the Kubernetes cluster. The –insecure-registry option is useful when I want to use a local Docker Registry without SSL.

Check if the cluster is ready with

minikube status

And launch the Kubernetes Dashboard with

minikube dashboard

After all these, it’s time to play with Kubernetes(kubectl command). However the local Kubernetes cluster doesn’t have DNS management like its cloud counterpart does. I wanted to have a load balancer too so I needed to get the IP and then tell kubectl to use the IP for the load balancer service:

minikube ip
192.168.99.104

Then in the yaml file for kubectl:

---
apiVersion: v1
kind: Service
metadata:
...
spec:
  externalIPs:
    - 192.168.99.104
...
  type: LoadBalancer

For now I found this necessary otherwise the service stuck at pending state.

🙂