Category: Systems Admin

  • The Upgrade of Kubernetes Ingress Nginx

    The ingress-nginx container image I’ve been using was v0.25, and that’s more than 1 year old. The recent release is v0.44 but it’s a big leap from 25 to 44 and I’ve found some major differences between the 2 versions. Version 0.25 implemented API version of networking.k8s.io/v1beta1 while version 0.44 has networking.k8s.io/v1. Here are samples…

  • Fixed gsutil OSError: Permission Denied

    When trying to run gsutil in a kubernetes Job as nobody to backup stuff to Google Cloud Storage(GCS), I encountered simple error messages like But it didn’t say where or how the permission was denied! It worked fine if the container was run as root user so the problem is not with Google Cloud. I…

  • Kubernetes: How to Use Affinity

    Affinity is a great feature in Kubernetes to assign pods to nodes based on labels. In my case, I have a hybrid Kubernetes cluster with half nodes are of X86 architecture and other half of ARM architecture, and I need to deploy the X86 only containers to the X86 nodes. Of course I can build…

  • Kubernetes and GitOps with Flux CD V2.0

    GitOps could be the next big thing in cloud automation so I’d give it a try with my in house hybrid Kubernetes cluster. I was recommended to try Flux CD and there’s a good reference project initiated by my colleage: k8s-gitops. However, in order to fully understand how to use Flux CD, I chose to…

  • Build Multi-arch Docker Images on Ubuntu Linux

    Since I’ve made my Raspberry PI Kubernetes cluster hybrid, now I have good reasons to build multi-arch(which means multi CPU architecture) Docker images so I don’t care if my pod is deployed to a Raspberry PI node or a X86 node. I followed a lot of instructions from this guide and finally made it work…

  • Hybrid Kubernetes Cluster (X86 + ARM)

    The one in the picture was my old laptop, then my daughter’s for a few years. Now she got a nice new 2-in-1 ultra book the school asked us parents to buy, this clunky one was gathering dust on shelves. I tried to sell it but got no one’s attention despite it has got i7…

  • Renew Certificates Used in Kubeadm Kubernetes Cluster

    It’s been more than a year since I built my Kubernetes cluster with some Raspberry PIs. There was a few times that I need to power down everything to let electricians do their work and the cluster came back online and seemed to be Ok afterwards, given that I didn’t shutdown the PIs properly at…

  • Use Variables in Kustomize

    Variables in Kustomize are handy helpers from time to time, with these variables I can link some settings together which should share the same value all the time. Without variable I probably need to use some template engine like Jinja2 to do the same trick. Some examples here. In my case, there’s a bug in…