Category: Systems Admin

  • OpenSSL Commands to Verify TLS Certs in Kubernetes Secrets

    Sometimes a TLS cert deployed into a Kubernetes cluster in a Secret doesn’t work as expected. Here are some handy commands to verify the certs. The sample commands work for Istio Ingressgateway, but should be adapted to other CNIs without huge efforts. Commands to verify the cert served by your web-app Commands to verify the…

  • Fixed: Duplicated Cluster IPs in a Kubernetes Cluster

    A Cluster IP is an IP address allocated in the cluster’s virtual LAN, usually allocated to Kubernetes Services. As a user nobody should care about which cluster IP a service would get because we will use the service-name.namespace-name DNS name which will map to that IP automatically. In my case, some services in a GKE…

  • The Upgrade of Kubernetes Ingress Nginx

    The ingress-nginx container image I’ve been using was v0.25, and that’s more than 1 year old. The recent release is v0.44 but it’s a big leap from 25 to 44 and I’ve found some major differences between the 2 versions. Version 0.25 implemented API version of networking.k8s.io/v1beta1 while version 0.44 has networking.k8s.io/v1. Here are samples…

  • Fixed gsutil OSError: Permission Denied

    When trying to run gsutil in a kubernetes Job as nobody to backup stuff to Google Cloud Storage(GCS), I encountered simple error messages like But it didn’t say where or how the permission was denied! It worked fine if the container was run as root user so the problem is not with Google Cloud. I…

  • Kubernetes: How to Use Affinity

    Affinity is a great feature in Kubernetes to assign pods to nodes based on labels. In my case, I have a hybrid Kubernetes cluster with half nodes are of X86 architecture and other half of ARM architecture, and I need to deploy the X86 only containers to the X86 nodes. Of course I can build…

  • Kubernetes and GitOps with Flux CD V2.0

    GitOps could be the next big thing in cloud automation so I’d give it a try with my in house hybrid Kubernetes cluster. I was recommended to try Flux CD and there’s a good reference project initiated by my colleage: k8s-gitops. However, in order to fully understand how to use Flux CD, I chose to…

  • Build Multi-arch Docker Images on Ubuntu Linux

    Since I’ve made my Raspberry PI Kubernetes cluster hybrid, now I have good reasons to build multi-arch(which means multi CPU architecture) Docker images so I don’t care if my pod is deployed to a Raspberry PI node or a X86 node. I followed a lot of instructions from this guide and finally made it work…

  • Hybrid Kubernetes Cluster (X86 + ARM)

    The one in the picture was my old laptop, then my daughter’s for a few years. Now she got a nice new 2-in-1 ultra book the school asked us parents to buy, this clunky one was gathering dust on shelves. I tried to sell it but got no one’s attention despite it has got i7…