Category: Systems Admin

  • Solved: Kustomize Unable to Parse SM or JSON Patch

    Recently I noticed that my ArgoCD manifests I used to patch using kustomize don’t work with latest version of kustomize anymore. The error I got looks like: My patch.yaml had something like this: There’s no syntax error in my files as this worked before. I tried to remove sections from the patch file and see…

  • Solved: Upgrading Mastodon from 4.2.x to 4.3.1

    It’s been almost 4 years since I started my own Mastodon instance. I took a look at the latest releases and my instance is way behind, so I decided to give v4.3.1 a try. With all the YAMLs and Kustomize stuff I did before, last few upgrades were as simple as a version number change.…

  • How I Upgraded MongoDB 6 Directly to 8

    Long story short, I had a server running Ubuntu 20.04 and unsurprisingly it’s been there for more than 4 years. For no reason I decided to give it an upgrade, actually 2 upgrades, ie. 20.04 –> 22.04 –> 24.04. And I totally regretted. The Ubuntu Linux system upgrade went well and I completely ignored the…

  • How to Use Dynamic Storage Provisioning in Kubernetes

    Here are the notes on how I enabled dynamic provisioning for persistent volumes in my garage Kubernetes cluster, using a combination of: NFS server I used a typical installation of NFS server on Ubuntu Linux. Here’s the documentation from Ubuntu. CSI driver for NFS From the Github repo there are Helm templates provided to install…

  • How to Provision Google Log-Based Metrics with Terraform and YAML

    TL; DR: here are my notes on provisioning Google log-based metrics using Terraform and YAML. The metrics are defined in YAML in a clean fashion and they will be created via terraform apply The HCL part for Terraform Ref.1, https://cloud.google.com/logging/docs/logs-based-metrics2, https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/logging_metric 🙂

  • Indexes for Structured Logs in Google Cloud

    TL; DR: Google Cloud accepts structured logs ingested from different sources but will not index fields automatically. So by default the ingested jsonPayload will not be optimised for query. Here’s how to add indexes to it using Terraform. 🙂

  • How to Prune Only Selected Fields in Jsonnet

    In Jsonnet’s standard libraries, there’s an std.prune function which will recursively go through an object and remove any empty field in the object which is quite handy, usually. In my case I wanted to prune all empty fields in a container object but spare the legit empty emptyDir field so the std.prune may cause some…

  • How to Do Kubernetes Cronjobs with Sidecars Properly in 2024

    TL; DR: Since Kubernetes v1.28, there’s no need to do fancy bash scripts to kill your sidecar when the main container finishes. So using sidecars in Kubernetes Jobs or Cronjobs are now much simpler and more elegant too. For example, I wanted to run a database backup job in a GKE + Cloud SQL environment,…