-
Solved: Google Managed Prometheus Kept Crashing
Context: I use Google Cloud Managed Service for Prometheus(GMP I call it for convenience) as the central piece of my observability stack. In a nut shell it’s Prometheus managed by Google. GMP runs a collector Prometheus pod in each node, as a DaemonSet. The collector Prometheus scrapes metrics within the node and forwards them to…
-
Monitoring MySQL with mysqld-exporter
TL; DR: I use mysqld-exporter with Prometheus to monitor my MySQL server. The mysqld-exporter runs as a container in my Kubernetes cluster. A sample manifest looks like: 🙂
-
How to Calculate Proportions Using 2 Metrics in Prometheus
Recently I needed to improve an alert defined using Prometheus Alert Manager to trigger when less than half of the minimum replicas are up, eg. I had a look at existing metrics collected by kube-state-metrics in my Kubernetes cluster, the number of replicas running well could be queried with the kube_pod_status_ready metric and the number…
-
How to Add Extra Scrape Configuration for Prometheus
After some recent updating to my home server running Ubuntu Server 20.04, the InfluxDB server suddenly stopped working. I’ve used InfluxDB + Telegraf for many years, the combination worked very well without much maintenance. But as I have deployed some new-ish stuff – Prometheus, Loki, etc. in my garage Kubernetes lab, I think it’s time…
-
Deploy the Loki Stack in a Kubernetes Cluster with ArgoCD
Loki and Promtail from Grafana Labs are new kids in the observability community. Are they good enough to replace Elasticsearch and Logstash? I would like to see. Here’s a sample ArgoCD Application to deploy Loki, Promtail, Prometheus and Grafana all from 1 Helm chart: grafana/loki-stack. Some settings of my installations are: loki, grafana and prometheus…