How to Regulate Egress Access in Kubernetes with Istio


Usually I don’t mind to give pods unlimited egress access, ie. the pods I deployed can access the whole internet if it needs to. However when the pods take input from users it’s a whole different story.

For example, running some sandbox applications such as an online Python learning environment, the workload can be abused by users to act as bots to access some websites. So here’s a cheat sheet on how to limit what the pods can access, with Istio 1.13 at the moment.

First, is to install or re-install Istio with below flags:

istioctl install <any existing flags> --set meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY --set meshConfig.accessLogFile=/dev/stdout

The first flag meshConfig.outboundTrafficPolicy.mode=REGISTRY_ONLY enable the whitelist mode of egress traffic. The second flag meshConfig.accessLogFile=/dev/stdout writes blocked outbound access to logs for monitoring.

Then a global domain whitelist can be deployed as a ServiceEntry:

apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: allowed-hosts
  namespace: istio-system
spec:
  hosts:
    - github.com
    - httpbin.org
  # make it namespaced with exportTo
  # exportTo:
  #   - "."
  ports:
    - number: 443
      name: https
      protocol: HTTPS
    - number: 80
      name: http
      protocol: HTTP
  resolution: DNS
  location: MESH_EXTERNAL

Next, Istio sidecar injection needs to be enabled for the workload’s namespace:

k label ns my-namespace istio-injection=enabled

The workload pods need to be re-deployed to have Istio sidecar installed:

k rollout -n my-namespace restart deploy my-linux-box

To monitor blocked access:

k logs -f -n my-namespace my-linux-box -c istio-proxy |grep BlackHoleCluster
[2022-03-23T00:18:58.542Z] "- - -" 0 UH - - "-" 0 0 6 - "-" "-" "-" "-" "-" BlackHoleCluster - 157.240.8.35:443 10.2.2.27:58410 facebook.com -

🙂