raynix 筆記

[ Solved ] Regex Rules in HTTPRoutes Silently Dropped by Istio

In my recent journey to adopt Gateway API in GKE clusters in production environments, I encountered a quite weird issue: When there are the default catch-all route and some regex routes defined as HTTPRoutes, the regex ones are silently dropped causing those requests to go through the default route unexpectedly.

For example, for the following HTTPRoutes:

# regex routes
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: regex-routes
  namespace: app
spec:
  hostnames:
    - www.example.com
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: default-gateway
      namespace: istio-system
  rules:
    - backendRefs:
        - group: ''
          kind: Service
          name: app-for-regex
          namespace: app
          port: 8080
          weight: 100
      matches:
        - path:
            type: RegularExpression
            value: /my-prefix-.*
---
# default route
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: default-routes
  namespace: app
spec:
  hostnames:
    - www.example.com
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: default-gateway
      namespace: istio-system
  rules:
    - backendRefs:
        - group: ''
          kind: Service
          name: app-canary
          namespace: app
          port: 8080
          weight: 0
        - group: ''
          kind: Service
          name: app-stable
          namespace: app
          port: 8080
          weight: 100

Once these are applied, unexpectedly requests to /my-prefix-.* went to app-stable instead of app-regex. Along the way of investigation, I checked the generated Envoy proxy config using command like

istioctl proxy-config route istio-gateway-pod.istio-system -o json |less

and to my surprise, the regex rules aren’t there at all – they are dropped for some reason. Then I tried the logs from istiod and istio-gateway pods but nothing was complaining…

Upon a deeper look, the default route which doesn’t have matches attribute actually got changed somehow, probably by some istio injection controller:

# default route
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: default-routes
  namespace: app
spec:
  hostnames:
    - www.example.com
  parentRefs:
    - group: gateway.networking.k8s.io
      kind: Gateway
      name: default-gateway
      namespace: istio-system
  rules:
    - backendRefs:
        - group: ''
          kind: Service
          name: app-canary
          namespace: app
          port: 8080
          weight: 0
        - group: ''
          kind: Service
          name: app-stable
          namespace: app
          port: 8080
          weight: 100
      matches: # <-- workaround: change this to matches: []
        - path:
            type: PathPrefix
            value: /

When I added matches: [] explicitly, suddenly everything starts to work as expected and I could see the regex rules in the Envoy proxy config! I don’t know what’s the logic behind the automatic pathPrefix injection but it seemed that’s the culprit.

I thought about adding this as an issue for istio, there’s already one though.

Hope this helps you out too. My environment is: Istio 1.30 + K8s 1.35 + Gateway API 1.5 🙂

admin

, ,