raynix 筆記

[ Solved ] `33554536:system library:OPENSSL_internal:Connection reset by peer` Error with GCLB + Gateway API + OSS Istio

I was trying to get a PoC of Google global external load-balancer to work with Gateway API and Istio as gateway class running in a standard GKE cluster. Everything was working as expected except I got 

curl -i https://test.my-poc-domain.com
HTTP/2 503 content-length: 239 
content-type: text/plain 
via: 1.1 google 
date: Wed, 14 Jan 2026 04:50:12 GMT 
strict-transport-security: max-age=63072000; 
includeSubDomains; preload 
alt-svc: h3=":443"; 
ma=2592000,h3-29=":443"; 
ma=2592000 
upstream connect error or disconnect/reset before headers. retried and the latest reset 
reason: remote connection failure, transport failure reason: TLS_error:|33554536:system library:OPENSSL_internal:Connection reset by peer:TLS_error_end

My setup looks like below

First I tried to test if the request can be served by the gateway:

# port forward the gateway https port
k port-forward -n istio-system istio-gateway-xxx 8443:8443

# local test
curl -i -k --http2 \
  --resolve test.my-poc-domain.com:8443:127.0.0.1 \
  https://test.my-poc-domain.com:8443

HTTP/2 200
...

So everything below gateway on the diagram should not be the cause, ie. the problem is probably there between GCLB and Gateway. Since it’s an TLS error in the output, I was suggested to use HTTP instead of HTTPS between GCLB and Gateway. And that worked! Now it’s narrowed down to HTTPS between GCLB and the Gateway.

The cert is managed by Google but I double checked still and the TLS cert is 100% valid.

I was at a loss. Then my friend at Google suggested the following openssl commands:

# still using the port forwarding from previous step
# this doesn't work
openssl s_client -connect 127.0.0.1:8443 -brief
40224BF501000000:error:0A000126:SSL routines::unexpected eof while reading:ssl/record/rec_layer_s3.c:701:

# this works
openssl s_client -connect 127.0.0.1:8443 -servername test.my-poc-domain.com -brief
Connecting to 127.0.0.1
depth=1 C=AU, ST=NSW, L=Sydney,...

Now it’s clear: The gateway backed by Istio asked for a server name(SNI) and if it’s not given it will reject the connection – coincidently GCLB doesn’t give SNI to backend, hence the 503 error.

To fix this, the hostname in the Gateway needs to be omitted, so effectively the Gateway will not ask for SNI and accept all hostnames.

Problem solved 🙂

admin

, ,