5G + Public IP with OpenVPN

I’ve done a proof of concept with SSH tunneling to add a public IP to my 5G home broadband connection, it works for my garage-hosted blogs but it’s not a complete solution. Since I still have free credit in my personal Google Cloud account, I decided to make an improvement with OpenVPN. The diagram looks like:

        [CloudFlare] 
             |
            HTTP
             |
     [VM:35.197.x.x:80]
             |
       [iptables DNAT]
             |
      [OpenVPN tunnel]
             |
[local server tun0 interface: 10.8.0.51:80]

Following an outstanding tutorial on DigitalOcean I set up an OpenVPN server on Debian 10 running in a Google Cloud Compute instance. There’s a few more thing to do for my case.

First I needed to add port forwarding from the public interface of the OpenVPN server to home server’s tunnel interface. Here’s my ufw configuration file:

# this is /etc/ufw/before.rules
# START OPENVPN RULES
# NAT table rules
*nat
:PREROUTING ACCEPT [0:0]
# port forwarding to home server
-A PREROUTING -i eth0 -p tcp -d <public ip> --dport 80 -j DNAT --to 10.8.0.51:80

:POSTROUTING ACCEPT [0:0] 
# Allow traffic from OpenVPN client to eth0, ie. internet access
-A POSTROUTING -s 10.8.0.0/8 -o eth0 -j MASQUERADE
COMMIT
# END OPENVPN RULES

Make sure to restart ufw after this.

Then in my home server, the OpenVPN client can be configured to run as a service:

# this is /etc/systemd/system/vpnclient.service
[Unit]
Description=Setup an openvpn tunnel to kite server
After=network.target

[Service]
ExecStart=/usr/sbin/openvpn --config /etc/openvpn/client1.conf
RestartSec=5
Restart=always

[Install]
WantedBy=multi-user.target

To enable it and start immediately:

sudo systemctl daemon-reload
sudo systemctl enable vpnclient
sudo systemctl start vpnclient

Also I need my home server to have a fixed IP for its tun0 network interface, so the nginx server can proxy traffic to this IP reliably. I followed this guide, except it suggested to do client-config-dir on both server and client sides but I only did on the server side and it worked for me:

# this is /etc/openvpn/server.conf
# uncomment the following line
client-config-dir ccd

# this is /etc/openvpn/ccd/client1
ifconfig-push 10.8.0.51 255.255.255.255

After this the OpenVPN server on the VM needs to be restarted:

sudo systemctl restart [email protected]

Reload the nginx server and it should be working. I tested it with curl -H "Host: raynix.info" 35.197.x.x and the request hit my home server.

🙂

5G is Fast but There’s No Public IP

I’m super happy that I can finally have a broadband that does have a broad bandwidth. However like all other cellular services the 5G gateway has a private IP as its external IP, ie. everything I got is behind huge NAT servers of Optus and they will not open any port just for me.

The NBN wasn’t as fast but there was a public IP assigned to the router… Should I go back to use NBN because of this reason? I’ve done countless internet/cloud solutions, can I do one for myself? I remember when I worked on a private network behind NAT gateways I could do SSH tunneling and expose a port in the private network to outside for 3rd party partners. All I need is a virtual machine in the cloud that has a public IP. It will look like this:

        [CloudFlare] 
             |
            HTTP
             |
     [VM:35.197.x.x:80]
             |
       [SSH Tunnel]
             |
[local server:192.168.1.x:80]

I need to create a SSH tunneling service between my home server and the cloud instance, then point CloudFlare DNS to the public IP of the cloud instance, that’s it! But hang on, I can’t bind to port 80 using a non-privileged user. So there’s a local port forwarding in the cloud instance to handle this:

        [CloudFlare] 
             |
            HTTP
             |
     [VM:35.197.x.x:80]
             |
         [VM:nginx]
             |
    [VM:127.0.0.1:8080]
             |
        [SSH Tunnel]
             |
[local server:192.168.1.x:80]

Ok let’s do it. First I created a cloud instance in Google Cloud, because it gave me $400 free credit last year and I haven’t used much yet. I opened port 80 for the instance, added my SSH public key and installed an nginx server forwarding traffic from port 80 to local port 8080. Here’s the simple nginx configuration:

# This file can be saved as /etc/nginx/site-enabled/proxy
# in the cloud instance
server{
        listen 80;
        server_name raynix.info;
	client_max_body_size 100M;

        location / {
		proxy_pass http://localhost:8080;
		proxy_set_header Host              $host;
		proxy_set_header X-Real-IP         $remote_addr;
		proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto https;
		proxy_set_header X-Forwarded-Host  $host;
		proxy_set_header X-Forwarded-Port  $server_port;
        }
}

Don’t forget to reload nginx. Done! The next step is to create an SSH tunneling service in my home server. My home server is running Ubuntu so the service is defined with systemd syntax:

# The file can be saved as /etc/systemd/system/mytunnel.service
# in my home server
# replace the id_rsa with yours of course
[Unit]
Description=Setup a secure tunnel to kite server
After=network.target

[Service]
ExecStart=/usr/bin/ssh -NT -i /home/ray/.ssh/id_rsa -o ServerAliveInterval=60 -o ExitOnForwardFailure=yes -R 8080:localhost:80 [email protected]

# Restart every >2 seconds to avoid StartLimitInterval failure
RestartSec=5
Restart=always

[Install]
WantedBy=multi-user.target

Then use the following commands to start and check the service:

$ systemctl daemon-reload
$ systemctl start mytunnel
$ systemctl status -l mytunnel

The status of the tunnel can also be verified in the cloud instance by:

$ sudo netstat -tlnp
...
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      1460/sshd: ray      

By now all the dots have been connected. I can test the round trip on my laptop with:

$ curl http://35.197.x.x -H 'Host: raynix.info'
# this should print out the HTML of my blog's homepage.

The last step is to update CloudFlare DNS to send traffic to the new cloud instance. Did it work? You’re looking at the result right now 🙂

5G Is Fast! Bye Bye NBN

A few months ago I knew Optus was rolling out the shiny 5G services but to my surprise it’s available in my suburb already. It took NBN years.

It’s a tough time at the moment as everyone is ordered to stay home, but it’s a perfect time to upgrade my home broadband and without paying more: The 5G plan costs $70/month and has unlimited data, same as my NBN50 price, but much faster. The deal is here.

I placed my order online on Sunday, it was processed on Monday and 3 days later I received the Nokia 5G gateway. It’s really big and heavy comparing to any DSL or Cable modem I’ve ever used.

Nokia FastMile 5G Gateway with plastic wraps

It’s much easier to install this thing, if this can be called an installation at all. Basically I only need to put it on a surface near a window and facing the 5G tower and power it on. It will compare signal strength among its 5 antennas and indicate the one it will use. (So why have 5 antennas in the first place?) For me I’m very lucky because I can see the telecomm tower through my window so I pointed the active antenna at the tower with laser precision then doing the speed test which I waited for too long.

Optus 5G Home Broadband Speed Test

It’s about 8 times faster than my NBN when downloading and 2.5 times faster when uploading. I used to prefer a wired connection, but not anymore.

There’s 1 issue though: there’s no public IP allocated to the gateway. I guess that makes sense as cellular devices never have a real IP anyway. This translates to that I can’t do either uPnP or port forwarding! How can I still host this little blog in a Raspberry Pi in garage then?

(the answer is in next blog) 🙂

Using Sealed Secrets in a Raspberry Pi Kubernetes Cluster

Sealed Secrets is a bitnami Kubernetes operator aimed to one-way encrypt secrets into sealed secrets so that they can be safely checked-in into GitHub or other VCS. It’s rather easy to install and use Sealed Secrets in a Kubernetes cluster on AMD64 architecture, but not so on my Raspberry Pi cluster.

First, the container image for the sealed-secrets-controller wasn’t built for ARM architecture. I managed to build it in my Raspberry Pi 2 with following commands:

git clone https://github.com/bitnami-labs/sealed-secrets.git
cd sealed-secrets
# golang build tools are needed here
make controller.image
# you can tag it to your docker registry instead of mine
docker tag quay.io/bitnami/sealed-secrets-controller:latest raynix/sealed-secrets-controller-arm:latest
docker push raynix/sealed-secrets-controller-arm

The next step is to use kustomize to override the default sealed-secrets deployment schema to use my newly built container image that runs on ARM

# kustomization.yaml
# controller.yaml is from https://github.com/bitnami-labs/sealed-secrets/releases/download/v0.9.7/controller.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: sealed-secrets
images:
  - name: quay.io/bitnami/sealed-secrets-controller
    newName: raynix/sealed-secrets-controller-arm
    newTag: latest
patchesStrategicMerge:
  - patch.yaml

resources:
  - controller.yaml
  - ns.yaml
# ns.yaml
# I'd like to install the controller into its own namespace
apiVersion: v1
kind: Namespace
metadata:
  name: sealed-secrets
# patch.yaml
# apparently the controller running on Raspberry Pi 4 needs more time to initialize
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sealed-secrets-controller
spec:
  template:
    spec:
      containers:
        - name: sealed-secrets-controller
          readinessProbe:
            initialDelaySeconds: 100

Then the controller can be deployed with command kubectl apply -k .

The CLI installation is much easier on a Linux laptop. After kubeseal is installed. The public key used to encrypt secrets can be obtained from the controller deployed above. Since I installed the controller in it’s own namespace sealed-secrets instead of the default kube-system the command to encrypt secrets is a bit different:

kubectl create secret generic test-secret --from-literal=username=admin --from-literal=password=password --dry-run -o yaml | \
  kubeseal --controller-namespace=sealed-secrets -o yaml > sealed-secrets.yaml

Then the generated file sealed-secrets.yaml can be deploy with kubectl apply -f sealed-secrets.yaml and a secret called test-secret will be created. Now feel free to check-in sealed-secrets.yaml into a public GitHub repository!

🙂