Posted on

Install Nvidia 396 Driver for Ubuntu 18.04

Just some quick notes to upgrade Nvidia driver from 390 to 396 on Ubuntu 18.04. By default Ubuntu 18.04 only has Nvidia driver 390.

# add ubuntu official ppa for nvidia
# all commands below run as root
add-apt-repository ppa:graphics-drivers/ppa
apt update
# install 396 files
apt install nvidia-dkms-396 nvidia-driver-396 nvidia-compute-utils-396 libnvidia-compute-396
# reboot at the end.

After reboot, the new driver can be verified by:

nvidia-smi 
 +-----------------------------------------------------------------------------+
 | NVIDIA-SMI 396.54                 Driver Version: 396.54                    |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
 |===============================+======================+======================|
 |   0  GeForce GTX 1070    Off  | 00000000:01:00.0 Off |                  N/A |
 | 44%   67C    P2   150W / 230W |   3211MiB /  8117MiB |    100%      Default |
 +-------------------------------+----------------------+----------------------+

🙂

Posted on

Home VPN with OpenVPN

Here are step to run a simple OpenVPN service at home, so that I can access home network easily while not at home.

First, clone the git repo for OpenVPN docker container:

git clone https://github.com/kylemanna/docker-openvpn.git

I can use the pre-built docker image from docker hub but it has just been breached so I’d rather build it myself:

cd docker-openvpn && docker build -t openvpn .

Create a docker volume to persist data if the OpenVPN container to be rebuilt:

export $OVPN_DATA=ovpn_data
docker volume create --name $OVPN_DATA

Generate OpenVPN configurations, if there’s no DNS record for the server, use the public IP of the home broadband alternatively.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm openvpn ovpn_genconfig -u udp://VPN.SERVERNAME.COM

Build a new secret key which will be used to generate user keys. I’d advise to use a strong password which can be saved in a password manager or vault. This is needed everytime when I create a new user.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it openvpn ovpn_initpki

Then the OpenVPN server container can be run as a service:

docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN openvpn

Generate the first user profile. The password for secret key will be needed. Then retrieve the OpenVPN configuration with the 2nd command.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it openvpn easyrsa build-client-full <username> nopass
docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm openvpn ovpn_getclient <username> > <username>.ovpn

This .ovpn file can be used to configure OpenVPN client softwares on laptops or phones.

At last, ensure UDP 1194 port is forwarded to the host of the docker container. This is usually done in the home broadband router.

Posted on

Ansible, CloudFormation and Jinja2

CloudFormation is the corner stone to provision infrastructure in AWS with code, however it’s not very DRY, ie. poor modularization, almost static variables and templates. So here comes Ansible.

However at the moment Ansible’s CloudFormation module doesn’t support Jinja2 in templates, like other modules do. Luckily there’s a work-around to get the Ansible-CloudFormation-Jinja2 trio working together.

A simple CloudFormation snippet with Jinja2 variable:

# roles/test-stack/templates/mycf.yaml.j2
...
Parameters:
  VpcId:
    Type: String
    Default: '{{ template_params.vpc_id }}'
...

I don’t put Jinja2 variable directly to replace the !Ref intrinsic function, because I think this is a softer approach so if there’s any parameter verification in the template it still works.

Then the template can be loaded to Ansible’s cloudformation module like this:

# roles/test-stack/tasks/main.yaml
- name: create a stack
  cloudformation:
    stack_name: '{{ stack_names.test_stack }}'
    state: present
    template_body: "{{ lookup('template', 'templates/mycf.yaml.j2') }}"
    tags: '{{ template_tags }}'
...

The inventory may look like this:

# inventory/local
all:
  hosts:
    dev:
      ansible_connection: local
      gather_facts: false

All parameters can go into the global variable file group_vars/all or host variable file host_vars/dev so later I can create a set of parameters for production. 

# group_vars/all
stack_names:
  test_stack: test1
...
# host_vars/dev
template_params:
  vpc_id: vpc-xxxxxxxxxxxxxxx

template_tags:
  app_env: dev

And the playbook can be simple as:

# deploy.yaml
- name: deploy stacks
  hosts: dev
  tags: dev
  roles:
    - test-stack

Finally, this CloudFormation stack can be deployed by running:

$ ansible-playbook -i inventory/local deploy.yaml --tags dev

🙂

Posted on

Working with a Big Corporation

So it’s been a while since I started this job in a big corporation. I always enjoy new challenges, now my wish got granted. Not in a very good way.

The things work in a quite different manner here. There are big silos and layers between teams and departments, so the challenges here are not quite technical in nature. How unexpected this is.

Still there are lots of things can be improved with technology, here’s one example. When I was migrating an old web application stack from on-premises infrastructure to AWS, the AWS landing zone has already been provisioned with a duo-VPC setup. I really really miss the days that working with Kubernetes clusters and I can just run kubectl exec -ti ... and get a terminal session quickly.

Now things look like year 2000 and I need to use SSH proxy command again, without old school static IP addresses though. Ansible dynamic inventory is quite handy in most cases but it failed due to some unknown corporate firewall rules. I still have bash, aws-cli and jq, so this is my handy bash script to connect to 1 instance of an auto scaling group, via a bastion host(they both can be rebuilt and change IP).

#!/bin/bash
function get_stack_ip(){
  aws ec2 describe-instances \
    --fileter "Name=tag-key,Values=aws:cloudformation:stack-name" "Name=tag-value,Values=$1" \
    |jq '.Reservations[] |select(.Instance[0].PrivateIpAddress != null).Instance[0].PrivateIpAddress' \
    |tr -d '"'
}

Then it’s easy to use this function to get IPs of the bastion stack and the target stack, such as:

IP_BASTION=$(get_stack_ip bastion_stack)
IP_TARGET=$(get_stack_ip target_stack)
ssh -o ProxyCommand="ssh [email protected]_BASTION nc %h %p" [email protected]_TARGET

🙂