AWS Lambda with Single CloudFormation Template

This is just a quick snippet of CloudFormation template to deploy a Python 3.7 Lambda function embedded in the template. The source code inside ZipFile can contain up to 4KB.

# CloudFormation template
AWSTemplateFormatVersion: 2010-09-09
Resources:
LambdaCode:
Type: AWS::Lambda::Function
DependsOn:
- LambdaRole
- LambdaPolicy
Properties:
Code:
ZipFile: |
import boto3
def handler(event, context):
...
Role: !GetAtt LambdaRole.Arn
Runtime: python3.7
Handler: index.handler

Note: the depending LambdaRole is not included, but can be found easily here .

🙂

Golang and Docker Multi-Stage Build

I have noticed a common pattern amonst some new utilities such as kubectl, kops and terraform: There’s only 1 single executable file to install, and by ‘install’ it can be put anywhere as long as it’s in $PATH. This was before I learned some Golang but it’s easy to find out that the reason behind this pattern is that they are all written in Go.

And in the containers’ realm, the new-ish multi-stage build steps of Docker released in 2017 are super beneficial to Golang containers. A TL;DR example looks like:

  1. use a 1GB Debian container with all Golang tools and build dependencies to build the Golang executable( FROM ... AS in the sample ).
  2. put the executable into a tiny run-time container such as Alpine Linux, resulting in a < 20MB container image(depending on the size of the app obviously) ( COPY --FROM in the sample )

A multi-stage ‘hello world’ Dockerfile looks like:

FROM golang:1.12.5-alpine3.9 as builder
ENV GO111MODULE=on
RUN apk update --no-cache && \
apk add git
WORKDIR /app
ADD ./ /app
RUN go build -o golang-test .

FROM alpine:3.9.4
WORKDIR /app
RUN addgroup -g 2000 golang && \
adduser -D -u 2000 -G golang golang
USER golang
COPY --from=builder /app/golang-test .
CMD ["/app/golang-test"]
EXPOSE 8000

Note: To be able to use the multi-stage feature, the Docker version has to be > 17.06.

🙂

Home VPN with OpenVPN

Here are step to run a simple OpenVPN service at home, so that I can access home network easily while not at home.

First, clone the git repo for OpenVPN docker container:

git clone https://github.com/kylemanna/docker-openvpn.git

I can use the pre-built docker image from docker hub but it has just been breached so I’d rather build it myself:

cd docker-openvpn && docker build -t openvpn .

Create a docker volume to persist data if the OpenVPN container to be rebuilt:

export $OVPN_DATA=ovpn_data
docker volume create --name $OVPN_DATA

Generate OpenVPN configurations, if there’s no DNS record for the server, use the public IP of the home broadband alternatively.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm openvpn ovpn_genconfig -u udp://VPN.SERVERNAME.COM

Build a new secret key which will be used to generate user keys. I’d advise to use a strong password which can be saved in a password manager or vault. This is needed everytime when I create a new user.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it openvpn ovpn_initpki

Then the OpenVPN server container can be run as a service:

docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN openvpn

Generate the first user profile. The password for secret key will be needed. Then retrieve the OpenVPN configuration with the 2nd command.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it openvpn easyrsa build-client-full <username> nopass
docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm openvpn ovpn_getclient <username> > <username>.ovpn

This .ovpn file can be used to configure OpenVPN client softwares on laptops or phones.

At last, ensure UDP 1194 port is forwarded to the host of the docker container. This is usually done in the home broadband router.

Ansible, CloudFormation and Jinja2

CloudFormation is the corner stone to provision infrastructure in AWS with code, however it’s not very DRY, ie. poor modularization, almost static variables and templates. So here comes Ansible.

However at the moment Ansible’s CloudFormation module doesn’t support Jinja2 in templates, like other modules do. Luckily there’s a work-around to get the Ansible-CloudFormation-Jinja2 trio working together.

A simple CloudFormation snippet with Jinja2 variable:

# roles/test-stack/templates/mycf.yaml.j2
...
Parameters:
VpcId:
Type: String
Default: '{{ template_params.vpc_id }}'
...

I don’t put Jinja2 variable directly to replace the !Ref intrinsic function, because I think this is a softer approach so if there’s any parameter verification in the template it still works.

Then the template can be loaded to Ansible’s cloudformation module like this:

# roles/test-stack/tasks/main.yaml
- name: create a stack
cloudformation:
stack_name: '{{ stack_names.test_stack }}'
state: present
template_body: "{{ lookup('template', 'templates/mycf.yaml.j2') }}"
tags: '{{ template_tags }}'
...

The inventory may look like this:

# inventory/local
all:
hosts:
dev:
ansible_connection: local
gather_facts: false

All parameters can go into the global variable file group_vars/all or host variable file host_vars/dev so later I can create a set of parameters for production.

# group_vars/all
stack_names:
test_stack: test1
...
# host_vars/dev
template_params:
vpc_id: vpc-xxxxxxxxxxxxxxx

template_tags:
app_env: dev

And the playbook can be simple as:

# deploy.yaml
- name: deploy stacks
hosts: dev
tags: dev
roles:
- test-stack

Finally, this CloudFormation stack can be deployed by running:

$ ansible-playbook -i inventory/local deploy.yaml --tags dev

🙂