Posted on

AWS Lambda with Single CloudFormation Template

This is just a quick snippet of CloudFormation template to deploy a Python 3.7 Lambda function embedded in the template. The source code inside ZipFile can contain up to 4KB.

# CloudFormation template
AWSTemplateFormatVersion: 2010-09-09
Resources:
  LambdaCode:
    Type: AWS::Lambda::Function
    DependsOn:
      - LambdaRole
      - LambdaPolicy
    Properties:
      Code:
        ZipFile: |
          import boto3
          def handler(event, context):
            ...
      Role: !GetAtt LambdaRole.Arn
      Runtime: python3.7
      Handler: index.handler

Note: the depending LambdaRole is not included, but can be found easily here .

🙂

Posted on

Golang and Docker Multi-Stage Build

I have noticed a common pattern amonst some new utilities such as kubectl, kops and terraform: There’s only 1 single executable file to install, and by ‘install’ it can be put anywhere as long as it’s in $PATH. This was before I learned some Golang but it’s easy to find out that the reason behind this pattern is that they are all written in Go.

And in the containers’ realm, the new-ish multi-stage build steps of Docker released in 2017 are super beneficial to Golang containers. A TL;DR example looks like:

  1. use a 1GB Debian container with all Golang tools and build dependencies to build the Golang executable( FROM ... AS in the sample ).
  2. put the executable into a tiny run-time container such as Alpine Linux, resulting in a < 20MB container image(depending on the size of the app obviously) ( COPY --FROM in the sample )

A multi-stage ‘hello world’ Dockerfile looks like:

FROM golang:1.12.5-alpine3.9 as builder
ENV GO111MODULE=on
RUN apk update --no-cache && \
  apk add git
WORKDIR /app 
ADD ./ /app
RUN go build -o golang-test .

FROM alpine:3.9.4
WORKDIR /app 
RUN addgroup -g 2000 golang && \
  adduser -D -u 2000 -G golang golang
USER golang
COPY --from=builder /app/golang-test .
CMD ["/app/golang-test"] 
EXPOSE 8000

Note: To be able to use the multi-stage feature, the Docker version has to be > 17.06.

🙂

Posted on

Home VPN with OpenVPN

Here are step to run a simple OpenVPN service at home, so that I can access home network easily while not at home.

First, clone the git repo for OpenVPN docker container:

git clone https://github.com/kylemanna/docker-openvpn.git

I can use the pre-built docker image from docker hub but it has just been breached so I’d rather build it myself:

cd docker-openvpn && docker build -t openvpn .

Create a docker volume to persist data if the OpenVPN container to be rebuilt:

export $OVPN_DATA=ovpn_data
docker volume create --name $OVPN_DATA

Generate OpenVPN configurations, if there’s no DNS record for the server, use the public IP of the home broadband alternatively.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm openvpn ovpn_genconfig -u udp://VPN.SERVERNAME.COM

Build a new secret key which will be used to generate user keys. I’d advise to use a strong password which can be saved in a password manager or vault. This is needed everytime when I create a new user.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it openvpn ovpn_initpki

Then the OpenVPN server container can be run as a service:

docker run -v $OVPN_DATA:/etc/openvpn -d -p 1194:1194/udp --cap-add=NET_ADMIN openvpn

Generate the first user profile. The password for secret key will be needed. Then retrieve the OpenVPN configuration with the 2nd command.

docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm -it openvpn easyrsa build-client-full <username> nopass
docker run -v $OVPN_DATA:/etc/openvpn --log-driver=none --rm openvpn ovpn_getclient <username> > <username>.ovpn

This .ovpn file can be used to configure OpenVPN client softwares on laptops or phones.

At last, ensure UDP 1194 port is forwarded to the host of the docker container. This is usually done in the home broadband router.

Posted on

Ansible, CloudFormation and Jinja2

CloudFormation is the corner stone to provision infrastructure in AWS with code, however it’s not very DRY, ie. poor modularization, almost static variables and templates. So here comes Ansible.

However at the moment Ansible’s CloudFormation module doesn’t support Jinja2 in templates, like other modules do. Luckily there’s a work-around to get the Ansible-CloudFormation-Jinja2 trio working together.

A simple CloudFormation snippet with Jinja2 variable:

# roles/test-stack/templates/mycf.yaml.j2
...
Parameters:
  VpcId:
    Type: String
    Default: '{{ template_params.vpc_id }}'
...

I don’t put Jinja2 variable directly to replace the !Ref intrinsic function, because I think this is a softer approach so if there’s any parameter verification in the template it still works.

Then the template can be loaded to Ansible’s cloudformation module like this:

# roles/test-stack/tasks/main.yaml
- name: create a stack
  cloudformation:
    stack_name: '{{ stack_names.test_stack }}'
    state: present
    template_body: "{{ lookup('template', 'templates/mycf.yaml.j2') }}"
    tags: '{{ template_tags }}'
...

The inventory may look like this:

# inventory/local
all:
  hosts:
    dev:
      ansible_connection: local
      gather_facts: false

All parameters can go into the global variable file group_vars/all or host variable file host_vars/dev so later I can create a set of parameters for production. 

# group_vars/all
stack_names:
  test_stack: test1
...
# host_vars/dev
template_params:
  vpc_id: vpc-xxxxxxxxxxxxxxx

template_tags:
  app_env: dev

And the playbook can be simple as:

# deploy.yaml
- name: deploy stacks
  hosts: dev
  tags: dev
  roles:
    - test-stack

Finally, this CloudFormation stack can be deployed by running:

$ ansible-playbook -i inventory/local deploy.yaml --tags dev

🙂