Ansible, CloudFormation and Jinja2

CloudFormation is the corner stone to provision infrastructure in AWS with code, however it’s not very DRY, ie. poor modularization, almost static variables and templates. So here comes Ansible.

However at the moment Ansible’s CloudFormation module doesn’t support Jinja2 in templates, like other modules do. Luckily there’s a work-around to get the Ansible-CloudFormation-Jinja2 trio working together.

A simple CloudFormation snippet with Jinja2 variable:

# roles/test-stack/templates/mycf.yaml.j2
...
Parameters:
VpcId:
Type: String
Default: '{{ template_params.vpc_id }}'
...

I don’t put Jinja2 variable directly to replace the !Ref intrinsic function, because I think this is a softer approach so if there’s any parameter verification in the template it still works.

Then the template can be loaded to Ansible’s cloudformation module like this:

# roles/test-stack/tasks/main.yaml
- name: create a stack
cloudformation:
stack_name: '{{ stack_names.test_stack }}'
state: present
template_body: "{{ lookup('template', 'templates/mycf.yaml.j2') }}"
tags: '{{ template_tags }}'
...

The inventory may look like this:

# inventory/local
all:
hosts:
dev:
ansible_connection: local
gather_facts: false

All parameters can go into the global variable file group_vars/all or host variable file host_vars/dev so later I can create a set of parameters for production.

# group_vars/all
stack_names:
test_stack: test1
...
# host_vars/dev
template_params:
vpc_id: vpc-xxxxxxxxxxxxxxx

template_tags:
app_env: dev

And the playbook can be simple as:

# deploy.yaml
- name: deploy stacks
hosts: dev
tags: dev
roles:
- test-stack

Finally, this CloudFormation stack can be deployed by running:

$ ansible-playbook -i inventory/local deploy.yaml --tags dev

🙂

Working with a Big Corporation

So it’s been a while since I started this job in a big corporation. I always enjoy new challenges, now my wish got granted. Not in a very good way.

The things work in a quite different manner here. There are big silos and layers between teams and departments, so the challenges here are not quite technical in nature. How unexpected this is.

Still there are lots of things can be improved with technology, here’s one example. When I was migrating an old web application stack from on-premises infrastructure to AWS, the AWS landing zone has already been provisioned with a duo-VPC setup. I really really miss the days that working with Kubernetes clusters and I can just run kubectl exec -ti ... and get a terminal session quickly.

Now things look like year 2000 and I need to use SSH proxy command again, without old school static IP addresses though. Ansible dynamic inventory is quite handy in most cases but it failed due to some unknown corporate firewall rules. I still have bash, aws-cli and jq, so this is my handy bash script to connect to 1 instance of an auto scaling group, via a bastion host(they both can be rebuilt and change IP).

#!/bin/bash
function get_stack_ip(){
aws ec2 describe-instances \
--fileter "Name=tag-key,Values=aws:cloudformation:stack-name" "Name=tag-value,Values=$1" \
|jq '.Reservations[] |select(.Instance[0].PrivateIpAddress != null).Instance[0].PrivateIpAddress' \
|tr -d '"'
}

Then it’s easy to use this function to get IPs of the bastion stack and the target stack, such as:

IP_BASTION=$(get_stack_ip bastion_stack)
IP_TARGET=$(get_stack_ip target_stack)
ssh -o ProxyCommand="ssh [email protected]_BASTION nc %h %p" [email protected]_TARGET

🙂

Run Fedora 29 on Dell XPS 15 9570

Here’s a list of things to do to get Fedora 29 running optimally on Dell XPS 15 9570:

First, disable Secure Boot with the stock Windows 10 and in BIOS otherwise Fedora installer on a USB stick won’t boot. I still don’t really see the necessity to have this Secure Boot, except to buy more time for Windows obviously.

Then I need to set SATA mode from RAID to AHCI in BIOS, or the Linux installer can’t find the drive. The SATA mode was set to RAID ON, which probably makes more sense if there’s 1 more drive in the laptop.

Hit F12 to choose boot device and install Fedora 29 using a USB drive, then the laptop will be booted into Fedora Live.

There were some warnings regarding nouveau drive so I had to disable nouveau and turn off nvidia device at start. The way bbswitch is installed has changed a bit so I installed it following this. After the nvidia device disabled at boot, the laptop is much much quieter.

According to Arch Linux Wiki, the laptop uses S2 suspend instead of S3. This can be fixed by added mem_sleep_default=deep to kernel parameters and then

grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

and reboot. The result can be verified by

$ cat /sys/power/mem_sleep
s2idle [deep]

I’ll see how long the battery can hold up. 🙂

依依不舍的2018

2018年已经过去了,我这才意识到。当事事都很顺利时,时间就过的很快,仿佛是快进模式。

有一件事让我受益匪浅。

2017年年底时,我老板得到更好的职位于是离开了AFL,这让我多少有些失望,因为我觉得我跟他蛮和得来的。和我这有棱有角的脾气能和得来的也确实不多。后来新老板来了我想,这位没准儿跟我就和不来吧。幸运的是事情不是像我预测的那样糟。

老板是意大利裔,在土澳长大。他居然对中国特别是四川颇有了解,因为他曾经在重庆的一家外企带过队伍。当时他面对食品一条街从头到尾的红辣椒真是一筹莫展,庆幸可口可乐里没有加辣椒。

老板最初的几个小项目我都参与并顺利完成了,这是个好的开端。一次我们谈工作时,我提出团队工作流程里一些残留问题。他问:为什么这个问题一直没得到解决呢?我说:困难不在技术层面,而是资深员工不赞成革新,可能需要领导层面介入才行。他没有反对,于是我期待他能在团队会议上提出这个问题并执行革新,但这迟迟没有发生。

后来老板做了团队拓展活动,离开办公室去讨论内部问题,包括工作流程问题,时间是一整天。我有些不解:革新方案是明摆着的,上行下效不就完事了么?为啥用一天?会议中,保守派给出了很多工作流程没有改变的历史原因,作为改良派的牵头人,我则据理力争给出革新后的工作流程的优势。然后全部团队成员投票,改良派的方案以一票之优势胜出!

事后老板跟我说,他当然知道改良派的方案是更好的,但是土澳人民的价值观在这就体现了出来,每个人的意见都很重要,因此得到多数人的支持才是最重要的。老板笑着说,比起中国,土澳做事情的确很慢,一个项目可能要争论很久才能批下来,修一条路,如果要拆迁的居民不同意,那就只能想办法绕开。这对我的启示真是不小,因为我尽管自以为自己支持民主,但事到临头时还是希望把自己的观点灌输给别人。

入乡随俗,还有很多事物需要学习。:)