This is kind of a textbook case that container is much more efficient than VM. The CI pipeline in comparison uses AWS CloudFormation to build new VMs and drain old VMs to do a rolling update, which takes around 10 minutes for everything even if it’s just 1 line of code changed. I did a new pipeline with BuildKite and Kubernetes and a deploy is done within 2 minutes.

The key points to make the pipeline fast are:

1. In the Dockerfile, the part that changes more frequently should be put at the bottom of the file, so that docker can maximise its build speed by using cached intermediate images.
2. Reload Kubernetes config maps with this:

   kubectl create configmap nginx-config --from-file path/to/nginx.conf -o yaml --dry-run |kubectl replace -f -

3. Reload containers with this(I use ECR):
   $BUILDKITE_BUILD_NUMBER obviously is the build number environment variable provided by BuildKite.

   kubectl set image deployment/my_deployment \
    nginx=my.ecr.amazonaws.com/nginx:$BUILDKITE_BUILD_NUMBER \
    php=my.ecr.amazonaws.com/php:$BUILDKITE_BUILD_NUMBER

4. Finally watch rolling update progress with this command:

   kubectl rollout status deployment/my_deployment

🙂