Hey there π I would love to learn more about your thoughts on onboarding software engineers and the challenges you're facing in your company. If you can spare 30 minutes of your time, I'd love to chat with you! Just send me an email!
Building and pushing Docker images is one of the most common tasks in the deployment pipeline of most engineering teams. We usually want to build the images ahead of time to create a repository of images we can use for our deployments, or are required to do so in case of serverless deployments where you just supply your image tag.
With GitHub Actions, CI has moved closer to the source code, and there are numerous ways to build and push Docker images. Iβve found a workflow that I use for all my side projects, which I want to share here.
I primarily organize code deployments around commits. This works well in combination with squashing pull requests into one commit that should produce a working system, so in theory, commits can be deployed and rolled back individually.
For container images that means weβll use the GitHub commit SHA as a tag. What remains is to decide whether the latest tag should be assigned. I think the latest tag leads to a lot of issues (declarative deployments not knowing when the image changed, unexpected breaking changes, etc.) that could be avoided altogether if we just used a consistent version.
This is why I started to restrict image tags to the commit SHA of a squash commit in the monorepo, effectively creating a link between image and code. Of course, this doesnβt help if you cannot figure out which version is running at a given point in time. For this, I set a build argument and turned it into an environment variable available at runtime.
This environment variable could then be used by the containerized application for monitoring, error reporting, etc.
ARG commit_sha=unknown
ENV BUILD_COMMIT_SHA=$commit_sha
In a couple of projects, I built multiple services with one codebase, allowing to share code easily, compared to using private packages or workspace support in the package manager. This flow used a single repository (monorepo) and was split up in frontend and backend, where the backend was a simple TypeScript setup with multiple entry points. I then exposed these entry points as targets in a Dockerfile, combining all possible variations of the service in one file.
FROM node:16.13.0-alpine3.13 AS build
...
FROM node:16.13.0-alpine3.13 AS runtime
...
FROM node:16.13.0-alpine3.13 AS api
...
FROM node:16.13.0-alpine3.13 AS auth
...
To build any particular service, you have to specify the target (e.g. api
, auth
).
In more recent projects, I started using go modules and pnpm workspaces for referring to internal libraries and components, which created a clear separation between services compared to stuffing multiple services in one setup.
As libraries and services often live in a different part of the repository hierarchy, it becomes necessary to build from the directory root. This is done by specifying a context to the Docker build process and a separate location for the Dockerfile, as it lives near the service to be built, not the root.
Letβs say our directory structure looks like this
.
βββ libraries
β βββ shared-go
β βββ shared-node
βββ services
βββ api
β βββ Dockerfile
βββ worker
βββ Dockerfile
βββ go.mod
In this case, if we want to build the API which depends on the shared-node
library, we must copy both resources into the Dockerfile, which is only possible by running Docker build from the root directory.
While itβs easy to specify a context, be careful that you only include files you really need. By default, docker build
will load all files in all subdirectories of the context, so if you have a monorepo which includes archives, media, binaries, and other types of large files, it will add unnecessary bloat to the image and makes the overall build much slower.
You can ignore files and directories by placing a .dockerignore
file in the directory you specified as context. This file is similar to .gitignore
except docker build
only takes a single .dockerignore
file into account, unlike .gitignore
which allows to ignore files in any directory.
The last point doesnβt need any special preparation but is still very helpful: Instead of using an external package registry, Iβve been using GitHub Packages for a while now, and it works just fine. One advantage of using GitHub Packages, other than quicker image uploads, is that you wonβt need to worry about authenticating, as you can simply use the provided GITHUB_TOKEN
in GitHub Actions.
Now, itβs pretty straightforward to configure a GitHub Action that can create and push images tagged with the Git commit SHA. But if you have several projects or repositories, copying the action over every time becomes frustrating. This is why I published the GitHub Action Iβm using for my projects in a public repository. I highly recommend using this not as a final setup but rather as a starting point for your own configuration, as you might care about different things, want a latest tag, etc.
With the action, building and pushing Docker images for a service is reduced calling the action with the required inputs.
name: Build images
on:
workflow_dispatch:
push:
branches:
- main
jobs:
build-images:
name: Build service image
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
# uses latest version of the action
- uses: brunoscheufler/docker-build-push@main
with:
imageName: my-service
context: .
commit: ${{ github.sha }}
registryBaseUrl: ghcr.io
registryRepository: my-repo
registryUsername: my-org
registryPassword: ${{ secrets.GITHUB_TOKEN }}
Thanks for reading this post π I would love to learn more about your thoughts on onboarding software engineers and the challenges you're facing in your company. If you can spare 30 minutes of your time, I'd love to chat with you! Just send me an email!