Empowering CI/CD workflows with compose-deploy

A few days ago, I published my introductory post on a freshly-released tool called compose-deploy, which can be used to deploy your Docker Compose application to a group of deployment targets using nothing more than ssh and built-in Docker (and Docker Compose) features. Since then, I have tried to think of additional situations where you might have to deploy your application that way and benefit from the ease of using this tool, quickly noticing that support for CI-like environments would be hugely beneficial to the purpose of compose-deploy. This is exactly the reason why I'm writing this blog post, to demonstrate how to wire up your CircleCI setup to automatically deploy your application to one or more servers of choice once you push some code to GitHub (or git remote of choice). In general, CI support is relatively trivial, however, I needed to add some minor updates to get to a point where deployments actually work. Let's just jump right into it.

Apart from a simple hello-world app in addition to your run-of-the-mill Dockerfile and docker-compose.yml, our project directory contains a compose-deploy configuration which should look similar to this:

name: demo-deploy
targets:
  - host: '<Your server address>'
    username: demo
composeFile: ./docker-compose.yml

In addition to that, we'll install compose-deploy as a devDependency for creating a simple yarn script that calls compose-deploy for the sake of having a seemingly simple installation.

Although we'll be using CircleCI for the example beause it's one of the most widely-used CI/CD services, the following steps should roughly work the same with other vendors!

First of all, you have to add your project to CircleCI, which you can simply do by heading over to the 'ADD PROJECTS' button in the sidebar. After selecting your operating system and language (for our example we'll be using Linux & Node), CircleCI prompts you to add a configuration file to your application repository. Stored inside of the .circleci directory, my config.yml looks like the following:

version: 2
jobs:
  build:
    # use Docker executor
    docker:
      # use Node.js CircleCI image
      - image: circleci/node:11
    working_directory: ~/repo
    steps:
      # add deployment ssh keys
      # - add_ssh_keys:
      #    fingerprints:
      #      - 'MY-KEY-FINGERPRINT'

      # clone git repo
      - checkout

      # set up docker
      - setup_remote_docker

      # only include the $DOCKER_REGISTRY env variable
      # if you don't want to use Docker Hub
      - run: |
          docker login \
          --username $DOCKER_USER \
          --password $DOCKER_PASS \
          $DOCKER_REGISTRY

      # install compose-deploy
      - run: yarn install

      # run compose-deploy via yarn
      - run: yarn deploy

I'll explain the important sections of the configuration later on, but for now, you can simply copy the contents of the configuration above into your project. After pushing to your git remote, CircleCI will fully create the project and run your first build. But hold on, we're not done yet! Obviously, we still have to add certain settings like auth credentials for Docker to push our built images, as well as auth details for our deployment targets.

Starting off with important environment variables, head over to the project settings page, navigate to Environment Variables section and add environment variables for DOCKER_USER, DOCKER_PASS, and optionally DOCKER_REGISTRY if you've provided that in your CircleCI workflow config. Continuing with SSH keys, we'll have to generate a new keypair that is authorized to access our deployment target and add the private key under SSH Permissions. We're now done with configuring sensitive data that should be kept out of your potentially public CircleCI config file.

You have now added the necessary environment variables for compose-deploy to set up Docker to be able to push built images to Docker Hub or your registry of choice, in addition to having added a new SSH key, which will be used by CircleCI's ssh-agent. We still need to add this key to our build container, which can be done by copying the displayed private key fingerprint and replacing the placeholder MY-KEY-FINGERPRINT in the sample config.yml from above with that. After uncommenting the add_ssh_keys instruction with the array of fingerprints, you can push your final setup commit, which should now trigger a CircleCI build that actually works 🎉

The whole build procedure simply

• adds the required SSH key from CircleCI to the local ssh-agent instance
• clones the project's git repository
• sets up a Docker instance to be used for building service images
• signs in to your Docker registry using your supplied credentials
• installs compose-deploy from the project's top-level package.json
• runs yarn deploy which simply calls compose-deploy, which
• builds and pushes your images and finally
• deploys your application on the deployment targets

All of the aforementioned steps are relatively simple once you've understood underlying mechanisms like compose-deploy using the exposed UNIX socket from the ssh-agent to accomplish ssh authentication, or the docker login step to be able to push images to the registry. It's all interconnected.

I hope I could help you with setting up your basic CI/CD pipeline for deploying your application with compose-deploy. I'll work on additional features to make these workflows even easier, for example enabling multi-environment deployments to development & production without having to struggle with any unnecessary complications caused by currently inflexible configuration.

If you've got any questions or suggestions, feel free to send a mail 👍