Multi Docker Build Task for Azure DevOps Pipelines

Extension Available on Visual Studio Marketplace

A common use case is to build multiple Docker images inside a pipeline, often from the same code repo or Visual Studio solution. The pipeline tasks that come out-of-the-box with Azure DevOps work well where there is a one-to-one mapping between repo and Docker image, but not so well in this case.

This is because the Docker steps require you to explicitly specify the path to the Dockerfile, and only allow exactly one Dockerfile. As a result, you will find yourself creating inefficient pipelines as you copy and paste bits of yaml to deal with each Docker image that you want to build. The desire to be DRY ('don't repeat yourself') is thwarted and you eventually give up trying to parameterize the Dockerfile paths, output image names, tags and architectures.

Our build task is the result of several years of in-house development and use of our own tools, which are now available to all users of Azure DevOps.

• Specify an unlimited number of Docker images to build
• Specify an unlimited number of tags to apply
• Specify private repository login credentials
• Optionally push the images to your container repository
• Works with all container registry providers including Azure Container Registry and Dockerhub

We do not rely on the Azure DevOps 'Service Connection' paradigm, making it easier to script and re-use. Instead, you directly pass in the credentials of your container repository. It is recommend that you store any sensitive credentials in Azure Key Vault, or in library variable secrets.

Case Study

Consider this fairly standard C# solution and project layout:

ParkSquare.Demo (folder)
|-- ParkSquare.Demo.sln
|-- azure-pipelines.yml
|   ...
|
|--	ParkSquare.Demo.WebApp (folder)
|	|--	ParkSquare.Demo.csproj
|	|--	SomeFiles.cs
|	|	...
|	|-	Dockerfile
|
|--	ParkSquare.Demo.Api (folder)
|	|--	ParkSquare.Demo.Api.csproj
|	|--	MoreFiles.cs
|	|	...
|	|--	Dockerfile
|
|-- ParkSquare.Demo.BackendDaemon (folder)
|	|--	ParkSquare.Demo.BackendDaemon.csproj
|	|--	YetMoreFiles.cs
|	|	...
|	|--	Dockerfile

As you can see, there is one solution and three projects - one for the frontend web app, one for an API, and another for some backend processing daemon. Each one builds into its own container. How best to handle this in the Azure DevOps pipeline yaml?

Solution 1 - Directory Masking

One solution would be have three separate pipelines, and mask the folders to prevent changes to different projects from triggering unwanted builds. This can quickly get unwieldy, as the pipeline yaml becomes complicated, specific, and less reusable. It will also cause problems with tools like SonarCloud, as different code could be built and analysed each time, causing SonarCloud statistics to fluctuate wildly. Further, it becomes harder to manage versioning as each component is built at a different time.

Solution 2 - Split Into Three Repositories

Another option is to split the solution into three repositories, one for each project. The resulting build pipelines remain fairly straightforward, but any assemblies that are shared across the three projects will need managing separately, e.g. using a private Nuget feed. Rebuilding the common code each time is possible, but generally undesirable.

Solution 3 - Multi Docker Task

The most elegant solution is to use our Multi Docker Build task. This allows you to retain the solution structure, and build all three images in one step.

Getting Started

To use our task, simply install the extension into your organization and add the task to your pipeline using the UI or by copy/pasting our example YAML.

Task Properties

In all cases, variables are permitted and will be evaluated correctly.

Images To Build

Here you specifiy one or more images that you want to build. The format is:

<docker file>|<image name>|<context path>

Use a new line for each specificiation.

Tags

When building the image, you can specify any tags to be applied. We recommend tagging with a specific version and also 'latest'. This is the default.

Example YAML

- task: multiDockerBuild@1
  displayName: 'Demo Task'
  inputs:
    registryUrl: '$(privateDockerRepoUrl)'
    registryUsername: '$(privateDockerRepoUsername)'
    registryPassword: '$(privateDockerRepoPassword)'
    repositoryName: 'my-repo'
    imagesToBuild: |
      ParkSquare.Demo.Api/Dockerfile|$(Build.Repository.Name)-api|$(Build.ArtifactStagingDirectory)/Api
      ParkSquare.Demo.WebApp/Dockerfile|$(Build.Repository.Name)-web|$(Build.ArtifactStagingDirectory)/WebApp
      ParkSquare.Demo.Daemon/Dockerfile|$(Build.Repository.Name)-daemon|$(Build.ArtifactStagingDirectory)/Daemon
    tags: |
      $(Build.BuildNumber)
      latest
      something-else
    pushAfterBuild: true