ARM64 Docker Images
ARM64 is a fascinating contender in the world of computing. It is incredibly prevelant in mobile devices like smartphones, probably most notably due to the efficiency, but also heat generation and power usage constraints. ARM64 had never really been anything to even talk about when it came to computing or cloud providers until recently.
The giants like AWS with Graviton2, and Azure, Google, and Oracle with Ampere Altra have all made a push towards ARM64 cloud compute, with significant cost savings and performance improvements for some workloads.
Sounds great and all (and it is), but really only when you're rolling your own software and you own every step of the CI and CD process. Building for ARM64 is easy with Docker, all you need is an ARM64 machine, or QEMU to emulate a different architecture.
Why?
Oracle Cloud Infrastructure and a Raspberry Pi 4 Model B really pushed me to explore how I can shift my services and self-hosted software over to ARM64. Considering Oracle give away a 4 vCPU 24GB RAM Ampere Altra compute instance for free. The problem is that not all software I want to run is built for ARM64, and this introduced a few challenges.
How?
Docker images were an obvious first step, the ability to build a single image tag for multiple architectures using Buildx was very appealing (initially I was also experimenting with multi-architecture Kubernetes clusters - which is an entirely different set of challenges). And I went through a few different iterations of my build process before settling on the standard below:
- Github Actions
- Github hosted runners
- Docker buildx
- Appending an Ampere Altra Oracle compute instance with 1 vCPU and 8GB RAM to the Docker buildx context using SSH
Doing this I am able to natively build AMD64 and ARM64 images at a reasonable performance level.
In the past I have successfully used QEMU to build my own software for AMD64 and ARM64. We have also had success with this in production at my workplace. But I did run into a couple of issues when using QEMU to emulate ARM64 for multi-architecture builds in one instance and I wasn't able to work around this.
Additionally, emulating ARM64 using QEMU is significantly slower than native. So don't be alarmed at longer build times if you go down this route.
To get started building multi-architecture images, the Docker buildx manual is really all that's needed.
My Github repo with all the code to build the images can be found here maxtroughear/docker-images
ClamAV
ClamAV isn't the first image I have rebuilt for ARM64, but it was one of the easiest and is what I would consider a standardised template to use for most other image builds.
Building ClamAV as a docker image was a prerequisite for building mailcow-dockerized, which is detailed below.
mailcow-dockerized
mailcow-dockizered now natively supports ARM64. Please use the natively supported versions instead (starting from 2024-01)
mailcow-dockizered is a collection of images that have been linked together using a docker network to create a relatively easy to manage all-in-one mail server.
Due to the way that the mailcow images are built, it is not possible to simply copy paste the afore mentioned ClamAV build workflow for the mailcow-dockerized images. There are a number of issues that must be addressed.
Deploying
Updating
cd /opt/mailcow-dockerized
sudo ./update.sh --skip-start
wget -q --output-document - https://github.com/maxtroughear/docker-images/raw/main/mailcow-dockerized-arm64/patch.sh | sudo bash
sudo docker compose pull
sudo docker compose up -d --remove-orphans --no-build
sudo docker image prune
Building multiple images
Due to the number of images that need to be built, it is possible to use docker-compose to define the build files and image repos and tags. I've overritten the image repos and tags in my own docker-compose.override.yml so that the images are stored in my Github Container Repository with tags that are still inline with the original mailcow-dockerized tags.
It would be annoying to have to write out each individual docker buildx build command for the number of images that mailcow has. docker compose build works to build every image of a docker-compose.yml file, but it doesn't support buildx and multi-architecture builds. Lucky for us we have docker buildx bake which allows for parallelised multi-architecture builds from a range of different sources:
docker-compose.ymldocker-compose.yamldocker-bake.jsondocker-bake.override.jsondocker-bake.hcldocker-bake.override.hcl
Making Dockerfiles compatible with ARM64
Some images clamd, dovecot, rspamd, and sogo aren't compatible with ARM64 out-of-the-box and require some modification.
clamd
For the original mailcow-dockerized clamd image, the official clamav/clamav image is used as a base. The official image does not yet support ARM64 so the previously mentioned ClamAV image is used instead.
dovecot
The original mailcow-dockerized dovecot image installs debian packages from the official dovecot apt repo. This repo only contains binaries for AMD64 so it cannot be used. In order to get up-to-date versions of dovecot, I resorted to using debian:bookworm-slim as a base image as it has recent versions of dovecot available in it's built-in repos. In the future I would like to avoid using a testing version of Debian as the base and instead build dovecot from source so that I can better control the version and packages used.
rspamd
Much the same as dovecot the rspamd official apt repo only contains AMD64 binaries so it cannot be used out-of-the-box. In this case I opted for using alpine:3.16 as the base image and installing rspamd from the alpine repos. In the future I would like to avoid using an entirely different operating system as the base image and instead build rspamd from source for the same reasons as dovecot.
sogo
Work-in-progress