ARM, docker, linux, qemu

Running and building ARM Docker containers in x86

We already covered how Linux executes files and how to run ARM binaries “natively” in Linux in the last two posts. We ended up running a whole ARM root filesystem transparently in a chroot jail using QEMU user mode and binfmt_misc support.

Now that we have that covered, nothing prevents us from applying that to Docker containers. At the end of the day, containers are chroots in steroids, and they also share kernel with the host so all the basic mechanics remain the same.

Running ARM containers

If you haven’t yet, install QEMU user mode, by default it will install also binfmt_misc support.

Now,  we only need to place the qemu-arm-static interpreter inside the container. This is as easy as mount-binding it in the container invocation and voila! We are running an armhf container in our x86 machine.

Building ARM containers

This is a bit of an uglier trick, because we need to copy the QEMU interpreter to the container. The problem is that it adds ~5MiB to the container, but the benefit of being able to build foreign containers natively is immense!

Consider the following simplified Dockerfile to create an armhf nginx container

We end up with a container that can be run both natively in an ARM device, and in an x86 system with a properly configured binfmt_misc support. Not bad!

We can even test it locally

We are now running the ARM nginx web server locally.

This means that we can create a build pipeline with automated testing without requiring any ARM boards. After all automated tests are passing, we can create the production image.

Reclaiming the space in Docker API 1.25+

If we really want to reclaim those extra 5MiB from qemu-arm-static, we can use the new experimental --squash  flag.

Honestly, it was about time that they did something to help us keep the image size down. The piling up of layers and their refusal to add a simple --chown  argument to the COPY command made the creation of Dockerfiles a black art where you had to chain all the installation steps including the cleanup in a single RUN statement, therefore denying the benefits of caching for big blocks of code.

Thankfully, they are beginning to react and we now have not only  --chown , but also the --squash  flag for the build command. In order to use it, we need to enable experimental features. Add the following to the daemon.json file, which might not exist

Restart the Docker daemon, after this.

Reclaiming the space in Docker API <1.25

We are left with the old hacky way of flattening the images, by exporting the container

, and finally rebuild the metadata

Author: nachoparker

Humbly sharing things that I find useful [ github dockerhub ]

2 Comments on “Running and building ARM Docker containers in x86

  1. Thanks a lot for posting!

    I tried this on Ubuntu 16.04 and upon the first step I got an error saying standard_init_linux.go:190: exec user process caused "no such file or directory". It seems that /usr/bin/qemu-arm-static was missing and I needed to install qemu-user-static using apt-get install qemu-user-static. After that I was set to go.

Leave a Reply

Your email address will not be published. Required fields are marked *