Running Docker on Apple Silicon M1

If you’re anything like me, you’ve been both terribly excited to get your hands on a Mac with an M1 chip – but at the same time apprehensive if all your favourite dev tools are going to work on it. And for me, high on that list of tools is Docker.

As of writing, Docker for Mac does not run on Apple Silicon, and so my quest for a work-around began even before I received my M1 Macbook Air, by setting up a Raspberry Pi with Docker and connecting to it over the network.

While this works fine, it’s not the most speedy way to run Docker. And so in my search for better alternatives I came across a tweet by Jacopo Mangiacacchi. He has created a command line utility that uses Apple’s Hypervisor framework to boot a Linux VM. This piqued my interest, because once we’ve got Linux running, adding Docker is but a small step.

Alongside Jacopo’s project there are a few others that aim to do the same thing:

The latter is the project I used in my first successful attempt to run a Postgres database in Docker on an M1 machine.

Outline

Here’s an outline of what’s needed to run this experiment:

  1. download a Linux ISO image
  2. extract the kernel and initrd files from it
  3. build the command line client vftool to launch the VM
  4. launch the VM and connect to it
  5. install Docker
  6. fix an issue with aufs, which (I believe) is due to running from a live CD

None of these steps are overly complex. In fact many of them are simple copy-and-paste commands for the command line. Let’s go through them one by one.

Download the Linux ISO image

To get this to work we need an ARM Linux ISO image. I didn’t do a whole lot of research and ended up downloading an image from this link: 64-bit ARM (ARMv8/AArch64) desktop image.

Then we follow the instructions as laid out in Jacopo’s README:

sudo mkdir /Volumes/Ubuntu
sudo hdiutil attach -nomount ~/Downloads/focal-desktop-arm64.iso
sudo mount -t cd9660 /dev/diskN /Volumes/Ubuntu # see below for how to determine N
open /Volumes/Ubuntu/casper

In order to determine which partition number N to use, check the output of the hdiutil attach command:

❯ sudo hdiutil attach -nomount focal-desktop-arm64.iso
/dev/disk4              FDisk_partition_scheme
/dev/disk4s1            0xCD
/dev/disk4s2            0xEF

We’re looking for the FDisk_partition_scheme partition.

Extract vmlinuz and initrd

We have now mounted the ISO image and opened the folder with vmlinuz and initrd, which we’re looking to extract. Copy them to the folder alongside your ISO image.

In order to be able to launch the kernel image, you need to unzip it as follows:

cp vmlinuz vmlinuz.gz
gunzip vmlinuz.gz

Build vftool

Next we’ll compile the command line tool to boot the vm. Clone the repository and build it:

git clone https://github.com/evansm7/vftool
cd vftool && xcodebuild

The binary is in build/Release and you can verify it’s running by launching it without arguments:

❯ ./build/Release/vftool
Syntax:
    ./build/Release/vftool <options>

Options are:
    -k <kernel path> [REQUIRED]
    -a <kernel cmdline arguments>
    -i <initrd path>
    -d <disc image path>
    -c <CDROM image path>
    -b <bridged ethernet interface> [otherwise NAT]
    -p <number of processors>
    -m <memory size in MB>

Launch the virtual machine

Now we’re ready to launch the Linux VM. I’ve copied vftool to the folder containing the ISO image and files vmlinuz and initrd and am running it as follows:

./vftool -k vmlinuz -i initrd -d focal-desktop-arm64.iso -m 4096 -a "console=hvc0"

Doing so should give you the following output:

❯ ./vftool -k vmlinuz -i initrd -d focal-desktop-arm64.iso -m 4096 -a "console=hvc0"
2020-11-26 20:18:25.753 vftool[5068:113057] vftool (v0.1 25/11/2020) starting
2020-11-26 20:18:25.753 vftool[5068:113057] +++ kernel at vmlinuz -- file:///Users/sas/Downloads/linux-vm/, initrd at initrd -- file:///Users/sas/Downloads/linux-vm/, cmdline 'console=hvc0 root=/dev/vda1', 1 cpus, 4096MB memory
2020-11-26 20:18:25.759 vftool[5068:113057] +++ fd 3 connected to /dev/ttys004
2020-11-26 20:18:25.759 vftool[5068:113057] +++ Waiting for connection to:  /dev/ttys004
2020-11-26 20:18:56.444 vftool[5068:113057] +++ Attaching disc focal-desktop-arm64.iso -- file:///Users/sas/Downloads/linux-vm/
2020-11-26 20:18:56.444 vftool[5068:113057] +++ Configuration validated.
2020-11-26 20:18:56.445 vftool[5068:113057] +++ canStart = 1, vm state 0
2020-11-26 20:18:56.846 vftool[5068:113446] +++ VM started

The important bit here is the terminal (tty) the VM is listening on: /dev/ttys004. This is how we are going to connect via screen:

screen /dev/ttys004

Connecting to the tty will start the VM’s boot process and you should see the familiar Linux boot sequence before arriving at the login screen:

The default user for the image I’ve linked to above is ubuntu, with no password.

Install Docker

Now that we’ve logged in, we can install Docker following the usual instructions. In a nutshell:

sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
   "deb [arch=arm64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

Note that we’re specifying [arch=arm64] here, so be careful when copy-pasting instructions, because the Docker instructions will be showing amd64 instead, for the x86 version.

For easier launching of Docker without sudo, let’s add ubuntu to the docker group and re-login:

sudo usermod -aG docker ${USER}
su - ${USER}

Run Docker

We’re almost there now, just a small issue to fix with an error which you’ll see if you run a docker container now:

ubuntu@ubuntu:~$ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
256ab8fe8778: Pull complete
Digest: sha256:e7c70bb24b462baa86c102610182e3efcb12a04854e8c582838d92970a09f323
Status: Downloaded newer image for hello-world:latest
[  629.331388] aufs test_add:264:dockerd[4370]: already stacked, /var/lib/docker/aufs/diff/65cc8680a79dddd4ac07ddc823f93c8de7301f58d43288fd4029041cecbd3269-init (overlay)
docker: Error response from daemon: error creating aufs mount to /var/lib/docker/aufs/mnt/65cc8680a79dddd4ac07ddc823f93c8de7301f58d43288fd4029041cecbd3269-init: mount target=/var/lib/docker/aufs/mnt/65cc8680a79dddd4ac07ddc823f93c8de7301f58d43288fd4029041cecbd3269-init data=br:/var/lib/docker/aufs/diff/65cc8680a79dddd4ac07ddc823f93c8de7301f58d43288fd4029041cecbd3269-init=rw:/var/lib/docker/aufs/diff/788ff1c3c3fac57d15c60b22ea668afe9453fd553ae23d940f7c6b9f6ad100ae=ro+wh,dio,xino=/dev/shm/aufs.xino: invalid argument.
See 'docker run --help'.

This error probably relates to us running off a live CD image and Stack Overflow has a work-around:

sudo sh -c "cat <<EOF > /etc/docker/daemon.json
{
    \"storage-driver\": \"vfs\"
}
EOF"
sudo service docker restart

With this we can successfully launch a container 🎉

docker run hello-world

One thing to bear in mind is the following the Docker docs have to say about the vfs driver:

The vfs storage driver is intended for testing purposes, and for situations where no copy-on-write filesystem can be used. Performance of this storage driver is poor, and is not generally recommended for production use.

It shouldn’t be hard to fix this issue properly (and please get in touch if you know how!) but in the meantime, this is already helping me run tests against a Postgres database running in Docker entirely on my M1 machine – and I hope it will help you get started with Docker on Apple Silicon as well.

Follow-up posts

If you want to set up your system to be able to run docker commands on your Mac without having to connect to the VM first, see this follow-up post.

In order to set up a VM with a persisted volume, check out this article.

If you have any questions or comments, please get in touch!