Running Docker on Apple Silicon M1 (persisted volume)

This is the third post in my series about using Docker on an M1 MacBook Air. You can find the first two parts here:

These two articles describe the basic setup to get Docker up-and-running on an M1 Mac. The big downside of the presented solution is that it lives in RAM only, meaning that any time you need to reboot your machine or stop the process running the VM, all changes are lost.

While parts of the setup are permanent and reinstalling Docker and a few other customisations is as easy as pasting a few shell commands in a terminal, taking only a minute or so, it still is rather annoying having to do that, even if you reboot only once or twice a month.

In order to fix this, we’re going to set up a properly installed Linux VM with a persisted volume, based on my first two posts and this article by Callum Smith, with some additions like a fixed IP address.

Here’s an outline of the steps required:

  1. Download an Ubuntu cloud image and associated files
  2. Build Qemu so we can resize the cloud image for our needs
  3. Boot the image and update its settings
  4. Resize the image
  5. Install docker

Once you’re done with these steps you’ll have a Docker virtual machine that is about twice as fast as the Docker Tech Preview 5.

1. Download cloud image and associated files

Let’s get started by downloading the boot images. We need three files to bootstrap the virtual machine:

  • the cloud image
  • the kernel, vmlinuz
  • the initial RAM disk, initrd

You can use the following terminal commands to download them:

curl https://cloud-images.ubuntu.com/releases/focal/release/unpacked/ubuntu-20.04-server-cloudimg-arm64-vmlinuz-generic -o vmlinuz.gz && gunzip vmlinuz.gz
curl https://cloud-images.ubuntu.com/releases/focal/release/unpacked/ubuntu-20.04-server-cloudimg-arm64-initrd-generic -o initrd
curl https://cloud-images.ubuntu.com/releases/focal/release/ubuntu-20.04-server-cloudimg-arm64.tar.gz -o diskimg.tar.gz && tar xvfz diskimg.tar.gz

2. Building Qemu

Next, we need the qemu-img command in order to be able to resize the disk image. Unfortunately, that means we need to build all of Qemu to get it. It’s not particularly difficult but it will take a little leg work.

Install Homebrew if you don’t have it already:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

This will install brew into /opt/homebrew/bin, so make sure it is in your PATH for subsequent commands to work.

You may be tempted to try and install Qemu via brew but when I last tried it (on January 10) it did not build successfully. So instead we’ll build Qemu ourselves.

First, we need to install a few dependencies that Qemu needs in oder to build successfully, and that’s what we installed brew for:

brew install ninja glib pixman pkg-config texinfo nettle gettext libffi

Now we’re ready to clone and build Qemu:

git clone https://github.com/patchew-project/qemu
cd qemu
mkdir build && cd build
../configure --target-list=aarch64-softmmu --extra-cflags=-I/opt/homebrew/opt/gnutls/include --extra-ldflags=-L/opt/homebrew/opt/gnutls/lib
make -j

On an M1, this should take about 4 minutes and you should have a qemu-img command in your build directory afterwards:

❯ file qemu-img
qemu-img: Mach-O 64-bit executable arm64

3. Boot the image

For this step we need vftool, which we can build following the steps outlined in part one of this series:

git clone https://github.com/evansm7/vftool
cd vftool && xcodebuild

The binary is in ./build/Release/vftool.

Run vftool with the following parameters to launch the image:

vftool -k vmlinuz -i initrd -d focal-server-cloudimg-arm64.img -m 4096 -a "console=hvc0"

and in a second terminal connect to the console via screen to kick off the boot process, again as described in part one of this series:

screen /dev/ttys007   # check the launch console for the tty id

Now we’re ready to update a few settings:

mkdir /mnt
mount /dev/vda /mnt
chroot /mnt
touch /etc/cloud/cloud-init.disabled
echo 'root:root' | chpasswd
ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

This mounts the volume and configures the root login. If you’re planning to use this for anything other than testing, you’ll obviously want to use a better password.

We’ll also update the network settings to configure a fixed IP address for the virtual machine. I’ve picked the IP address 192.168.64.17 here, because 17 is an amazing number.

cat <<EOF > /etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by the datasource.  Changes
# to it will not persist across an instance reboot.  To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        enp0s1:
            dhcp4: no
            addresses:
                - 192.168.64.17/24
            gateway4: 192.168.64.1
            nameservers:
                addresses: [8.8.8.8, 1.1.1.1]
    version: 2
EOF

We also need to disable the default configuration:

cat <<EOF > /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
network: {config: disabled}
EOF

Finally, we exit and unmount the volume:

exit
umount /dev/vda

Now stop the VM by typing Ctrl-C in the terminal where we launched the virtual machine so we can proceed with resizing the image.

4. Resize the image

There are two things we need to do here: resize the image with qemu-img on the host and tell the file system inside the VM about the change.

First run

qemu-img resize focal-server-cloudimg-arm64.img +5G

on the host. This increases the size by 5 GB, which you can obviously adjust to your needs.

Then re-run the command to launch the virtual machine – but note the additional root=/dev/vda parameter:

vftool -k vmlinuz -i initrd -d focal-server-cloudimg-arm64.img -m 4096 -a "console=hvc0 root=/dev/vda"

Note: This is also the boot command you’ll be using from now on to launch your VM, more on that below.

As before, run the screen command to connect and boot the virtual machine, then log in via the root credentials you set above and run

resize2fs /dev/vda

inside the VM.

And now we’re set. The image is now ready to receive any other updates you may want to make to it.

You can place the launch command in a file with a .command extension to create a shortcut for launching in Finder. For instance, I’ve placed the VM’s files in /Application/UbuntuVM and created a file /Applications/UbuntuVM/Ubuntu.command:

#!/bin/sh
cd /Applications/UbuntuVM
./vftool -k vmlinuz -i initrd -d image -m 4096 -a "console=hvc0 root=/dev/vda"

Also, consider duplicating your VM after completing the initial setup as a restore point. I’ve noticed that the image can become corrupted when the Mac is rebooted and the VM not shut down cleanly.

5. Install docker

Finally, follow the instructions in the first article to install docker. Or, in a nutshell, run

sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository \
   "deb [arch=arm64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

The second part of the series explains how you can set up a Docker context to make interacting with your Docker virtual machine from the host machine easier.

I hope this helps you getting started with Docker on an M1 Mac. I’ve been using this mechanism for almost two months now and it’s been working great.

If you have any questions or comments, please get in touch!