Qemu Overlay Images
Efficiently Managing Virtual Machines with QCOW2 Overlays

I provision multiple VMs daily. To streamline this process, I initially created base images for Ubuntu and Rocky Linux using Packer, as these are the distributions I use most frequently.

However, each time I needed a new VM, I found myself doing the following:

1. Making a complete copy of the base image.
2. Booting up the new VM from this copied image.

This approach consumed considerable time and disk space—especially time, hence creating multiple VMs per day quickly becomes tedious.

What I learned (and found genuinely exciting!) is that QCOW2 supports what's known as an overlay image. An overlay image acts as a tiny read-write layer on top of your existing base image.

Here's how it works:

1. You keep your base image untouched.
2. You create an overlay image that's essentially empty and references your base image.
3. All new modifications (writes) go directly into this overlay image, while read operations fetch data from the base image as needed (just like the layered filesystem used in Docker).

This solution was perfect for my needs:

• Overlay images start extremely small (just about 100 KiB in my experience).
• Provisioning time is drastically reduced since I'm no longer copying large disk images.
• Disk space is significantly saved since only the differences from the base image are stored.

Discovering this was a fantastic productivity boost for my daily workflow

Here's a simple example of creating an overlay image:

Suppose you have a base image called ubuntu-base.qcow2, and you want to create an overlay named ubuntu-overlay.qcow2:

qemu-img create -o backing_file=./ubuntu-base.qcow2,backing_fmt=qcow2 -f qcow2 ./ubuntu-overlay.qcow2

• -o backing_file...,backing_fmt=qcow2:
  • -o: Passes extra options.
  • backing_file: Path to the base image.
  • backing_fmt: Format of the base image (must be qcow2).
• -f qcow2: Format of the new overlay image.
• ./ubuntu-overlay.qcow2: Path and name of the new overlay image.

After creating the overlay image, you can verify that it correctly references the base image using the following command:

qemu-img info ./ubuntu-overlay.qcow2

Look for output like this:

image: ubuntu-overlay.qcow2
file format: qcow2
virtual size: 25 GiB (26843545600 bytes)
disk size: 107 MiB
backing file: ./ubuntu-base.qcow2
backing file format: qcow2
...

This confirms that:

• The overlay is in qcow2 format.
• The backing file is correctly set to ubuntu-base.qcow2.
• The backing file format is qcow2.

If these fields are missing or incorrect, the virtual machine may not boot properly.

Backing Image: Also called the base image, this is your original, untouched QCOW2 file. It's the read-only reference used by overlay images.

Overlay Image: This is the read-write layer created on top of the backing image. All modifications are stored here, leaving your backing image untouched.

It's important to remember that if your backing image gets moved or deleted, your overlay image becomes unusable since it depends directly on the backing image.

That's infortinate, but it's OK :)