Virtio performance and filesystems

Some time ago there was a comment to my post about booting Linux/sparc64 under QEMU, saying that virtio performance sucks.

At first I couldn't reproduce the problem. Tried a few caching strategies, but the I/O performance was not drastically affected by them. And then it came to my mind that the reason might be the guest file system.

Let's take a look:

$ dd if=/dev/zero of=disk-tmp bs=1024k count=300
300+0 records in
300+0 records out
314572800 bytes (315 MB) copied, 0.430909 s, 730 MB/s

  # so, the theoretical maximal raw performance of this partition on my disk is ~ 730 MB/s.

$ sparc64-softmmu/qemu-system-sparc64 -drive file=../boot-disk.qcow2,if=virtio,index=0 -drive file=disk-tmp,if=virtio,index=1
[...]
root@debian-wheezy:~#  mkfs.ext4dev /dev/vdb1
[...]
root@debian-wheezy:~#  mount /dev/vdb1  /mnt/
root@debian-wheezy:~#  dd if=/dev/zero of=/mnt/100M-file bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 3.2384 s, 32.4 MB/s

# while it seems to be 20 times slower than the host speed it is still a lot (the number 730MB/s is the theoretical limit, without performing regular syncs. QEMU does execute syncs to ensure the data integrity for the unlikely case of the crash.

Let's take a look on the read performance:

root@debian-wheezy:~#  umount /mnt/
root@debian-wheezy:~#  mount /dev/vdb1  /mnt/
[  582.673975] EXT4-fs (vdb1): mounted filesystem with ordered data mode. Opts: (null)
root@debian-wheezy:~#  dd if=/mnt/100M-file of=/dev/null bs=1024k
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 1.08423 s, 96.7 MB/s
root@debian-wheezy:~#

This looks even better. But what happens with the usual ext4?

root@debian-wheezy:~#  mkfs.ext4 /dev/vdb1
[...]
root@debian-wheezy:~#  mount /dev/vdb1  /mnt/
root@debian-wheezy:~#  dd if=/dev/zero of=/mnt/100M-file bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 3.54709 s, 29.6 MB/s

Ok, that's a bit less than the newer version. And what about ext3?

root@debian-wheezy:~#  mkfs.ext3 /dev/vdb1
root@debian-wheezy:~#  dd if=/dev/zero of=/mnt/100M-file bs=1024k count=100
100+0 records in
100+0 records out
104857600 bytes (105 MB) copied, 13.8049 s, 7.6 MB/s

Wow. That's slow. The ext3 partition access with the default options is more than 4 times slower than the ext4 partition access with the default options.

The short conclusion: if you need performance, don't use the ext3 file system with the default settings for your virtual machines.

/ happy hacking