ZFS send performance tuning


I'm using zfs send to make a full backup of my FreeBSD 10 desktop.

It has two ssd drives in a stripe configuration. That gives me over 1GB/s IO speed at any time, but like any stripe set it doubles the risk of failure.

So it's not unlikely to experience a failure some day!

I do backups daily via Bacula and also wanted an initial dump of the full zpool.

During that dump I noticed a lot of performance issues. No suprise considering my endpoint is a cubietruck mounting a moosefs share built from a lot of lowend ARM CPU systems.

So, this command was my way out:


zfs send -R zroot@2014sep | mbuffer -q -m 1G | lzma -zc -1 | \
ssh -c arcfour128 cubie2 \
"mbuffer -s 128k -m 256M | dd of=/mountpoint/backups/myhost/myhost-2014sep.lz bs=1024k"

zfs send -R dumps the whole pool at the specified snapshot point.

mbuffer is used to buffer incoming IO in memory and by that it can iron out the twice-and-tripe network latencies that come with my setup.

lzma is the fastest/best compression to pick, and told to write the incoming data to stdout.

on the remote end, mbuffer received the data again and buffers it while DD is finally writing to the rather slow, yet large, network share.

Performance was still horrible, but nonetheless a lot better than without this.

  • No labels