Skip to content

ZFS performance metrics with iozone

I ran iozone on many different ZFS pool configurations, to get an idea of which drives are best for L2ARC (cache) and the ZIL. I also wanted to get an idea of whether using gpart affects performance.

The configurations shown in the tables below have the formation [cache]_[gpart/gnop]_[zil]. Where [cache] is the L2ARC type. [gpart/gnop] says whether it’s set up under gpart, or direct. I call the direct method gnop since I have to initially use gnop to force a 4K sector size (I’m using the Western Digital Advanced Format drives for my pool). When I use gpart, I layer gnop on top of the initial creation to force 4K sector size. [zil] specifies the ZFS intent log device.

The disk types are:

Sandisk-2x8GB: these are two 8GB SATA SSD’s connected to the same controller (a PCI SATAII controller).
STT-2x8GB-usb: these are two USB 3.0 8GB drives connected to USB 2.0 ports.

I ran iozone using the command iozone -a -g 8G, and then looking only at the file sizes 1G or above. I have the whole results to make sure nothing funny happens for smaller file sizes. The results below are the average over all file sizes 1G or above and all record sizes performed at those file sizes.

Config Writer Re-Writer Reader Re-Reader Random Read Random Write Backward Read Record Rewrite Stride Read Fwrite Re-fwrite Fread Re-fread
STT-2x8GB-USB-cache_gnop_no-zil 75582 68640 57186 48690 35297 67435 43228 1018166 38217 68893 67935 53560 44769
SanDisk-2x8GB-cache_gnop_no-zil 76220 71511 73913 76134 72732 70053 60786 1001725 75326 69631 70712 67502 70613
corsair-16GB-usb-cache_gnop_no-zil 76683 72090 57963 49971 36033 68870 44593 1028012 37445 69079 71713 55417 46366
no-cache_gnop_no-zil 78223 72358 77134 78507 45160 67282 54004 1002780 44819 69790 70766 73954 72714
Kingston-64GB-sata-cache_gnop_SanDisk-2x8GB-zil 60654 53490 84669 93001 94166 60960 68226 1048091 88588 54835 54295 83393 90911
Kingston-64GB-sata-cache_gnop_no-zil 77465 72126 85473 93171 94973 69970 62113 995109 95945 71804 72567 75761 82436
Kingston-64GB-sata-cache_gpart_SanDisk-2x8GB-zil 60144 55071 87213 94107 95649 59756 65990 1012787 86949 55410 55610 83307 91718
Kingston-64GB-sata-cache_gpart_no-zil 76406 72145 82577 90769 92336 70305 64775 1019952 97475 72893 72343 75447 85428

I also computed the delta in bandwidth for a re-action versus action. For example, Delta Re-Write is throughput of re-write – throughput of write.

Config Delta Re-read Delta Re-Write Delta Re-fwrite Delta Re-fread
iozone_STT-2x8GB-USB-cache_gnop_no-zil -8495 -6941 -958 -8791
iozone_SanDisk-2x8GB-cache_gnop_no-zil 2220 -4709 1081 3111
iozone_corsair-16GB-usb-cache_gnop_no-zil -7991 -4593 2634 -9051
iozone_no-cache_gnop_no-zil 1372 -5865 975 -1239
iozone_Kingston-64GB-sata-cache_gnop_SanDisk-2x8GB-zil 8331 -7164 -539 7517
iozone_Kingston-64GB-sata-cache_gnop_no-zil 7697 -5338 762 6674
iozone_Kingston-64GB-sata-cache_gpart_SanDisk-2x8GB-zil 6893 -5072 200 8410
iozone_Kingston-64GB-sata-cache_gpart_no-zil 8192 -4260 -550 9980

Despite some things I’ve read, USB 2.0 devices can adversely affect the performance. One can see that many cases with the USB devices used as caches, the re-read delta is 8MBps slower.

Another curiousity is that the ZIL devices actually slow things down for writes (and re-writes). I’m not sure if iozone uses synchronous writes, but I did see that the ZIL devices had activity during the tests, so I must conclude that something is going on there. Since I’m using this mostly as a Samba server, I’m probably going to skip the ZIL in my setup.

Be the first to like.

Post a Comment

Your email is never published nor shared. Required fields are marked *
*
*