I ran iozone on many different ZFS pool configurations, to get an idea of which drives are best for L2ARC (cache) and the ZIL. I also wanted to get an idea of whether using gpart affects performance.
The configurations shown in the tables below have the formation [cache]_[gpart/gnop]_[zil]. Where [cache] is the L2ARC type. [gpart/gnop] says whether it’s set up under gpart, or direct. I call the direct method gnop since I have to initially use gnop to force a 4K sector size (I’m using the Western Digital Advanced Format drives for my pool). When I use gpart, I layer gnop on top of the initial creation to force 4K sector size. [zil] specifies the ZFS intent log device.
The disk types are:
Sandisk-2x8GB: these are two 8GB SATA SSD’s connected to the same controller (a PCI SATAII controller).
STT-2x8GB-usb: these are two USB 3.0 8GB drives connected to USB 2.0 ports.
I ran iozone using the command iozone -a -g 8G, and then looking only at the file sizes 1G or above. I have the whole results to make sure nothing funny happens for smaller file sizes. The results below are the average over all file sizes 1G or above and all record sizes performed at those file sizes.
| Config | Writer | Re-Writer | Reader | Re-Reader | Random Read | Random Write | Backward Read | Record Rewrite | Stride Read | Fwrite | Re-fwrite | Fread | Re-fread |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| STT-2x8GB-USB-cache_gnop_no-zil | 75582 | 68640 | 57186 | 48690 | 35297 | 67435 | 43228 | 1018166 | 38217 | 68893 | 67935 | 53560 | 44769 |
| SanDisk-2x8GB-cache_gnop_no-zil | 76220 | 71511 | 73913 | 76134 | 72732 | 70053 | 60786 | 1001725 | 75326 | 69631 | 70712 | 67502 | 70613 |
| corsair-16GB-usb-cache_gnop_no-zil | 76683 | 72090 | 57963 | 49971 | 36033 | 68870 | 44593 | 1028012 | 37445 | 69079 | 71713 | 55417 | 46366 |
| no-cache_gnop_no-zil | 78223 | 72358 | 77134 | 78507 | 45160 | 67282 | 54004 | 1002780 | 44819 | 69790 | 70766 | 73954 | 72714 |
| Kingston-64GB-sata-cache_gnop_SanDisk-2x8GB-zil | 60654 | 53490 | 84669 | 93001 | 94166 | 60960 | 68226 | 1048091 | 88588 | 54835 | 54295 | 83393 | 90911 |
| Kingston-64GB-sata-cache_gnop_no-zil | 77465 | 72126 | 85473 | 93171 | 94973 | 69970 | 62113 | 995109 | 95945 | 71804 | 72567 | 75761 | 82436 |
| Kingston-64GB-sata-cache_gpart_SanDisk-2x8GB-zil | 60144 | 55071 | 87213 | 94107 | 95649 | 59756 | 65990 | 1012787 | 86949 | 55410 | 55610 | 83307 | 91718 |
| Kingston-64GB-sata-cache_gpart_no-zil | 76406 | 72145 | 82577 | 90769 | 92336 | 70305 | 64775 | 1019952 | 97475 | 72893 | 72343 | 75447 | 85428 |
I also computed the delta in bandwidth for a re-action versus action. For example, Delta Re-Write is throughput of re-write – throughput of write.
| Config | Delta Re-read | Delta Re-Write | Delta Re-fwrite | Delta Re-fread |
|---|---|---|---|---|
| iozone_STT-2x8GB-USB-cache_gnop_no-zil | -8495 | -6941 | -958 | -8791 |
| iozone_SanDisk-2x8GB-cache_gnop_no-zil | 2220 | -4709 | 1081 | 3111 |
| iozone_corsair-16GB-usb-cache_gnop_no-zil | -7991 | -4593 | 2634 | -9051 |
| iozone_no-cache_gnop_no-zil | 1372 | -5865 | 975 | -1239 |
| iozone_Kingston-64GB-sata-cache_gnop_SanDisk-2x8GB-zil | 8331 | -7164 | -539 | 7517 |
| iozone_Kingston-64GB-sata-cache_gnop_no-zil | 7697 | -5338 | 762 | 6674 |
| iozone_Kingston-64GB-sata-cache_gpart_SanDisk-2x8GB-zil | 6893 | -5072 | 200 | 8410 |
| iozone_Kingston-64GB-sata-cache_gpart_no-zil | 8192 | -4260 | -550 | 9980 |
Despite some things I’ve read, USB 2.0 devices can adversely affect the performance. One can see that many cases with the USB devices used as caches, the re-read delta is 8MBps slower.
Another curiousity is that the ZIL devices actually slow things down for writes (and re-writes). I’m not sure if iozone uses synchronous writes, but I did see that the ZIL devices had activity during the tests, so I must conclude that something is going on there. Since I’m using this mostly as a Samba server, I’m probably going to skip the ZIL in my setup.
Post a Comment