This is a follow-up on my previous post. I ran iozone both with and without the [ccie]-e[/ccie] flag. This flag includes a sync/flush in the speed calculations. This flush should tell ZFS to commit the in-process data to disk, thus flushing it to the SLOG device (or to the on-disk ZIL).
I ran 4 tests: one without the [ccie]-e[/ccie] flag, and one with it; I then repeated the combination with the SLOG device removed.
Here are the results:
| Sync? | SLOG? | Write Speed (KB/s) |
|---|---|---|
| No | Yes | 2922009 |
| Yes | Yes | 13650 |
| No | No | 3955196 |
| Yes | No | 227990 |
In cases where the sync was disabled, we can see ridiculously fast rates–the ZFS system is just writing to kernel memory and calling it done. (This is a TxG and will be written to disk at worst case periodically.)
In the one case where sync is enabled and the SLOG is in place, the write speed drops considerably–to 13.6 MB/s. This indicates that the sequential rate isn’t really dominating. If it were, it would be closer to 68 MB/s. Instead, it seems like the random-write speed is somehow in the mix. This result is a bit surprising. I will try (tonight) another test with a python script that opens the file with SYNC set, to see what that does.
As you can see below, iozone was run with a record length of 16384K (the max it would allow) and a file size of 163840K (10 records).
Without Sync / With SLOG
Iozone: Performance Test of File I/O
Version $Revision: 3.420 $
Compiled for 64 bit mode.
Build: freebsd
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Wed Apr 2 05:51:16 2014
File size set to 163840 KB
Record Size 16384 KB
Command line used: iozone -s 163840 -r 16384 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkw
KB reclen write rewrite read reread read write rea
163840 16384 2922009 4685031
iozone test complete.
With Sync / With SLOG
Iozone: Performance Test of File I/O
Version $Revision: 3.420 $
Compiled for 64 bit mode.
Build: freebsd
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Wed Apr 2 05:51:41 2014
Include fsync in write timing
File size set to 163840 KB
Record Size 16384 KB
Command line used: iozone -e -s 163840 -r 16384 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkw
KB reclen write rewrite read reread read write rea
163840 16384 13650 11330
iozone test complete.
Remove SLOG
sudo zpool remove tank gpt/tank_zil1
Without Sync / Without SLOG
Iozone: Performance Test of File I/O
Version $Revision: 3.420 $
Compiled for 64 bit mode.
Build: freebsd
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Wed Apr 2 05:52:45 2014
File size set to 163840 KB
Record Size 16384 KB
Command line used: iozone -s 163840 -r 16384 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkw
KB reclen write rewrite read reread read write rea
163840 16384 3955196 5757868
iozone test complete.
With Sync / Without SLOG
Iozone: Performance Test of File I/O
Version $Revision: 3.420 $
Compiled for 64 bit mode.
Build: freebsd
Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
Al Slater, Scott Rhine, Mike Wisner, Ken Goss
Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone,
Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root,
Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer,
Vangel Bojaxhi, Ben England, Vikentsi Lapa.
Run began: Wed Apr 2 05:52:53 2014
Include fsync in write timing
File size set to 163840 KB
Record Size 16384 KB
Command line used: iozone -e -s 163840 -r 16384 -i 0
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkw
KB reclen write rewrite read reread read write rea
163840 16384 227990 232362
iozone test complete.
Post a Comment