{"id":662,"date":"2012-02-18T20:44:08","date_gmt":"2012-02-18T20:44:08","guid":{"rendered":"http:\/\/poojanwagh.opalstacked.com\/techblog\/?p=662"},"modified":"2012-02-18T20:44:43","modified_gmt":"2012-02-18T20:44:43","slug":"ssd-investigations-on-my-zfs-server","status":"publish","type":"post","link":"https:\/\/tech.poojanblog.com\/blog\/un\/ssd-investigations-on-my-zfs-server\/","title":{"rendered":"SSD investigations on my ZFS server"},"content":{"rendered":"

From a previous post<\/a>, I noticed the SSD that I’m using in my server showed only a 50 MB\/s write speed. That’s weird because the specs show around 145 MB\/s.<\/p>\n

So, I decided to investigate some more. I removed the SSD from the zpool and did some dd tests on it.<\/p>\n
\nserver# dd if=\/dev\/zero of=\/dev\/ada3 bs=4M count=32
\n32+0 records in
\n32+0 records out
\n134217728 bytes transferred in 1.587130 secs (84566307 bytes\/sec)
\nserver# dd if=\/dev\/zero of=\/dev\/ada3 bs=4M count=128
\n128+0 records in
\n128+0 records out
\n536870912 bytes transferred in 5.888121 secs (91178650 bytes\/sec)
\nserver# dd if=\/dev\/zero of=\/dev\/ada3 bs=4M count=1024
\n1024+0 records in
\n1024+0 records out
\n4294967296 bytes transferred in 45.720133 secs (93940394 bytes\/sec)
\nserver# dd if=\/dev\/zero of=\/dev\/ada3 bs=4M count=16384
\n^C6528+0 records in
\n6527+0 records out
\n27376222208 bytes transferred in 277.460367 secs (98667145 bytes\/sec)
\n<\/code>\n

That last run (which I interrupted) showed 94 MB\/s.<\/p>\n

 <\/p>\n

The disks I am using (in a mirror) as a ZIL are much smaller. Let’s check their write performance:<\/p>\n<\/p>\n

server# zpool status
\npool: backup
\nstate: ONLINE
\nscan: none requested
\nconfig:<\/p>\n

NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 STATE\u00a0\u00a0\u00a0\u00a0 READ WRITE CKSUM
\nbackup\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/zfs-backup\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0<\/p>\n

errors: No known data errors<\/p>\n

pool: tank
\nstate: ONLINE
\nscan: scrub repaired 0 in 12h12m with 0 errors on Sat Feb 11 03:35:06 2012
\nconfig:<\/p>\n

NAME\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 STATE\u00a0\u00a0\u00a0\u00a0 READ WRITE CKSUM
\ntank\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmirror-0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/WD20EARS\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/WD15EARS\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nmirror-2\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/WD5000BPVT\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/Maxtor7Y250M0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\nlogs
\nmirror-1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/zil0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0
\ngpt\/zil1\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 ONLINE\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0\u00a0\u00a0\u00a0\u00a0 0<\/p>\n

errors: No known data errors
\nserver# zpool remove tank mirror-1
\nserver# dd if=\/dev\/zero of=\/dev\/gpt\/zil0 bs=4M count=1024
\n1024+0 records in
\n1024+0 records out
\n4294967296 bytes transferred in 88.818498 secs (48356676 bytes\/sec)
\n<\/code>\n

They are around 46 MB\/s write speed. Hmm. If I were willing to give up the mirror on the ZIL, I could break up the 64 GB SSD into a two-part partition and use one part as a ZIL and the rest as the L2ARC cache.<\/p>\n

I’d also free up a couple of SATA ports in the process–which is probably good, because I’m using a 4X SATA PCI card that’s probably very limited on bandwidth anyway. Rather than pushing this platform to its extreme, it might make sense to live with less.<\/p>\n