Skip to content

Adding cache device to ZFS

I went ahead and bought this USB stick to put in the back of my Portwell machine. The idea is to use it as a device cache to speed up my ZFS tank.
Based on this post by Alexander Leidinger, it should reduce latency (when data is in the cache), but not otherwise. I was also curious on whether it would reduce power consumption. I have no idea if FreeBSD powers down USB devices when they are not being actively used.
I don’t write to the disks frequently. At most, there’ll be a read here and there. (I use Allway Sync which keeps some metadata on the remote storage, that does need to get read. However, like rsync, it won’t write unless necessary.)

I had the Kill-a-Watt running continuously (909 hours) before I did this switch, and it measured $29.92/year (at 0.122 cents/kWh, 25.95 kWh). When I looked at it, the Kill-a-Watt registered 28 watts. (I think that’s instantaneous, not average.)

After doing so, I get the following dmesg:

usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ad0: 152627MB <Hitachi HTE543216L9A300 FB2OC45C> at ata0-master UDMA33
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
Root mount waiting for: usbus3 usbus2 usbus1 usbus0
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
Root mount waiting for: usbus3
Root mount waiting for: usbus3
Root mount waiting for: usbus3
uhub3: 8 ports with 8 removable, self powered
ugen3.2: <MICRONET> at usbus3
umass0: <MICRONET FANTOM DRIVE, class 0/0, rev 2.00/2.10, addr 2> on usbus3
umass0:  SCSI over Bulk-Only; quirks = 0x0000
Root mount waiting for: usbus3
umass0:0:0:-1: Attached to scbus0
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <MICRONET FANTOM DRIVE 2.10> Fixed Direct Access SCSI-4 device
da0: 40.000MB/s transfers
da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C)
ugen3.3: <SMI Corporation> at usbus3
umass1: <SMI Corporation USB, class 0/0, rev 2.00/0.00, addr 3> on usbus3
umass1:  SCSI over Bulk-Only; quirks = 0x0000
umass1:1:1:-1: Attached to scbus1
da1 at umass-sim1 bus 1 target 0 lun 0
da1: <ST 16GB 0000> Removable Direct Access SCSI-0 device
da1: 40.000MB/s transfers
da1: 15296MB (31326208 512 byte sectors: 255H 63S/T 1949C)
usbd_set_config_index:523: could not read device status: USB_ERR_SHORT_XFER
ugen3.4: <Simpletech> at usbus3
umass2: <MSC Bulk-Only Transfer> on usbus3
umass2:  SCSI over Bulk-Only; quirks = 0x0000
umass2:2:2:-1: Attached to scbus2
da2 at umass-sim2 bus 2 target 0 lun 0
da2: <Hitachi HDS721010CLA332 > Fixed Direct Access SCSI-2 device
da2: 40.000MB/s transfers
da2: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
GEOM: da1: partition 1 does not start on a track boundary.
GEOM: da1: partition 1 does not end on a track boundary.
Root mount waiting for: usbus3
ugen3.5: <MICRONET> at usbus3
umass3: <MICRONET FANTOM DRIVE, class 0/0, rev 2.00/2.10, addr 5> on usbus3
umass3:  SCSI over Bulk-Only; quirks = 0x0000
Root mount waiting for: usbus3
umass3:3:3:-1: Attached to scbus3
da3 at umass-sim3 bus 3 target 0 lun 0
da3: <MICRONET FANTOM DRIVE 2.10> Fixed Direct Access SCSI-4 device
da3: 40.000MB/s transfers
da3: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
ugen3.6: <vendor 0x13d1> at usbus3
ugen2.2: <Cesko> at usbus2

You’ll notice from my previous post that the drive that used to be da1 is now da2 and the drive that used to be da2 is now da3. zpool status shows:

> zpool status
pool: tank
state: UNAVAIL
scrub: none requested
config:

NAME        STATE     READ WRITE CKSUM
tank        UNAVAIL      0     0     0  insufficient replicas
raidz1    UNAVAIL      0     0     0  corrupted data
da0s1   ONLINE       0     0     0
da1s1   ONLINE       0     0     0
da2s1   ONLINE       0     0     0

I don’t know for sure, but I think glabel would have solved all this USB port-numbering nonsense. The newly inserted USB key is now drive da1, bumping all the other drives up by one number. Anyway, at this point, it’s fairly simple to play musical chairs (USB ports) with the last three drives. After doing so (and a fair amount of memory game with USB ports/cables), I get the following dmesg:

usbus0: 12Mbps Full Speed USB v1.0
usbus1: 12Mbps Full Speed USB v1.0
usbus2: 12Mbps Full Speed USB v1.0
usbus3: 480Mbps High Speed USB v2.0
ad0: 152627MB <Hitachi HTE543216L9A300 FB2OC45C> at ata0-master UDMA33
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
ugen1.1: <Intel> at usbus1
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
ugen2.1: <Intel> at usbus2
uhub2: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
ugen3.1: <Intel> at usbus3
uhub3: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3
Root mount waiting for: usbus3 usbus2 usbus1 usbus0
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub2: 2 ports with 2 removable, self powered
Root mount waiting for: usbus3
Root mount waiting for: usbus3
Root mount waiting for: usbus3
uhub3: 8 ports with 8 removable, self powered
ugen3.2: <MICRONET> at usbus3
umass0: <MICRONET FANTOM DRIVE, class 0/0, rev 2.00/2.10, addr 2> on usbus3
umass0:  SCSI over Bulk-Only; quirks = 0x0000
Root mount waiting for: usbus3
umass0:0:0:-1: Attached to scbus0
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <MICRONET FANTOM DRIVE 2.10> Fixed Direct Access SCSI-4 device
da0: 40.000MB/s transfers
da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C)
usbd_set_config_index:523: could not read device status: USB_ERR_SHORT_XFER
ugen3.3: <Simpletech> at usbus3
umass1: <MSC Bulk-Only Transfer> on usbus3
umass1:  SCSI over Bulk-Only; quirks = 0x0000
umass1:1:1:-1: Attached to scbus1
da1 at umass-sim1 bus 1 target 0 lun 0
da1: <Hitachi HDS721010CLA332 > Fixed Direct Access SCSI-2 device
da1: 40.000MB/s transfers
da1: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
ugen3.4: <MICRONET> at usbus3
umass2: <MICRONET FANTOM DRIVE, class 0/0, rev 2.00/2.10, addr 4> on usbus3
umass2:  SCSI over Bulk-Only; quirks = 0x0000
umass2:2:2:-1: Attached to scbus2
da2 at umass-sim2 bus 2 target 0 lun 0
da2: <MICRONET FANTOM DRIVE 2.10> Fixed Direct Access SCSI-4 device
da2: 40.000MB/s transfers
da2: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
ugen3.5: <SMI Corporation> at usbus3
umass3: <SMI Corporation USB, class 0/0, rev 2.00/0.00, addr 5> on usbus3
umass3:  SCSI over Bulk-Only; quirks = 0x0000
umass3:3:3:-1: Attached to scbus3
da3 at umass-sim3 bus 3 target 0 lun 0
da3: <ST 16GB 0000> Removable Direct Access SCSI-0 device
da3: 40.000MB/s transfers
da3: 15296MB (31326208 512 byte sectors: 255H 63S/T 1949C)
ugen3.6: <vendor 0x13d1> at usbus3
ugen2.2: <Cesko> at usbus2
GEOM: da3: partition 1 does not start on a track boundary.
GEOM: da3: partition 1 does not end on a track boundary.
Trying to mount root from ufs:/dev/ad0s1a
WARNING: / was not properly dismounted
ZFS filesystem version 13
ZFS storage pool version 13

Just to be safe, I then did a zpool scrub tank. Okay, I got impatient and cancelled it after about 25% completion.

After that, I used glabel to give the USB flash drive a persistent label (might as well start now): sudo glabel label -v pico_usb /dev/da3. Then, I added the device as a cache:

sudo zpool add tank cache /dev/label/pico_usb

I’m running a scrub now. Next, up, I’ll use glabel on the other drives to have them have persistent labels (irrespective of which port I plug them into).

Be the first to like.

Post a Comment

Your email is never published nor shared. Required fields are marked *
*
*