Poojan's Tech Blog

I’ve installed a FreeBSD system using gmirror onto a couple of USB drives. I’ve noticed that there’s a considerable lag when I’m doing certain things (running top for example). I’ve narrowed it down to the /tmp and /var file systems. These two file systems get written to often during normal system operation. I’ve recently aleviated this problem using the md facility–which creates a ramdisk.

Since I’m using USB drives for the root (a d the default /tmp and /var), the write latency is higher than read latency. In addition, gmirror exacerbates this tendency.

Update 2011-03-05 (again)

I now have everything contained within /etc/fstab. /tmp exists in memory (md), /var sits on USB, but a few key subdirectories of [/var] exist in memory as well. My /etc/fstab looks like so:

I’ve commented out all the md-related lines in /etc/rc.conf:

Update 2011-03-05

I’ve since this concluded that most of FreeBSD expects certain things within /var to be persistent across boots (namely /var/db). I’ve since put /var back on my USB drives. However, I’ve followed the advice given here: http://www.cabstand.com/usbflash.html#flash_parts and set noatime on my mounts, so that a read access doesn’t require a write. The system generally seems pretty responsive. My fstab now looks like:

I may go forward and make /var/run and /var/log md files as advised in the above link.

Update 2011-02-14

The below procedure had problems with samba. Unfortunately, samba puts a bunch of stuff in /var/log/samba, /var/run/samba and /var/db/samba. Not sure why you’d want another level of indirection below each of these directories. Unfortunately, this behavior is hard-coded into the samba ports Makefile. To get around it, you can either follow the original instructions using fstab below and modify /usr/local/etc/rc.d/samba with:

[cce_bash]
samba_start_precmd() {
mkdir -p /var/log/samba /var/run/samba /var/run/samba
# XXX: Never delete winbindd_idmap, winbindd_cache and group_mapping
[/cce_bash]

Which ensures that the directories are created before samba is started. Or, you can use the method with rc.conf. Either way, you’re editing two files: A)/usr/local/etc/rc.d/samba and /etc/fstab; or B)/etc/rc.conf and /etc/fstab.

Original Instructions (using fstab)

So, moving the /tmp and /var file systems into RAM makes things go much faster. It’s pretty easy to do so. You only have to edit /etc/fstab.

Here’s my /etc/fstab:

You can see that I labeled two new USB drives (4G each) to contain swap. I called them swap0 and swap1. The memory-based file system will use swap as a backing store (it’ll spill over into the swap space if the system runs low on RAM).

Alternative method using /etc/rc.conf and /etc/fstab

Edit /etc/rc.conf to look like so:

[cce_bash]
mdconfig_md0=”-t swap -s 32m”
mdconfig_md1=”-t swap -s 512m”
mdconfig_md1_cmd=”mkdir -p /var/log/samba /var/db/samba /var/run/samba”
[/cce_bash]

This instructs rc.conf (really mdconfig and mdconfig2 startup scripts) to create a md0 of size 32 MB and md1 of size 512MB. It also runs a command to create the directories samba expects (but does not create).

Unfortunately, you also have to add entries to fstab to mount the md devices as file systems:

Fixing portsnap

One problem with the above is that portsnap tends to dump things to /var/db/portsnap. Since /var is now in a memory file system, it’ll get cleared on each reboot. Instead, I now store portsnap fetches in /usr/local/portsnap. (This is the default if you install portsnap from the ports collection, but the base install of FreeBSD includes a portsnap which stores it in /var/db/portsnap.)

To make this change, I edited /etc/portsnap.conf:

[cc_bash]
# Default directory where compressed snapshots are stored.
# WORKDIR=/var/db/portsnap
WORKDIR=/usr/local/portsnap
[/cc_bash]