Skip to content

Replaced my 250GB hard drive with a 1TB hard drive

25-Feb-10

In a prior incarnation of my “RAID” system, I was using 1 partition on one drive plus 4 partitions on another drive to set up a RAID-Z2 system.

My thinking was that splitting up by partition at least protects me against sector failure (if not whole drive failure). The 4 partitions were 250 GB each on a 1TB Hitachi/SimpleTech drive.

I should’ve anticipated it, but don’t you know, the 1 TB drive failed–after about 1 week. So, my data would’ve been lost. (Except I kept a copy on another computer; I’m not crazy.)

I knew I’d need at least 3 hard drives to prevent this from happening again. So, I sent the Hitachi drive for warranty and ordered a Fantom 1 TB hard drive. (BTW: I really like the Fantom; it looks slick, is constructed well, and runs rather cool.)

In the meantime, I spotted a 500 GB Fantom for a good price (not too enthused about the mail-in-rebates, though), so I got that as well. Once I had the two Fantoms in hand, I could set up a RAIDZ (2 data + 1 parity) configuration with what I already had: a 250 GB Cavalry hard drive (from before), a 500 GB Fantom, and a 1 TB Fantom. Setting it up this way meant that I’d only be making use of 750 GB of space (out of the 1.75 TB that all the drives have together). 250 GB of that 750 is used for parity, so I really only get 500GB of usable space. Such is the price for reliability.

A couple days ago, the replacement 1 TB Hitachi arrived. So, I decided to take out the 250 GB hard drive and replace that slice with the 1 TB Hitachi. I did a:
sudo zpool offline tank da1s1
I then powered down and unplugged the 250GB Cavalry drive. Seeing as how FreeBSD can be quite flaky in disk numbering, I made sure to keep track of which USB port it was plugged into.

Incidently, I had a Kill-A-Watt on the whole setup. Before unplugging the Cavalry, the Kill-A-Watt estimated $36.33 per year and 33 Watts usage. After unplugging, it said 22 Watts usage. Wow! That explains why the Cavalry runs burning hot to the touch (makes a good foot warmer in winter).

After plugging in the Hitachi drive (I inadvertently powered down the machine in the process), I did the following to re-silver:
sudo zpool replace tank da1s1 da1s1

Right now, it’s sitting at the following:


> zpool status
pool: tank
state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress for 0h12m, 11.90% done, 1h35m to go
config:

NAME             STATE     READ WRITE CKSUM
tank             DEGRADED     0     0     0
raidz1         DEGRADED     0     0     0
da0s1        ONLINE       0     0     0  4.89M resilvered
replacing    DEGRADED     0     0     0
da1s1/old  OFFLINE      0 7.68K     0
da1s1      ONLINE       0     0     0  7.23G resilvered
da2s1        ONLINE       0     0     0  4.88M resilvered

errors: No known data errors

Pretty slick! I’ll now have 1 TB of storage (out of the 2.5 TB total hard drive space).

I know it seems crazy, but I think I’ll continue to keep the slices incongruent. For example, when I need more space, I’ll replace the 500 GB drive with a 2 TB drive. I’ll immediately have 2 TB available (out of a total of 4 TB hard drive space, 3 TB are being used). One of the mistakes I made presently is that I have two drives of the same size. So, at some point, I’m going to replace one of them but not get any additional space from it. More likely, 2 TB of space will suffice for many years to come.

By the way, the Kill-A-Watt now shows 30 Watts usage. Of course, that’s while the drives are active (re-silvering). Hopefully, they’ll lower their power when the re-silvering is finished.

Be the first to like.

Liking TortoiseGit better than git-cheetah

20-Feb-10

I’ve installed both, and I can’t figure out what git-cheetah does other than start gitk (which comes with msysgit anyway).TortoiseGIT installation screenshot

However, TortoiseGit (like TortoiseHG & TortoiseSVN) are beautiful. In addition, it supports both 32-bit and 64-bit versions of Windows.

Be the first to like.

FreeBSD and ZFS drive ordering

14-Feb-10

So, as it turns out, ZFS relies on the FreeBSD slice names to determine what goes where. This can be a problem, since the slice drive names can move around–if, for example, you switch what USB ports your drives plug into. Which is what I just did.
I have no idea what ordering they were before, but I figured out how to find out what they are now. I did a:

dmesg | egrep “da.:”

Which resulted in:

da0: <MICRONET FANTOM DRIVE 2.10> Fixed Direct Access SCSI-4 device
da0: 40.000MB/s transfers
da0: 476940MB (976773168 512 byte sectors: 255H 63S/T 60801C)
da1: <Maxtor 6 Y250M0 > Fixed Direct Access SCSI-2 device
da1: 40.000MB/s transfers
da1: 239372MB (490234752 512 byte sectors: 255H 63S/T 30515C)
da2: <MICRONET FANTOM DRIVE 2.10> Fixed Direct Access SCSI-4 device
da2: 40.000MB/s transfers
da2: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
Be the first to like.

Wow! Sprint 3G is pretty good on the uplink

11-Feb-10

Whereas my 4G connection (albeit from my basement) is ~ 3Mbps download and 100 kbps upload, the 3G connection is ~ 1Mbps download but a whopping 410 kbps upload. Latency is also lower (78 ms versus 106 ms).

Be the first to like.

Clear conenct manager breaks Sprint 4G hack

11-Feb-10

I know it was inevitable. I had been using Clear’s Connection manager, since it’s been far superior to Sprint’s. However, today, the connection manager upgraded itself. It no longer recognizes the Sprint network. (Really, Sprint & Clear are the same network. What I mean is that Clear connection manager no longer authenticates using my Sprint account.) This was inevitable, since I was using a hacked version of Clear Connection Manager.

Firefox download image of Sprint SmartView
At least now, Sprint supports Windows 7. Here’s hoping.

Be the first to like.

Sprint 4G latency

11-Feb-10

I’m currently using a wireless (fixed, not mobile) ISP. The latency I get using the Motorola Canopy equipment is 22 ms. My download speeds are 1818 kbps.

With Sprint’s 4G WiMax, the latency is 106 ms. The download speed is 2.9 Mbps.

What has me concerned is that I use this connection to VNC into a computer at work. Sure, the upload/download speeds are great, but I wonder if the latency is hurting me. I also wonder if the latency improves when the signal conditions get better. (On the Clear connection utility, I have 6 green dots out of 10.)

Be the first to like.

Locking down a publicly-exposed ssh server

26-Jan-10

I expose an ssh server to the internet. This allows me to sync up git or hg repositories under a secure connection (without requiring SSL)–but more importantly, keeping the data stored in those repositories under my control.
Since I’m fairly paranoid about security, I chose a random port number (my router routes the random public port to the usual internal port: 22). In addition, I disable password login, which means that I can only log-in using an ssh key. More importantly, other people can’t login using passwords–which means that there’s not possibility of a dictionary (nor simple brute-force) attack.
Here’s the sshd_config file for FreeBSD that does this:

# $OpenBSD: sshd_config,v 1.74 2006/07/19 13:07:10 dtucker Exp $
# $FreeBSD: src/crypto/openssh/sshd_config,v 1.42.2.4 2006/11/11 00:51:28 des Exp $

# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options change a
# default value.

# Note that some of FreeBSD’s defaults differ from OpenBSD’s, and
# FreeBSD has a few additional options.

#VersionAddendum FreeBSD-20061110

#Port 22
#Protocol 2
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

# HostKey for protocol version 1
#HostKey /etc/ssh/ssh_host_key
# HostKeys for protocol version 2
#HostKey /etc/ssh/ssh_host_dsa_key

# Lifetime and size of ephemeral version 1 server key
#KeyRegenerationInterval 1h
#ServerKeyBits 768

# Logging
# obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin no
#StrictModes yes
#MaxAuthTries 6

#RSAAuthentication yes
#PubkeyAuthentication yes
#AuthorizedKeysFile .ssh/authorized_keys

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#RhostsRSAAuthentication no
# similar for protocol version 2
#HostbasedAuthentication no
# Change to yes if you don’t trust ~/.ssh/known_hosts for
# RhostsRSAAuthentication and HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don’t read the user’s ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# Change to yes to enable built-in password authentication.
PasswordAuthentication no
#PermitEmptyPasswords no

# Change to no to disable PAM authentication
ChallengeResponseAuthentication no

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

# Set this to ‘no’ to disable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication and
# PasswordAuthentication. Depending on your PAM configuration,
# PAM authentication via ChallengeResponseAuthentication may bypass
# the setting of “PermitRootLogin without-password”.
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and ChallengeResponseAuthentication to ‘no’.
#UsePAM yes

#AllowTcpForwarding yes
#GatewayPorts no
#X11Forwarding yes
X11Forwarding no
#X11DisplayOffset 10
#X11UseLocalhost yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#UsePrivilegeSeparation yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS yes
#PidFile /var/run/sshd.pid
#MaxStartups 10
#PermitTunnel no
PermitTunnel no

# no default banner path
#Banner /some/path

# override default of no subsystems
Subsystem sftp /usr/libexec/sftp-server

# restrict users/IP’s
AllowUsers *@192.168.1.* Poojan@*

# Example of overriding settings on a per-user basis
#Match User anoncvs
# X11Forwarding no
# AllowTcpForwarding no
# ForceCommand cvs server

I’ve disabled all authentication types (including PAM)–at least the ones that aren’t disabled by default. Since public key authentication is on by default, I don’t have to change it.

You’ll notice that I’ve disabled tunneling (to disable people from penetrating past the FreeBSD machine into my home network) and chosen not to enable X forwarding (no need for it in this case).

Be the first to like.

Bash script to search for 3-letter domains

23-Jan-10

Here it is for the .us TLD:

#!/bin/bash

tld=”us”

for k in 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do

whois — “${k}.${tld}” | egrep “Domain Name|Not found”

for l in – 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do

whois — “${k}${l}.${tld}” | egrep “Domain Name|Not found”

for m in 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do

whois — “${k}${l}${m}.${tld}” | egrep “Domain Name|Not found”
done
done
done

And here it is for the .nu TLD:

#!/bin/bash
for k in 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do

whois “${k}.nu” | grep “ASCII”

for l in 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do

whois “${k}${l}.nu” | grep “ASCII”

for m in 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z; do
whois “${k}${l}${m}.nu” | grep “ASCII”
done
done
done

You’ll note that the expression being greped for is different between the two: there’s no universal method of detecting the existence/non-existence of domains that works for all TLD’s.

Be the first to like.

Using self-hosted URL shortener YOURLS

20-Jan-10

Update

This works better:
[cclN_php]define(‘YOURLS_SITE’, ‘http://’ . $_SERVER[‘SERVER_NAME’]);[/cclN_php]
assuming yourls is installed in the root directory of your web site (http://foo.bar.com/ for example). If otherwise (http://foo.bar.com/yourls/) you’d want:
[cclN_php]define(‘YOURLS_SITE’, ‘http://’ . $_SERVER[‘SERVER_NAME’] . ‘/yourls’);[/cclN_php]

Original Post

I’m now using Shorten2Ping to get my post updates from my blogs to Ping.FM (from which it goes to LinkedIn, Identi.ca and army.twit.tv).

While setting this up, I noticed that there’s an option for a self-hosted URL shortener called YOURLS. This thing rocks! It shortens URL’s, collects statistics, and allows custom shortened URL’s.

I wanted to use the same install (database etc) across all my blogs (poojanblog.com, www.circuitdesign.info, tech.poojanblog.com). WebFaction makes doing so very easy: you can tie any subdirectory of a domain to a specific “web app” (UNIX directory). So, for example, both poojanblog.com/u and www.circuitdesign.info/u point to the same directory on my web host.

The only issue with this is that YOURLS returns one and only one domain for the shortened URL, set by a PHP define for YOURLS_SITE in includes/config.php.

Luckily, I found a way around this. I put the following in config.php:

[cclN_php]define(‘YOURLS_SITE’, dirname($_SERVER[‘REQUEST_URI’])); // LOL. Wild guess.[/cclN_php]

This happens to be taken from includes/load-yourls.php as a fall-back if the YOURLS_SITE isn’t defined. Basically, it picks up the domain name from the HTTP header.

Be the first to like.

Setting up automated ZFS snapshots on FreeBSD

20-Jan-10

Using the port sysutils/zfs-snapshot-mgmt.

The make is taking a while, because it had to pull in Ruby. Luckily, my new FreeBSD machine has a whopping 160 GB hard drive. No worries. However, I wonder if I should’ve used the alternative. Just didn’t want to install something that doesn’t register with the FreeBSD packages/ports.

Initially left the default /usr/local/etc/zfs-snapshot-mgmt.conf file except edited it to have tank/Users and tank as the filesystems to snapshot. However, the latter was giving a NoMethod Ruby exception so I commented it out. Things are looking good:


NAME USED AVAIL REFER MOUNTPOINT
tank/Users@auto-2010-01-20_07.30 0 – 30.0K –
tank/Users/Wife@auto-2010-01-20_07.30 0 – 30.6K –
tank/Users/Wife/Documents@auto-2010-01-20_07.30 0 – 39.0M –
tank/Users/Wife/Music@auto-2010-01-20_07.30 0 – 1.55G –
tank/Users/Wife/Pictures@auto-2010-01-20_07.30 0 – 3.84G –
tank/Users/Wife/Videos@auto-2010-01-20_07.30 0 – 24.0K –
tank/Users/Poojan@auto-2010-01-20_07.30 0 – 30.6K –
tank/Users/Poojan/Documents@auto-2010-01-20_07.30 1.02M – 2.35G –
tank/Users/Poojan/Music@auto-2010-01-20_07.30 0 – 36.5G –
tank/Users/Poojan/Pictures@auto-2010-01-20_07.30 0 – 24.0K –
tank/Users/Poojan/Videos@auto-2010-01-20_07.30 0 – 310M –
tank/Users/Public@auto-2010-01-20_07.30 0 – 30.6K –
tank/Users/Public/Documents@auto-2010-01-20_07.30 0 – 50.6K –
tank/Users/Public/Music@auto-2010-01-20_07.30 0 – 2.61G –
tank/Users/Public/Pictures@auto-2010-01-20_07.30 0 – 6.57G –
tank/Users/Public/Videos@auto-2010-01-20_07.30 0 – 310M –
Be the first to like.