Skip to content

Disk space calculation for streaming my music

29-Aug-10

I’ve been wondering how much disk space I would need if I wanted to run an ampache server on my web host. I get 10 GB of space on my web host. Is it enough to house copies of my music? Even if I down-graded them to 64 kbs AAC+?

Last night, I went through all my music and found duplicates files. I’m now down to (according to dirstat) about:

FLAC: 25.7 GB
MP3: 27.4 GB
M4A: 7.8 GB

The FLAC is around 700 kbps (CD is 1411 kbps and I’m assuming 50% compression). I’m estimating the MP3′s (conservatively) to be 200 kbps. The M4A are probably around 200 kbps as well.

So, when compressed, this would end up being around:

FLAC: 2.4 GB
MP3: 8.8 GB
M4A: 2.5 GB

Total: 13.7 GB

I generally can’t notice the difference at 64 kbps AAC+. However, this is a bit past what I have available right now. Also, I’d be paying for high bandwidth yet using low bandwidth to stream. I’ll think about it.

Setting up public (anonymous) shares on Samba

23-Jul-10

It’s documented here: http://micheljansen.org/blog/entry/182. However, I only found this post after learning each lesson individually.

I wanted to set up a \\server\Public share that anyone on the network could connect to. Certain users can write, but everyone can read.

I tried enabling guest ok in /usr/local/etc/smb.conf (and changed the file/directory masks, so anyone can read by default):

[Public]
  comment = Public (user-wide) directories
  browseable = yes
  writeable = no
  path = /tank/Users/Public
  guest ok = yes
  write list = Me, Wife
  create mask = 0775
  directory mask = 0775

That didn’t work. Then, I read that you can specify which Unix account guest should map to. However, by default it is nobody, which should work.

I finally found the key: A Samba directive called map to guest which tells Samba that any user that doesn’t pass the login should be mapped to the guest account. I placed this in my [Public] share definition. Samba complained. I then placed it in the [global] section, and it worked!

Limit number of instances of a specific process and user

22-Jul-10

I’m using this to limit the number of processes that Ampache spawns. Sometimes, it goes hay-wire.

#!/bin/sh

cmd="$1"
lim="$2"
user="$3" || "www"
numprocs=`ps -U www | grep "$cmd" | wc -l`
echo "Found $numprocs comparing to $lim"
if [ "$numprocs" -gt "$lim" ]; then
        echo "found $numprocs processes"
        killall aacplusenc
fi

Using git with large files

20-Jul-10

I’m using git to keep snapshots of large files. These are outputs of long runs of tools, that would take a long time to re-construct. Using git has the benefit of true version control on the input design files, and an archive of the large, time-consuming outputs.

The idea here, however, is to save time, not disk space. I’ve found that when I check in, git runs for a long time–likely trying to compress the data. It’s unlikely that the large files can be compressed very much (the input-to-output process is a bit chaotic). So, I want it to try less with finding deltas, etc.

The two options that do that are:

git config --add pack.window = 3  # default is 10

and

git config --add pack.depth = 3   # default is 50

The first says not to look at more than 3 “adjacent” (close in type, size, and name) objects for candidates of a delta. The second option says not to look at more than 3 previous versions for a delta. Basically, if you do a lot of branching and merging, the second one is of advantage. However, in my case, the development is pretty linear, so the best delta for any object is the one I checked in previously (not 50 back).

Setting up bags, recipes, and users in TiddlyWeb(Wiki)

15-Jul-10

Up to now, I’ve been using TiddlyWeb as a server-hosted version of TiddlyWiki. This is a fraction of what’s possible with TiddlyWeb. One can create bags which are self-contained stores of tiddlers. One can combine these bags using recipes. Different users can have access to different bags and to different recipes.
By default, TiddlyWebWiki comes with a recipe aptly named default which is owned by a user named administrator. This recipe pulls in tiddlers from two bags: system and common.
Basically, system has the innards that make TiddlyWiki work over TiddlyWeb (the combination of which is called TiddlyWebWiki). common contains stuff that’s supposed to be common between all users.
To customize permissions, one needs to do the following:

  1. Create users
  2. Set up a bag other than common to house per-user (or group) tiddlers
  3. Set up a recipe that exposes the bag to users (or groups thereof)

Setting up a new user is performed by running

twanager adduser UserName UserssAwe$om3P@assw0rd USERROLE WORKROLE BLAHBLAROLE

A role is essentially a user group that one can use to set group permissions on bags/recipes. To set up a new bag, etc. Here’s what you do:

(tiddlyweb)[username@host main_inst]$ twanager bag personal

Now, entere the following and terminate it with CTRL-D (^D):

{"policy": {"read": ["UserName", "R:ADMIN"], "create": ["UserName", "R:ADMIN"], "manage": ["R:ADMIN"], "accept": ["UserName", "R:ADMIN"], "write": ["UserName", "R:ADMIN"], "owner": "UserName", "delete": ["UserName", "R:ADMIN"]}, "desc": "UserName's personal bag"}
^D

If you make a mistake, you can edit the fields that define this bag manually with

vi store/bags/personal/policy
vi store/bags/personal/description

Now, for the recipe:

twanager recipe personal

And type in:

desc: UserName's personal recipe
policy: {"read": ["R:ADMIN", "UserName"], "create": ["UserName", "R:ADMIN"], "manage": ["R:ADMIN"], "accept": ["UserName", "R:ADMIN"], "write": ["R:ADMIN"], "owner": "UserName", "delete": ["R:ADMIN"]}

/bags/system/tiddlers
/bags/common/tiddlers
/bags/personal/tiddlers
^D

Since /bags/personal/tiddlers is specified last in the list, that’s the default bag that UserName‘s tiddlers will go.

You can edit this recipe definition after the fact with:

vi store/recipes/personal

I haven’t tried this, but another option is to use a template recipe such as:

desc: Any user's personal recipe
policy: {"read": ["R:ADMIN", "R:USER"], "create": ["R:USER", "R:ADMIN"], "manage": ["R:ADMIN"], "accept": ["R:USER", "R:ADMIN"], "write": ["R:ADMIN"], "owner": "", "delete": ["R:ADMIN"]}

/bags/system/tiddlers
/bags/common/tiddlers
/bags/{{ user }}/tiddlers
^D

where {{ user }} will get replaced with the login of the current user. I’m not sure how the bag for each user gets created, though. Also, in the above, I specified the role USER (specified with "R:USER"), which is something you’d set up when you add a user.
Finally, to access the newly-created TiddlyWiki described by the above storage scheme (recipe), one would visit http://TiddlyWikiURI/recipes/personal/tiddlers.wiki.

Using Linux command lockfile in bash scripts

14-Jul-10

I needed to create a script that runs a regression test. I intended to schedule this with a cron job. However, I found that sometimes, my cron jobs would collide with each other.

So, I looked to Linux’ lockfile command to create a file lock as a semaphore.

Here’s the incantation:

# if we're already running, exit
if ! lockfile -r 0 regression.lock; then
   echo "regression.sh already running in area $1"
   exit 1
fi
./tools/regression.py
rm -f regression.lock

In this case, lockfile is set with a retry count of 0 -r 0. However, I can chose to make this a retry of 1000 (for example).

One can specify a timeout period (for example 60 seconds) with -s 60. Note, however, that this timeout period starts when the lockfile was last modified/created, and it forces the removal of the lockfile. This helps prevent zombie processes that were killed before they could remove their lockfile.

Getting vim to match begin/end

12-Jul-10

The default vim has a matchpairs option which matches pairs of single bytes.

However, I have a file which has a lot of nested conditions, and the language requires begin/end pairs for multi-line conditionals. I know I have more begin statements than end statements. So, I need help from vim in matching these up.

As it turns out, there’s a matchit macro in $VIMRUNTIME/macros that lets you extend the % command to match more complex expressions (including regular expressions). It also exists here. Install instructions are included on the vim.org page. Basically, you unzip in your ~/.vim directory and then and run a helptags command.



				

Looking into building a dual-core ATOM PC computer

26-Jun-10

I’ve been thinking about building an ATOM-based dual-core PC for a while now.

I have the portwell single-core, but I’ve been thinking it might be fun (sort of) to build an enclosed system with a dual-core ATOM, power supply, and a few USB hard drives (because I already have them, that’s why). I’d take a page (pun intended) from the book of Google and enclose the system using Lego blocks.

When I add it up, it doesn’t seem as convincing:

Atom D510 motherboard: $80
4GB DDR2-800
: $100
Power supply: $40
Lego: $50

Total: $270

There’s no exact equivalent (most computers that I found online come with 2 GB RAM). But, they seem to range in price from $240 to $400.

Looking for a new router

09-Jun-10

Update 2010-06-25

I got the Buffalo WHR-HP-GN. I have it sitting around 10 feet away from my HP PC and it’s connected to my router using 100 Mb/s Ethernet. I just ran a speed test using TTCP, and I get from 2674 KB/s to 3700 KB/s. So, I’m doing about 10x better. Of course, the Linksys is sitting considerably farther away from the HP PC than the Buffalo is. However, 1) the Linksys can’t act as an access point, so it can’t work where the Buffalo is, and 2) I’ve now fixed the problem and don’t care to move routers around.

Original Post

I’ve had it with my Linksys. I’ve run two performance tests using TTCP. I generally run the receiver on my FreeBSD machine (for which a port exists). From my gigabit wired Dell Desktop (circa 2004) I get 30830 KB/s throughput. However, from my late-2009 model HP Desktop connected via 802.11n, I only get 365 KB/s. You read that right. I’ve noticed that the wireless drops down to 13 MBps when the computer is actually doing something; otherwise, it claims a 135 Mbps connection.

This could be the HP’s 802.11n card, but I have a feeling it is the WRT110N router. Besides, I’m pretty pissed that Linksys’ latest firmware has an advertising service (when a site doesn’t exist, it points you at a ad/search page). In the case of the WRT110N, you cannot disable it.

So, I’m looking around for a new router. After a hiatus, Buffalo is back on the market. I’m looking at their plethora of 802.11n offerings. The $90 WZR-HP-G300NH has 64 MB of ram and 32 MB of flash available. It also has a USB port for NAS if I want that. The other option would be to use my old Linksys WRT with Tomato or something (and disable wireless on it) and buy a cheaper Buffalo 802.11n to use as an access point.

The fourth option which I’m seriously contemplating is to run gigabit cable around my house (literally on the outside), through the attic, and to the HP computer. However, when I add up two cables (12.50 each) and a gigabit router ($35), I end up at $60, which isn’t that much away from the wireless router (and it saves me having to tool around the attic, which is definitely worth $30 to me). Plus, there’s also a Wii next to the HP that needs wireless anyway.

I’m looking at the $60 WHR-HP-G300N and the $40 WHR-HP-GN. Both of these are dd-wrt compatible. However, it seems that the G300N doesn’t support WDS. So, it looks like the WHR-HP-GN is the best value. I can spend $50 more to get the WZR-HP-G300NH, but I don’t think the extra RAM, USB, and gigabit Ethernet ports are worth it.

Setting up TiddlyWeb (TiddlyWiki)

24-May-10

I’ve been a fan of TiddlyWiki for a long time now. It’s an HTML file that contains self-editing powers. You download this file, load it in your web browser, and it presents a note-taking application where the notes are stored within that same HTML file (very portable).

I really like this functionality, but I’d like to have it stored somewhere central (a web server for example), so that I don’t have to worry about syncing this file. From looking around, it seemed like TiddlyWeb was a good (albeit possibly over-kill) solution. It works as a server-side TiddlyWiki implementation, but it’s more general: it presents an API that can manage general blobs of information (tiddlers). I got this set up.

More…