Above average nerdiness

Category Archives: NAS

SMART monitoring via SNMP

To keep track of the health of the disks in my ix2-200s, I setup SMART monitoring via SNMP using a lot of information from this post by Glen Pitt-Pladdy. However, I made a few changes.

Firstly, I use the output of smartd stored in /var/lib/smartmontools/ rather than the cronjob he uses. The format is quite different, and there’s a few extra bits of logic required to get the right values. Secondly, I tried rewriting it in python as an excuse to practice some python. Unfortunately, python takes much longer to start up than perl does, which means SNMP performance is noticeably affected by using the python version. Thirdly, since the state files don’t contain the device name, I switch to using the drive serial numbers as the index for the SNMP data. This might cause problems when it comes to swapping disks, but at least you can be sure you’re monitoring the right disk.

Lastly, and this is more of a bug than a feature, the format of the smartd state files is such that some raw fields are stored as 48 bit numbers, which makes the bitwise operations I use to extract the raw temperature fail in some situations on 32 bit perl. The proper solution would be to use Math::Int64, but this slows things down and it’s not a problem for my particular disks.

I’ve put it all up on my github for those that are interested.

Cross-compiling Debian armel using LXC on Ubuntu 12.04

I’ve previously struggled to find a reliable way of building packages on my ubuntu laptop for my ix2-200 running squeeze. I’ve tried scratchbox/scratchbox2 in various configurations, and although it will work for some packages, not all packages work all the time, and I usually end up fiddling with scratchbox 2 settings/scripts/modes to make it work or using a native qemu session.  The problems seem to be related to running scratchbox 2 directly on Ubuntu and thus which native tools that scratchbox 2 tries to use.

Recently my wife got a MacBook Air and so she needs a TimeMachine compatible system to back it up to if she wants to do so over the network. This means I needed to get the netatalk package working on one of my ix2-200s, but because the latest version of MacOS X requires AFP 3.3 support for TimeMachine, I needed 2.2.x rather than the 2.1.x that is currently available in squeeze. At the same time, I was also having a play with LXC that comes with Ubuntu 12.04 and lets you set up lightweight Debian and Ubuntu containers for process isolation etc. so I decided to see if I could get a reliable cross-compile environment going for this. I’m happy with the results and l like how I there is no dependency on packages installed in the underlying OS, although there are probably better ways to do it using tools directories and one of the other scratchbox modes.

The first step is to install LXC and create a Debian Squeeze container:

apt-get install lxc
lxc-create -t debian -n build
lxc-start -n build /sbin/init -d
lxc-console --name build

Once logged into the console (using the root account and password that it tells you in the output to lxc-create), you can then install all the required packages to run scratchbox. I use scratchbox2 and qemu from testing as well as the emdebian toolchain from its own repo, as the stable stuff is just a bit old, so I need to create the right APT preferences.

To enable the testing and emdebian repos, put the following in /etc/apt/preferences.d/10testing. I used cat to do this, because vi wasn’t installed:

Package: *
Pin: release a=stable
Pin-Priority: 900

Package: *
Pin: release o=Debian
Pin-Priority: -10

Put the following in /etc/apt/apt.conf.d/10testing:

APT::Default-Release "stable";

Edit your “/etc/apt/sources.list” and add this

deb testing main
deb-src testing main

deb testing/updates main
deb-src testing/updates main

deb squeeze main

Install the emdebian keychain to avoid unauthorised packages:

apt-get update
apt-get install emdebian-archive-keyring
apt-get update

Then install the packages needed for cross compiling, including scratchbox2 and qemu-user which we will reinstall later:

apt-get install gcc-4.4-arm-linux-gnueabi build-essential cdbs hardening-includes libcrack2
apt-get install scratchbox2 qemu-user libavahi-client-dev gcc-multilib autoconf realpath debootstrap sudo

Install qemu-user and scratchbox2 from testing:

apt-get -t testing install qemu-user scratchbox2

Install the build dependencies for netatalk, otherwise sb2 in simple mode won’t find all the files required to build netatalk:

apt-get build-dep netatalk

At this point you want to create a normal user for youself with access to sudo to continue the rest of the setup:

adduser <username>
adduser <username> sudo

Once you’ve logged in as your new user, initialise the sb2 environment in their home directory.

mkdir debian && cd debian
sb2-init -n -c /usr/bin/qemu-arm debian arm-linux-gnueabi-gcc
fakeroot /usr/sbin/debootstrap --variant=scratchbox --foreign --arch armel squeeze ~/debian
sb2 -eR debootstrap/debootstrap --second-stage

Create the new sources.list in ~/debian/etc/apt/sources.list:

deb squeeze main
deb-src squeeze main

deb squeeze/updates main
deb-src squeeze/updates main

Update the apt database and install the dependencies for building netatalk:

sb2 -eR apt-get update
sb2 -eR apt-get install libavahi-client-dev # this is missing from the dependencies in the testing netatalk source package
sb2 -eR apt-get build-dep netatalk

You will get an error because dbus fails to install, but you can just go ahead and remove it:

sb2 -eR apt-get remove dbus

Now retrieve the source for netatalk in another directory:

cd ~
apt-get source -t testing netatalk
cd netatalk-2.2.2
sb2 dpkg-buildpackage -rfakeroot -b -d -uc -us

And that’s it! There are plenty of other sites out there on how to configure netatalk for TimeMachine, so I’ll leave that as an exercise for the reader.

Kernel patch to support LEDs, buttons and sensors on the ix2-200

This is a kernel patch based on the kernel sources release by Iomega for the iConnect and some guess work as to the GPIO pins to use. It has the following features:

  • control of LEDs via leds-gpio
# echo 1 > /sys/class/leds/backup_led/brightness
  • i2c sensors support
# sensors
Adapter: mv64xxx_i2c adapter
fan1:          0 RPM  (min =    0 RPM)
temp1:       +57.0°C  (high = +70.0°C)
temp2:       +56.2°C  (low  =  +0.0°C, high = +70.0°C)
                      (crit = +85.0°C, hyst = +75.0°C)
  • buttons support
  • removing redundant device configuration etc.
One note about the LEDs is that the leds-gpio driver lacks the support to switch them to their full brightness, so they will be a bit dimmer than you might be used to. This can be addressed by twiddling the appropriate GPIO directly, which is documented in the patch.
Update: Alan pointed out that my old patch didn’t apply cleaning on 3.3 or above. I’ve changed my patch to only update the setup file. I’ve also extracted the patch from 3.3 that fixed the GPIO config problem, which you will need if you’re using version 3.2 or older. I’ve put them on google drive, so hopefully they’ll stay available.
Setup file Patch
mpp.h patch
Kernel 3.4.4 Config
Kernel 3.0.18 Config

Compiling kernels for the Marvell Kirkwood SoC under Ubuntu 11.10

I’ve been doing a few alterations to the kernel to support the ix2-200 LEDs, buttons, i2c sensor etc. and so I needed a fast and reliable way of compiling the kernel. I can do this directly under Ubuntu 11.10 using the arm cross compiler:

apt-get install gcc-4.6-arm-linux-gnueabi

Once that’s installed, I can just use the following environment settings to build as per the usual kernel build process:

export CFLAGS="-march=armv5te -mfloat-abi=soft -marm"
export CROSS_COMPILE="arm-linux-gnueabi-"
export ARCH="arm"

I use git to track the latest kernels, and I suggest you do too since it makes keeping up to date quite easy. This means that the first step is to retrieve the kernel sources like so:

git pull

Once this is complete, just checkout the version you want to use. I’d recommend the 3.0.x series, since that works well for me under Debian 6.0. The latest as of this writing is 3.0.26 and as of 3.0.13, the issues with mv_cesa have been fixed.

cd linux-stable
git checkout v3.0.26

At this point you can apply patches and copy a previously prepared config, such as from my earlier post

patch -p1 < 0001-Modified-rd88f6281-setup-to-support-iomega-ix2-200.patch
cp ~/path/to/config .config

I use a standard set of environment variables for when I’m compiling a kernel:

export LOCALVERSION="" # don't append a special version string the kernel (but you can if you like)
export INSTALL_MOD_PATH=../modules # install the kernel modules in a special directory outside of the kernel structure

The next step is to setup the configuration by running make oldconfig.

make oldconfig

If you’re using the 3.0.x series and the configuration as above you probably won’t be asked any questions, but you’ll need to figure out the answer yourself if you do. The next step is to actually compile the kernel:

make -j8 uImage
make -j8 modules
make modules_install

Once these are done, I tar up the modules directory and then copy all the required files to the target.

tar -czf ../modules.tar.gz ../modules
scp ../modules.tar.gz <ix2-200>:
scp <ix2-200>
scp .config <ix2-200>:config-3.0.26
scp arch/arm/boot/uImage <ix2-200>:uImage-3.0.26

Once that’s done, I ssh to the ix2-200 and copy the files to the right place (as root):

cp /home/user/*-3.0.26 /boot
tar xzf modules.tar.gz
cp -r modules/lib /

I can now create the initial RAM disk and convert it to uboot format:

update-initramfs -v -c -k 3.0.26
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs -d initrd.img-3.0.26 uInitrd-3.0.26

Now, if if I’m feeling brave, I will just write it to flash and reboot but I would suggest having a known good uImage/uInitrd sitting on a USB drive somewhere in case it goes wrong:

flash_eraseall /dev/mtd0
flash_eraseall /dev/mtd1
nandwrite -p /dev/mtd0 /boot/uImage-3.0.26
nandwrite -p /dev/mtd1 /boot/uInitrd-3.0.26

Other notes:

  • At some point I’d like to get packaging automated so that I can just install a .deb like other howtos do.
  • I haven’t included the steps to use the cryptodev module, which I use for giving accelerating SSH via a modified openssl.
  • There are some packages in debian-backports that I think are required to make Debian 6.0 fully support modern kernels. I haven’t looked that closely as to what exactly might not work, but I haven’t run into any problems either.

Installing Debian to disk on an ix2-200

Warning: This completely removes the existing filesystems from disk and kernel images from the flash that ship with the ix2-200 and leaves you with a basic Debian install. Only do this if you understand what this means and have nothing important stored on the device.

Most of the information in this post is based on Diogo Gomes’ work on the iConnect.

uBoot Parameters

The first step in installing Debian that I did was to reconfigure the uBoot environment to fix a number of issues. This is done from the uBoot prompt, which is only accessible via a serial interface.

Update the MAC address of the disconnected ethernet port on the SoC to an address from an unused range so that udev doesn’t get confused:

setenv ethaddr AA:00:00:00:00:01

Enable booting of a standard linux kernel

setenv mainlineLinux yes
setenv arcNumber 1682

Save and reset


Booting the Debian installer and completing the install

I used the published debian installer images from here copied to a ext2 formatted usb stick. I connected the usb stick to the front port of the ix2-200.

You should also have the ix2-200 connected to the internet somehow. A simple way of doing this for me was to connect to the Ethernet port of my laptop running Ubuntu and bridge it with the virtual bridge I use for running VirtualBox VMs and provides DHCP and NAT. Internet connectivity can than be used through another connection e.g. a home wireless connection. You can do this by running the following command on the laptop:

sudo brctl addif virbr0 eth0

From uBoot on the ix2-200, start the usb subsystem and load the netboot installer kernel and initial RAM disk into memory:

usb start
ext2load usb 0:1 0x01100000 /uInitrd
ext2load usb 0:1 0x00800000 /uImage

Set the boot arguments for the installer:

setenv bootargs console=ttyS0,115200n8 base-installer/initramfs-tools/driver-policy=most

Boot the installer:

bootm 0x00800000 0x01100000

You can now go through the normal debian installation almost to the end. A few tips to get you through:

  • Select eth1 for the network interface (eth0 doesn’t work).
  • I used a single RAID1 instance across a single partition on each disk with LVM and volumes on top of that.

At the last step, the installer will tell you that it can’t make the system bootable. To get around this, we need to install the uboot utilities and save the uboot images to the usb stick.

First select the ‘Execute a shell’ from the Debian installer menu, and then chroot to your fresh Debian install:

chroot /target

Install the utilities:

apt-get install mtd-utils uboot-mkimage uboot-envtools

Change to the boot directory and convert kernel and initrd files to uboot format:

cd /boot
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs -d initrd.img-2.6.32-5-kirkwood uInitrd
mkimage -A arm -O linux -T kernel  -C none -a 0x00008000 -e 0x00008000 -n Linux-2.6.32-5 -d vmlinuz-2.6.32-5-kirkwood uImage

Mount your usb memory stick and copy the new files to the memory stick:

mount /dev/sdXX /mnt
cp uImage /mnt/uImage-new
cp uInitrd /mnt/uInitrd-new

Exit the shell and reboot into uboot by selecting the “Finish the installation” option. At this point we should be able to now boot from disk using the uInitrd and uImage files on usb. Note that I also setup the mtd in the bootargs_console setting so we can write the kernel and initrd files to flash once we know it’s working. Replace the root value with what ever you setup during the install:

usb start
setenv bootargs_console 'console=ttyS0,115200 mtdparts=orion_nand:0x300000@0x100000(uImage),0x1000000@0x540000(uInitrd) root=/dev/mapper/'
setenv bootargs $(bootargs_console)
ext2load usb 0:1 0x00800000 /uImage-new
ext2load usb 0:1 0x01100000 /uInitrd-new
bootm 0x00800000 0x01100000

Congratulations! You should now be in your debian install.

Configuration to boot from internal flash and disk

Login and become root, and execute the following command to confirm that the nand flash has been configured correctly:

# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00300000 00004000 "uImage"
mtd1: 01000000 00004000 "uInitrd"

We can now erase and flash the uImage and uInitrd files:

# flash_eraseall /dev/mtd0
Erasing 16 Kibyte @ 300000 -- 100 % complete.
# flash_eraseall /dev/mtd1
Erasing 16 Kibyte @ 1000000 -- 100 % complete.
# nandwrite -p /dev/mtd0 /boot/uImage
# nandwrite -p /dev/mtd1 /boot/uInitrd

Reboot to get back the uBoot prompt to setup loading the kernel and initrd from flash. Replace the root value with what ever you setup during the install:

setenv mtdids 'nand0=orion_nand'
setenv mtdparts 'mtdparts=orion_nand:0x100000@0x000000(uboot)ro,0x20000@0xA0000(uboot_env),0x300000@0x100000(uImage),0x1000000@0x540000(uInitrd)'
setenv bootargs_console 'console=ttyS0,115200 mtdparts=orion_nand:0x300000@0x100000(uImage),0x1000000@0x540000(uInitrd) root=/dev/mapper/red_vg0-root'
setenv bootcmd 'setenv bootargs $(bootargs_console); nand read 0x800000 uImage; nand read 0x1100000 uInitrd; bootm 0x00800000 0x01100000'

You should now be booting Debian from flash and disk.

First adventures with an ix2-200 NAS

A few months back I bought a 2TB iomega ix2-200 ostensibly to  use for backups. I chose this particular model because it was cheap – only AU$289 from a store here in Melbourne. Many other similar models cost more than that and they don’t even include the disks. I did actually use it for backups for a while before getting curious and finding out how to get a shell on the underlying OS. This turned out to be simple – username root, password soho<whatever you set for admin password>. From here it was easy enough to find that it was running a modified Debian (EMC Lifeline 2.1) under the hood.

Before I could let it just get back to the the lowly task of backing up all my data, I next did the obvious thing and took it to bits to see what was inside. I already new it was running a Marvell Kirkwood 88F6821 SoC and that it had 256MB of RAM. I discovered that it also had a 32MB NAND flash and an RS-232 header. I decided that it would be cool to try and get it to run something like FreeNAS so that I could ZFS, so I soldered up some sockets to a 3.5mm stereo jack and used my TTL-232R-3V3-AJ USB serial adaptor and using minicom, I could then get access to the uBoot boot loader.

ix2-200 PCB

ix2-200 PCB with serial connector attached

I found the pin outs at NAS-Central.

I tried working with FreeBSD for a while, but with only 256MB of RAM system performance with ZFS would have been terrible so abandoned that idea and decided to give btrfs a go under Linux. Debian for ARM is easy to get a hold of and I ended up following the instructions by Diogo Gomez for installing Debian on the iomega iConnect. This got me to the stage of booting Linux from a USB stick. Next, in part one I’ll cover getting linux installed to the hard disks.