|Todd on Installing Debian to disk on a…|
|Jorge Ferreira on Installing Debian to disk on a…|
|nigelr on Installing Debian to disk on a…|
|Guillermo on Installing Debian to disk on a…|
|arvati on Installing Debian to disk on a…|
Above average nerdiness
I get my internet through a TP-Link TD-8817 and I’ve noticed the sync rate vary over time just by checking the web interface from time to time . So that I can track this and perhaps correlate it with something, I’ve written a script in perl with the Expect module, the appropriate templates to graph it in cacti, and put it up on github. Here’s the actual session transcript showing how to get the data:
Trying modem... Connected to modem Escape character is '^]'. Password: ********** Copyright (c) 2001 - 2012 TP-LINK TECHNOLOGIES CO., LTD TP-LINK> wan adsl c near-end interleaved channel bit rate: 10313 kbps near-end fast channel bit rate: 0 kbps far-end interleaved channel bit rate: 1293 kbps far-end fast channel bit rate: 0 kbps TP-LINK> exit Connection closed by foreign host.
Here’s an example graph from my own connection:
To keep track of the health of the disks in my ix2-200s, I setup SMART monitoring via SNMP using a lot of information from this post by Glen Pitt-Pladdy. However, I made a few changes.
Firstly, I use the output of smartd stored in /var/lib/smartmontools/ rather than the cronjob he uses. The format is quite different, and there’s a few extra bits of logic required to get the right values. Secondly, I tried rewriting it in python as an excuse to practice some python. Unfortunately, python takes much longer to start up than perl does, which means SNMP performance is noticeably affected by using the python version. Thirdly, since the state files don’t contain the device name, I switch to using the drive serial numbers as the index for the SNMP data. This might cause problems when it comes to swapping disks, but at least you can be sure you’re monitoring the right disk.
Lastly, and this is more of a bug than a feature, the format of the smartd state files is such that some raw fields are stored as 48 bit numbers, which makes the bitwise operations I use to extract the raw temperature fail in some situations on 32 bit perl. The proper solution would be to use Math::Int64, but this slows things down and it’s not a problem for my particular disks.
I’ve put it all up on my github for those that are interested.
I’ve previously struggled to find a reliable way of building packages on my ubuntu laptop for my ix2-200 running squeeze. I’ve tried scratchbox/scratchbox2 in various configurations, and although it will work for some packages, not all packages work all the time, and I usually end up fiddling with scratchbox 2 settings/scripts/modes to make it work or using a native qemu session. The problems seem to be related to running scratchbox 2 directly on Ubuntu and thus which native tools that scratchbox 2 tries to use.
Recently my wife got a MacBook Air and so she needs a TimeMachine compatible system to back it up to if she wants to do so over the network. This means I needed to get the netatalk package working on one of my ix2-200s, but because the latest version of MacOS X requires AFP 3.3 support for TimeMachine, I needed 2.2.x rather than the 2.1.x that is currently available in squeeze. At the same time, I was also having a play with LXC that comes with Ubuntu 12.04 and lets you set up lightweight Debian and Ubuntu containers for process isolation etc. so I decided to see if I could get a reliable cross-compile environment going for this. I’m happy with the results and l like how I there is no dependency on packages installed in the underlying OS, although there are probably better ways to do it using tools directories and one of the other scratchbox modes.
The first step is to install LXC and create a Debian Squeeze container:
apt-get install lxc lxc-create -t debian -n build lxc-start -n build /sbin/init -d lxc-console --name build
Once logged into the console (using the root account and password that it tells you in the output to lxc-create), you can then install all the required packages to run scratchbox. I use scratchbox2 and qemu from testing as well as the emdebian toolchain from its own repo, as the stable stuff is just a bit old, so I need to create the right APT preferences.
To enable the testing and emdebian repos, put the following in /etc/apt/preferences.d/10testing. I used cat to do this, because vi wasn’t installed:
Package: * Pin: release a=stable Pin-Priority: 900 Package: * Pin: release o=Debian Pin-Priority: -10
Put the following in /etc/apt/apt.conf.d/10testing:
Edit your “/etc/apt/sources.list” and add this
deb http://cdn.debian.net/debian testing main deb-src http://cdn.debian.net/debian/ testing main deb http://security.debian.org/ testing/updates main deb-src http://security.debian.org/ testing/updates main deb http://www.emdebian.org/debian/ squeeze main
Install the emdebian keychain to avoid unauthorised packages:
apt-get update apt-get install emdebian-archive-keyring apt-get update
Then install the packages needed for cross compiling, including scratchbox2 and qemu-user which we will reinstall later:
apt-get install gcc-4.4-arm-linux-gnueabi build-essential cdbs hardening-includes libcrack2 apt-get install scratchbox2 qemu-user libavahi-client-dev gcc-multilib autoconf realpath debootstrap sudo
Install qemu-user and scratchbox2 from testing:
apt-get -t testing install qemu-user scratchbox2
Install the build dependencies for netatalk, otherwise sb2 in simple mode won’t find all the files required to build netatalk:
apt-get build-dep netatalk
At this point you want to create a normal user for youself with access to sudo to continue the rest of the setup:
adduser <username> adduser <username> sudo
Once you’ve logged in as your new user, initialise the sb2 environment in their home directory.
mkdir debian && cd debian sb2-init -n -c /usr/bin/qemu-arm debian arm-linux-gnueabi-gcc fakeroot /usr/sbin/debootstrap --variant=scratchbox --foreign --arch armel squeeze ~/debian http://cdn.debian.net/debian/ sb2 -eR debootstrap/debootstrap --second-stage
Create the new sources.list in ~/debian/etc/apt/sources.list:
deb http://mirror.internode.on.net/pub/debian/ squeeze main deb-src http://mirror.internode.on.net/pub/debian/ squeeze main deb http://security.debian.org/ squeeze/updates main deb-src http://security.debian.org/ squeeze/updates main
Update the apt database and install the dependencies for building netatalk:
sb2 -eR apt-get update sb2 -eR apt-get install libavahi-client-dev # this is missing from the dependencies in the testing netatalk source package sb2 -eR apt-get build-dep netatalk
You will get an error because dbus fails to install, but you can just go ahead and remove it:
sb2 -eR apt-get remove dbus
Now retrieve the source for netatalk in another directory:
cd ~ apt-get source -t testing netatalk cd netatalk-2.2.2 sb2 dpkg-buildpackage -rfakeroot -b -d -uc -us
And that’s it! There are plenty of other sites out there on how to configure netatalk for TimeMachine, so I’ll leave that as an exercise for the reader.
This is a kernel patch based on the kernel sources release by Iomega for the iConnect and some guess work as to the GPIO pins to use. It has the following features:
# echo 1 > /sys/class/leds/backup_led/brightness
# sensors lm63-i2c-0-4c Adapter: mv64xxx_i2c adapter fan1: 0 RPM (min = 0 RPM) temp1: +57.0°C (high = +70.0°C) temp2: +56.2°C (low = +0.0°C, high = +70.0°C) (crit = +85.0°C, hyst = +75.0°C)
I’ve been doing a few alterations to the kernel to support the ix2-200 LEDs, buttons, i2c sensor etc. and so I needed a fast and reliable way of compiling the kernel. I can do this directly under Ubuntu 11.10 using the arm cross compiler:
apt-get install gcc-4.6-arm-linux-gnueabi
Once that’s installed, I can just use the following environment settings to build as per the usual kernel build process:
export CFLAGS="-march=armv5te -mfloat-abi=soft -marm" export CROSS_COMPILE="arm-linux-gnueabi-" export ARCH="arm"
I use git to track the latest kernels, and I suggest you do too since it makes keeping up to date quite easy. This means that the first step is to retrieve the kernel sources like so:
git pull git.kernel.org:/pub/scm/linux/kernel/git/stable/linux-stable.git
Once this is complete, just checkout the version you want to use. I’d recommend the 3.0.x series, since that works well for me under Debian 6.0. The latest as of this writing is 3.0.26 and as of 3.0.13, the issues with mv_cesa have been fixed.
cd linux-stable git checkout v3.0.26
At this point you can apply patches and copy a previously prepared config, such as from my earlier post
patch -p1 < 0001-Modified-rd88f6281-setup-to-support-iomega-ix2-200.patch cp ~/path/to/config .config
I use a standard set of environment variables for when I’m compiling a kernel:
export LOCALVERSION="" # don't append a special version string the kernel (but you can if you like) export INSTALL_MOD_PATH=../modules # install the kernel modules in a special directory outside of the kernel structure
The next step is to setup the configuration by running make oldconfig.
If you’re using the 3.0.x series and the configuration as above you probably won’t be asked any questions, but you’ll need to figure out the answer yourself if you do. The next step is to actually compile the kernel:
make -j8 uImage make -j8 modules make modules_install
Once these are done, I tar up the modules directory and then copy all the required files to the target.
tar -czf ../modules.tar.gz ../modules scp ../modules.tar.gz <ix2-200>: scp System.map <ix2-200>:System.map-3.0.26 scp .config <ix2-200>:config-3.0.26 scp arch/arm/boot/uImage <ix2-200>:uImage-3.0.26
Once that’s done, I ssh to the ix2-200 and copy the files to the right place (as root):
cp /home/user/*-3.0.26 /boot tar xzf modules.tar.gz cp -r modules/lib /
I can now create the initial RAM disk and convert it to uboot format:
update-initramfs -v -c -k 3.0.26 mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs -d initrd.img-3.0.26 uInitrd-3.0.26
Now, if if I’m feeling brave, I will just write it to flash and reboot but I would suggest having a known good uImage/uInitrd sitting on a USB drive somewhere in case it goes wrong:
flash_eraseall /dev/mtd0 flash_eraseall /dev/mtd1 nandwrite -p /dev/mtd0 /boot/uImage-3.0.26 nandwrite -p /dev/mtd1 /boot/uInitrd-3.0.26 reboot
Warning: This completely removes the existing filesystems from disk and kernel images from the flash that ship with the ix2-200 and leaves you with a basic Debian install. Only do this if you understand what this means and have nothing important stored on the device.
Most of the information in this post is based on Diogo Gomes’ work on the iConnect.
The first step in installing Debian that I did was to reconfigure the uBoot environment to fix a number of issues. This is done from the uBoot prompt, which is only accessible via a serial interface.
Update the MAC address of the disconnected ethernet port on the SoC to an address from an unused range so that udev doesn’t get confused:
setenv ethaddr AA:00:00:00:00:01
Enable booting of a standard linux kernel
setenv mainlineLinux yes setenv arcNumber 1682
Save and reset
I used the published debian installer images from here copied to a ext2 formatted usb stick. I connected the usb stick to the front port of the ix2-200.
You should also have the ix2-200 connected to the internet somehow. A simple way of doing this for me was to connect to the Ethernet port of my laptop running Ubuntu and bridge it with the virtual bridge I use for running VirtualBox VMs and provides DHCP and NAT. Internet connectivity can than be used through another connection e.g. a home wireless connection. You can do this by running the following command on the laptop:
sudo brctl addif virbr0 eth0
From uBoot on the ix2-200, start the usb subsystem and load the netboot installer kernel and initial RAM disk into memory:
usb start ext2load usb 0:1 0x01100000 /uInitrd ext2load usb 0:1 0x00800000 /uImage
Set the boot arguments for the installer:
setenv bootargs console=ttyS0,115200n8 base-installer/initramfs-tools/driver-policy=most
Boot the installer:
bootm 0x00800000 0x01100000
You can now go through the normal debian installation almost to the end. A few tips to get you through:
At the last step, the installer will tell you that it can’t make the system bootable. To get around this, we need to install the uboot utilities and save the uboot images to the usb stick.
First select the ‘Execute a shell’ from the Debian installer menu, and then chroot to your fresh Debian install:
Install the utilities:
apt-get install mtd-utils uboot-mkimage uboot-envtools
Change to the boot directory and convert kernel and initrd files to uboot format:
cd /boot mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs -d initrd.img-2.6.32-5-kirkwood uInitrd mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-2.6.32-5 -d vmlinuz-2.6.32-5-kirkwood uImage
Mount your usb memory stick and copy the new files to the memory stick:
mount /dev/sdXX /mnt cp uImage /mnt/uImage-new cp uInitrd /mnt/uInitrd-new
Exit the shell and reboot into uboot by selecting the “Finish the installation” option. At this point we should be able to now boot from disk using the uInitrd and uImage files on usb. Note that I also setup the mtd in the bootargs_console setting so we can write the kernel and initrd files to flash once we know it’s working. Replace the root value with what ever you setup during the install:
usb start setenv bootargs_console 'console=ttyS0,115200 mtdparts=orion_nand:0x300000@0x100000(uImage),0x1000000@0x540000(uInitrd) root=/dev/mapper/' setenv bootargs $(bootargs_console) ext2load usb 0:1 0x00800000 /uImage-new ext2load usb 0:1 0x01100000 /uInitrd-new bootm 0x00800000 0x01100000
Congratulations! You should now be in your debian install.
Login and become root, and execute the following command to confirm that the nand flash has been configured correctly:
# cat /proc/mtd dev: size erasesize name mtd0: 00300000 00004000 "uImage" mtd1: 01000000 00004000 "uInitrd"
We can now erase and flash the uImage and uInitrd files:
# flash_eraseall /dev/mtd0 Erasing 16 Kibyte @ 300000 -- 100 % complete. # flash_eraseall /dev/mtd1 Erasing 16 Kibyte @ 1000000 -- 100 % complete. # nandwrite -p /dev/mtd0 /boot/uImage ... # nandwrite -p /dev/mtd1 /boot/uInitrd ...
Reboot to get back the uBoot prompt to setup loading the kernel and initrd from flash. Replace the root value with what ever you setup during the install:
setenv mtdids 'nand0=orion_nand' setenv mtdparts 'mtdparts=orion_nand:0x100000@0x000000(uboot)ro,0x20000@0xA0000(uboot_env),0x300000@0x100000(uImage),0x1000000@0x540000(uInitrd)' setenv bootargs_console 'console=ttyS0,115200 mtdparts=orion_nand:0x300000@0x100000(uImage),0x1000000@0x540000(uInitrd) root=/dev/mapper/red_vg0-root' setenv bootcmd 'setenv bootargs $(bootargs_console); nand read 0x800000 uImage; nand read 0x1100000 uInitrd; bootm 0x00800000 0x01100000' saveenv reset
You should now be booting Debian from flash and disk.
A few months back I bought a 2TB iomega ix2-200 ostensibly to use for backups. I chose this particular model because it was cheap – only AU$289 from a store here in Melbourne. Many other similar models cost more than that and they don’t even include the disks. I did actually use it for backups for a while before getting curious and finding out how to get a shell on the underlying OS. This turned out to be simple – username root, password soho<whatever you set for admin password>. From here it was easy enough to find that it was running a modified Debian (EMC Lifeline 2.1) under the hood.
Before I could let it just get back to the the lowly task of backing up all my data, I next did the obvious thing and took it to bits to see what was inside. I already new it was running a Marvell Kirkwood 88F6821 SoC and that it had 256MB of RAM. I discovered that it also had a 32MB NAND flash and an RS-232 header. I decided that it would be cool to try and get it to run something like FreeNAS so that I could ZFS, so I soldered up some sockets to a 3.5mm stereo jack and used my TTL-232R-3V3-AJ USB serial adaptor and using minicom, I could then get access to the uBoot boot loader.
I found the pin outs at NAS-Central.
I tried working with FreeBSD for a while, but with only 256MB of RAM system performance with ZFS would have been terrible so abandoned that idea and decided to give btrfs a go under Linux. Debian for ARM is easy to get a hold of and I ended up following the instructions by Diogo Gomez for installing Debian on the iomega iConnect. This got me to the stage of booting Linux from a USB stick. Next, in part one I’ll cover getting linux installed to the hard disks.
Yes, I begin this blog the way I mean to continue: with a self-troll.