So I got around to backing up the boot drive in my server and proceeded to format it so I could play with Ubuntu Server 7.10. I’d been thoroughly impressed by the experience I had under the desktop version, so I expected much the same from the server version too. In the end, it didn’t go quite as smooth as I expected, but to be fair none of the problems were really Ubuntu’s fault. Following is a bit of a long-winded description of what not to do if you plan on running a system with either software RAID, or bridged network cards.
First thing to do after firing through the installer was to obviously set up the RAID arrays again. One of the much-touted features of Linux’s software RAID feature is its controller independence: if a controller dies, you can replace it with anything else and it should still work. I’ve also had arrays survive re-installations of Gentoo. But little did I realise the can of worms that moving distributions would open.
I’ve got two arrays in the system. One RAID 0, for all the less important data. It’s where all the downloads go, and stuff that gets accessed most at LANs gets stored. The other is a smaller RAID 1 array, for photos, documents and other data that needs a little redundancy. After a quick apt-get install mdadm and a subsequent reboot to allow the md drivers to scan for arrays, it found the RAID 0 array and set it up as /dev/md0. Mounted that, and all the data was still there.
The problem was with the RAID 1 array. dmesg revealed that it had found the correct 2 disks in the array, but for some reason I couldn’t mount it. Running fdisk on a software RAID array should show no partitions, because the RAID drivers handle all that - the /dev/mdx device is just formatted with whatever filesystem you choose. That was the case on the RAID 0 array, but the RAID 1 apparently contained a single partition (/dev/md1p1). Furthermore, this partition couldn’t be mounted because the device node /dev/md1p1 didn’t exist, and quite rightly. I started to get just a little worried when I couldn’t even mount a single drive independent of the array (which is possible in theory on a RAID 1 device, since it’s just mirrored).
e2fsck then suggested running a check, which in hindsight is not a good idea when it claims every block on the disk is corrupted and starts deleting inodes like they’re going out of fashion. I cancelled that pretty damn quick. In the end the only solution I could come up with was to take one of the disks out and back it up with explore2fs - a Windows tool for reading Linux ext2/3 partitions. I could have also easily used Ext2 IFS, but that’s a little more work to set up. explore2fs recovered all the data thankfully so I formatted it as a NTFS disk and copied all the data back. After moving things around and re-creating the RAID 0 array, it was back in business.
I don’t think this problem was Ubuntu’s fault at all. RAID is handled by the Linux kernel, which is constantly being chopped and changed. Lesson learnt anyhow.
I also used to use a gigabit ethernet switch for fast file sharing between the desktops and server. The switch was an eBay cheapie, so when it died it was no big loss. Still, I missed the ~50Mb/s transfers from the RAID disks. As a solution in the meantime I decided to try network bridging. For the uninitiated, all it does is basically join a set of ethernet cards into one big interface. It essentially makes the computer into a network switch by passing ethernet frames between the seperate cards. The advantages are obvious over routing the interfaces: you get to keep everything on one subnet, and there is much less CPU overhead. What it meant for me was I could plug my main desktop into a gigabit card, having 1000Mbps access to the server, then have the 100Mbps switch plugged into the other card and it would all work as if it was directly connected to the switch. The only downside I could see was that Linux won’t do packet fragmentation on a bridged network. What this meant was that I couldn’t use 8,000 byte Jumbo frames on the gigabit link, because they wouldn’t be broken down to 1,500 bytes for the 100Mbit link. Only a minor con though - my previous gigabit switch didn’t support jumbo frames either.
So I did some research, and all it boiled down to was changing the interface name in Ubuntu’s network configuration from ‘eth0′ to ‘br0′, then adding a bridge_devices line with the appropriate network card interfaces. One networking restart later, and it was all working perfectly. That is, until I rebooted.
After the reboot, the switch facing card still worked (it’s an onboard Marvell Yukon chipset), but the cheap Realtek-based PCI-E card I was using for the gigabit segment would not join the bridge no matter what I tried. I could set an IP and ping the desktop but trying to add it to the bridge got nothing but ‘invalid argument’ errors.
Two hours later I noticed something strange in the persistent network config file (which defines which MAC addresses get assigned to which ethx interface). There were multiple entries for the Realtek card. Hmm. I got the bright idea to switch the Realtek card out with a generic 10/100 card. What do you know, everything suddenly worked. So now the 10/100 card faces the switch, and the Yukon chip is being used for the gigabit side of things. Netperf claims 100Mbytes/sec flat on a TCP stream test between the 2 machines, and in reality a Samba transfer sits on around 45-50Mbytes/sec. Not too shabby at all!
So Ubuntu is running like a charm now. A few things have impressed me: for instance, setting up MPD for Icecast streaming on Gentoo was nothing less than a chore. It required ALSA to be installed, it required a null ALSA output to get it to actually send Icecast any data, and as a result if the machine doesn’t have a sound card it’s a bit of work to get ALSA to stop throwing a dummy spit every time it tries to start. On Ubuntu, it needed little more than an apt-get install mpd. Fantastic. Setting up things like Apache, Samba and rtorrent were equally painless.
Ubuntu will definitely be the distribution of choice for myself from now on. Gentoo’s been good, but unless things change, it’s not going to keep up with distros like Ubuntu and Fedora.
0 Responses to “First impressions of Ubuntu Server”