Recovering a (Lost) ReiserFS Partition

The partition table on my sister's laptop always seemed fishy to me. It just didn't have the right feel about it. The cylinder numbers and sizes and all, but I never cared. As the laptop went through a series of installations, Ubuntu or Windows, the abnormality showed up.

After installing Windows, I found hard to install GRUB on MBR. The grub-install would refuse, and so would the grub. I was troubled when the output of cfdisk said that 30G is unusable space, out of my 60G disk.

But, I thought its just another routine partition table corruption, and I had a tool in my swiss-army knife - testdisk (http://www.cgsecurity.org/wiki/TestDisk). Testdisk analyses your disk for lost partitions and tries to guess the correct partition table. Then it shows the guess to the user, who can then choose to write the table or not.

In my case, though testdisk correctly guessed the partitions (it found more than actual ones), it could not guess a proper partition table. Instead, it asked me to specify which partition is primary, logical or bootable primary. Now that was difficult for me. And I didn't want to take risks.

One good thing about testdisk is that it lets you peek inside partitions and read filenames. I was particularly interested in recovering a ReiserFS partition, which was 20G in size. Fortunately, testdisk was able to see this partition, and the filenames too. It was found lying from cylinders 3856 to 6287.

Anybody who knows a bit about recovery knows that if you know the cylinder numbers, you've solved the problem. Recovering a partition becomes an easy job. I started by dd'ing out from cylinder 3856 about 1MB. But the result was not as expected. It wasn't a superblock. It was just 'data'. Just data? How could it be that testdisk was lying?

$ file mydump
mydump: data

I checked the calculations again, yet to no avail. It was data there. One thing was for sure -- the superblock must begin somewhere near cylinder 3856. I got a weird idea. I quickly made a dummy file of 200M using dd. I made a loop device out of it, and did a reiserfs format over it.

$ dd if=/dev/zero of=check-rfs bs=1M count=200
$ sudo losetup -f check-rfs
$ sudo mkfs.reiserfs /dev/loop0
$ hd check-rfs | less

I intended to find out the pattern with which a reiserfs superblock starts. The output of hexdump showed something like this:

00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00010000 00 32 00 00 ec 11 00 00 13 20 00 00 12 00 00 00 |.2....... ......|
00010010 00 00 00 00 00 20 00 00 00 04 00 00 93 98 2f 43 |..... ......../C|
00010020 84 03 00 00 1e 00 00 00 00 00 00 00 00 10 cc 03 |................|
00010030 02 00 01 00 52 65 49 73 45 72 32 46 73 00 00 00 |....ReIsEr2Fs...|
00010040 03 00 00 00 02 00 01 00 02 00 00 00 00 00 00 00 |................|

Look how smartly the filesystem developers have written ReIsEr2Fs there! The third line, which has 2 and a space, actually contains disk label.

With this amount of information, I tried to find the superblock in the 10MB dump.

$ hd mydump | grep -i reiser2fs

And fortunately, it was there! At an offset of about 1d200h from cylinder 3856.

0001d200 02 00 01 00 52 65 49 73 45 72 32 46 73 00 00 00 |....ReIsEr2Fs...|

Looking a few 100 bytes before this pattern, I found exactly the same fingerprint of ReiserFS superblock! After calculating and adding, I did a dd dump from the correct byes, and here it was -- the superblock!

$ file dump-again
dump-again: ReiserFS V3.6 block size 4096 num blocks 4883760 r5 hash


I took a bigger dump (2G), and tried to mount it. But mount refused, simply because there was not enough space on the loop device than the partition required (20G).
I looked for some IRC help, and enouf on ##linux suggested the offset option of the mount command. It can do just what I wanted -- to mount a range of bytes of a disk as a device.

$ sudo mount /dev/sda recovered/ -o ro,loop,offset=31716679680
$ ls recovered/
bin cdrom etc initrd initrd.img.old lost+found mnt proc sbin sys tools var vmlinuz.old
boot dev home initrd.img lib media opt root srv tmp usr vmlinuz

And there it was! Recovered intact. The next thing I did was to backup the 15G data it had. And I got a treat from my sister, too. :-P

Simplicity is often wonderful. And so is Linux.

Reinventing Rails

These holidays, I was busy developing spendwrite.com. The idea behind spendwrite is a community driven website where users share their experiences about the various places they visit in their town, like restaurants, coffee shops, dentists or shopping malls etc. Everyone reviews the outlets and rates them, so one can easily find the best of places around them.

In the process of development, I learned that it requires a lot of courage and patience to develop a web application. It also requires foresight.

To start with, I chose PHP as the language. Though I love the Ruby on Rails framework, I didn't choose it because its painfully slow. What I intended was, a functionality similar to the one offered by RoR framework, without a real framework in place.

MVC is the leading architecture used by modern frameworks. Rails is no exception, and so many other frameworks use it. So I decided to have an MVC architecture for my website.

I'd learned a lot about MVC and wrote little applications using Rails as well. But the advantage of using MVC was not entirely clear to me before I developed spendwrite.com.

To begin with, I had vague ideas only. I created four directories, like Rails does, called 'models', 'views', 'controllers' and 'helpers'. To be true to MVC, a templating engine was required to play the role of 'views'. Enter Smarty, the templating engine we all know and love. With its caching abilities, it was clearly better than any other templating engines available for php.

Initially, I was creating a model for each table in my database. But as I wrote more code, I found out that much of the code to the models are common. Thats where ActiveRecord component of RoR clicked to my mind. ActiveRecord has a basic CRUD code same for all models. On top of that, individual models define their own methods.

So, I did the same. I carefully wrote CRUD functions, and made this code common to all models. This modular approach was fruitful. Models were tidy now, and the code was easier to manage and debug. Once the CRUD is solid, the models work flawlessly.

Controllers. I must confess, that the habit of writing all-code-in-one-file doesn't go overnight. I wrote a lot of code in controllers, which I shouldn't have. For example, the code for validating user input. Though I had defined helpers like validate_numericality_of and validate_minlength_of etc. in a helpers file, yet writing validation logic in controllers didn't make sense. It took some time for me to realize that validation actually belongs to models, and not to controllers.

Code rearrangement followed. By having all the validation done in models, you relieve controllers of the repetitious job.

For authorization, I'd love to implement something like before_filter in Rails. Since I really couldn't find equivalent, I wrote a model for authorization, and every controller must call a method must_authorize(); if it wants so.

As I write more and more code, I come to realise the beauty of MVC. If you follow the MVC principles strictly, you write less code and error free code. And I also realize how beautifully Rails has implemented MVC architecture.

Rails is no doubt the most productive framework available today for web development. The more you don't use Rails, the more you realize it. :-P

Learning by experimenting is the best way to learn. And while I do, I find myself reinventing Rails. Any coincidence?

Linux is fun for me

Do you've a SysRQ button on your keyboard? No. You must've the PrintScreen button, next to Scroll Lock? Yes. Then you can try out kernel keyboard shortcuts.

The fastest (and most unsafe) way to reboot your computer is
to press these these combinations, one after one.
Alt+SysRQ+s - Sync data to disks for all mounted disks immediately.
Alt+SysRQ+u - Remount all filesystems as readonly.
Alt+SysRQ+b - Reboot immediately.

There're a lot of kernel functions availible, depending upon your kernel version.
You can find them in kernelsource/Documentation/sysrq.txt file.

You can enable all of kernel functions by,

echo 1 >/proc/sys/kernel/sysrq

Pressing Alt+SysRQ+h should tell you about various sysRQ functions availible (in dmesg output). For example, on my system (Feisty Fawn), they are

SysRq : HELP : loglevel0-8 reBoot Crashdump tErm Full kIll saK showMem Nice powerOff showPc unRaw Sync showTasks Unmount shoW-blocked-tasks

The saK (secure access key) can be used to allow secure login. The output of kernel functions, if any, goes to /var/log/messages (or dmesg).

Google will free the mobiles

Google has unpacked the gPhone using Android. Its the next biggest thing in mobile industry. Having your phone run open source code and hacking it the way we hack Linux. Well, it looks promising now.

The Open Handset Alliance founded by Google includes big names from the mobile world - Nvidia, Intel, Motorola, Samsung, LG, Qualcomm etc. Together these companies shall come forward to manufacture and design gPhones, that will run the open system called Android.

Now, gPhone ain't a phone like iPhone. Its not even a phone for that matter. Its just a prototype. Their idea is to have thousands of gPhones, just like we've thousands of Linux distros! The handsets shall be coming to market next year.

Well, some big players are still missing. Nokia, Sony Ericcson, Blackberry, Apple, Verizon and AT&T. And yes, Microsoft.

A video by the developers introducing Android
Another video by kids, who want their phone to make coookies for them.
Open Handset Alliance
Official Google Blog: Where's my Gphone?

Google had acquired Android Inc. two years ago, in a very quiet manner. Android is being developed as a mobile platform to be released under Apache v2 license. The complete documentation and Android SDK shall be released on November 12, 2007.

We know there've been other initiatives as well. OpenMoko, Qutopia, and now Android. The mobile shall be finally freed!

TuxCoders made it at Nerdz!

We're excited. We're happy! And we're the winners. At Nerdz 2007, Jamia Hamdard!

For those who don't know, Nerdz is the annual technical fest organized by Department of Computer Sciences at Jamia Hamdard. They host a variety of contests, including on the spot programming, overnight coding, debugging etc.

So we were the team known as TuxCoders. We were four - me, shamail, aaveg & bharat. And we bagged three prizes! So we really rocked!

Envision, the predesigned software contest. Shamail & Aaveg presenting JamP2P. There was no second thought about being the first.

Crack the Shell, the unix shell programming contest. Me & Shamail. And we won by a big margin! Who needs to say more?

Google Dance, the googling competition. Google and find answers to weird questions. Me & Bharat. And we got the second prize, again!

It felt somewhat deceiving, when we had to pay 500 bucks as the participation charges. It seemed a big risk! However, our confidence rose as the day passed. And the end, we bagged three trophies! And free food for the two days for all participants! So we made a good deal in the end.

Nerdz '07 was a success, and we enjoyed a lot! I wished we had more guys from our college. But there's always a "next time"! But now, its time to celebrate! TuxEnjoy!

Get rid of RAID!

RID. That is RAID without the A. I'll show you how you can experiment with RAID without having multiple drives, which are, not-so-inexpensive actually.



If you don't know what is RAID, a half-an-hour glance through this TLDP guide will get you going. RAID is actually the feature of Linux that helps you append multiple drives (or partitions on different drives) as one device, with different kinds of features. RAID can also mirror your drive data, and protect your data from crashes.

So, to get started using RAID, you need multiple drives. Wait! Is that really true? Do you know about loop devices? That is, devices within devices. Can I fool the RAID by asking it to make an array from loop devices?

The answer is yes! Its possible! Here's a short experiment you can do to believe me!

First, let us create two files, that will act as two devices to RAID.
$ dd if=/dev/zero of=disk0 bs=1M count=256
$ cp disk0 disk1
The first command takes some time. Now you've two zero'd files, ready to be used as loop devices.

Using losetup, we'll setup the loop devices. If you're not already using any loops, you'll get loop0 and loop1 for the two files.
$ sudo losetup -f disk0
$ sudo losetup -f disk1
If they are successful, loop devices have been created. Just to check, you can run
$ sudo losetup /dev/loop0
$ sudo losetup /dev/loop1

Two loop devices ready! Its time to ask RAID to make an array out of them. This is simple, with a self-explanatory command, as
$ sudo mdadm --create --verbose /dev/md0 --level=raid0 --raid-devices=2 /dev/loop0 /dev/loop1
Take a deep breath, and read that command again. It's actually simple. If you're going along with me, you should see something like this,
mdadm: chunk size defaults to 64K
mdadm: array /dev/md0 started.

Great! We've made the RAID array device. Now its time we make it meaningful. Lets format it using ext3 filesystem!
$ sudo mkfs.ext3 /dev/md0
This writes inode tables, superblocks and creates journal. ( If you're just curious, let me explain that the "actual" writing was done on our files disk0 and disk1 )

Now we can just mount this device. Simply type,
$ mkdir myraid
$ sudo mount /dev/md0 myraid
If its successful, try moving inside.
$ cd myraid; ls
You'll see the lost+found folder there. Copy here a few big files to see it works actually. Unmount and mount again. It works!

To see that whether RAID has really done the job, you can
$ df -h | grep md0
/dev/md0 496M 11M 460M 3% /home/kazim/myraid
There we are, a drive with 496M!

Now you can do everything you wanted to do with RAID. For example, try to make the disk1 crash! Since we used raid0 level, a crash means no recovery! If we used raid4 with spare disks, we could even see recovery process going on!

How do you make disk1 'crash' ? Well, you should find a good answer yourself. For data corruption, you can destroy its superblock, or shred it, or do anything bad you can think of! RAID won't recover from a data corruption. But you can simulate hardware crash by using the --set-faulty option of mdadm. (Try crashing a mirror device on a RAID1 with a spare device and see how RAID recovers data.)

Specifically, however, if you try to delete the drives (while md0 running),
$ sudo rm disk0
$ sudo rm disk1
There'll be no data loss. You would ask, "why?". Well, go and revise your OS concepts. And you'll know why 'removing' is not 'crashing'.

Happy RAIDing!

Web 2.0 - the hype?

If you still don't understand why humans have made so much hype about the Web 2.0, and you think that its a crap, you should watch

Web 2.0 - The Machine is Us/ing Us (YouTube Video)

Web is like growth of a civilization. The civilization of the netizens. And when we say 2.0, we appreciate our growth. We are more organised. We are teaching the machine.

Read Prof Wesch's blog, on what he has to say about the video he made.

We've seen revolutionary changes in the way we work on internet. We think the change deserves a name, Web 2.0.