Reinventing Rails

These holidays, I was busy developing spendwrite.com. The idea behind spendwrite is a community driven website where users share their experiences about the various places they visit in their town, like restaurants, coffee shops, dentists or shopping malls etc. Everyone reviews the outlets and rates them, so one can easily find the best of places around them.

In the process of development, I learned that it requires a lot of courage and patience to develop a web application. It also requires foresight.

To start with, I chose PHP as the language. Though I love the Ruby on Rails framework, I didn't choose it because its painfully slow. What I intended was, a functionality similar to the one offered by RoR framework, without a real framework in place.

MVC is the leading architecture used by modern frameworks. Rails is no exception, and so many other frameworks use it. So I decided to have an MVC architecture for my website.

I'd learned a lot about MVC and wrote little applications using Rails as well. But the advantage of using MVC was not entirely clear to me before I developed spendwrite.com.

To begin with, I had vague ideas only. I created four directories, like Rails does, called 'models', 'views', 'controllers' and 'helpers'. To be true to MVC, a templating engine was required to play the role of 'views'. Enter Smarty, the templating engine we all know and love. With its caching abilities, it was clearly better than any other templating engines available for php.

Initially, I was creating a model for each table in my database. But as I wrote more code, I found out that much of the code to the models are common. Thats where ActiveRecord component of RoR clicked to my mind. ActiveRecord has a basic CRUD code same for all models. On top of that, individual models define their own methods.

So, I did the same. I carefully wrote CRUD functions, and made this code common to all models. This modular approach was fruitful. Models were tidy now, and the code was easier to manage and debug. Once the CRUD is solid, the models work flawlessly.

Controllers. I must confess, that the habit of writing all-code-in-one-file doesn't go overnight. I wrote a lot of code in controllers, which I shouldn't have. For example, the code for validating user input. Though I had defined helpers like validate_numericality_of and validate_minlength_of etc. in a helpers file, yet writing validation logic in controllers didn't make sense. It took some time for me to realize that validation actually belongs to models, and not to controllers.

Code rearrangement followed. By having all the validation done in models, you relieve controllers of the repetitious job.

For authorization, I'd love to implement something like before_filter in Rails. Since I really couldn't find equivalent, I wrote a model for authorization, and every controller must call a method must_authorize(); if it wants so.

As I write more and more code, I come to realise the beauty of MVC. If you follow the MVC principles strictly, you write less code and error free code. And I also realize how beautifully Rails has implemented MVC architecture.

Rails is no doubt the most productive framework available today for web development. The more you don't use Rails, the more you realize it. :-P

Learning by experimenting is the best way to learn. And while I do, I find myself reinventing Rails. Any coincidence?

Linux is fun for me

Do you've a SysRQ button on your keyboard? No. You must've the PrintScreen button, next to Scroll Lock? Yes. Then you can try out kernel keyboard shortcuts.

The fastest (and most unsafe) way to reboot your computer is
to press these these combinations, one after one.
Alt+SysRQ+s - Sync data to disks for all mounted disks immediately.
Alt+SysRQ+u - Remount all filesystems as readonly.
Alt+SysRQ+b - Reboot immediately.

There're a lot of kernel functions availible, depending upon your kernel version.
You can find them in kernelsource/Documentation/sysrq.txt file.

You can enable all of kernel functions by,

echo 1 >/proc/sys/kernel/sysrq

Pressing Alt+SysRQ+h should tell you about various sysRQ functions availible (in dmesg output). For example, on my system (Feisty Fawn), they are

SysRq : HELP : loglevel0-8 reBoot Crashdump tErm Full kIll saK showMem Nice powerOff showPc unRaw Sync showTasks Unmount shoW-blocked-tasks

The saK (secure access key) can be used to allow secure login. The output of kernel functions, if any, goes to /var/log/messages (or dmesg).

Google will free the mobiles

Google has unpacked the gPhone using Android. Its the next biggest thing in mobile industry. Having your phone run open source code and hacking it the way we hack Linux. Well, it looks promising now.

The Open Handset Alliance founded by Google includes big names from the mobile world - Nvidia, Intel, Motorola, Samsung, LG, Qualcomm etc. Together these companies shall come forward to manufacture and design gPhones, that will run the open system called Android.

Now, gPhone ain't a phone like iPhone. Its not even a phone for that matter. Its just a prototype. Their idea is to have thousands of gPhones, just like we've thousands of Linux distros! The handsets shall be coming to market next year.

Well, some big players are still missing. Nokia, Sony Ericcson, Blackberry, Apple, Verizon and AT&T. And yes, Microsoft.

A video by the developers introducing Android
Another video by kids, who want their phone to make coookies for them.
Open Handset Alliance
Official Google Blog: Where's my Gphone?

Google had acquired Android Inc. two years ago, in a very quiet manner. Android is being developed as a mobile platform to be released under Apache v2 license. The complete documentation and Android SDK shall be released on November 12, 2007.

We know there've been other initiatives as well. OpenMoko, Qutopia, and now Android. The mobile shall be finally freed!

TuxCoders made it at Nerdz!

We're excited. We're happy! And we're the winners. At Nerdz 2007, Jamia Hamdard!

For those who don't know, Nerdz is the annual technical fest organized by Department of Computer Sciences at Jamia Hamdard. They host a variety of contests, including on the spot programming, overnight coding, debugging etc.

So we were the team known as TuxCoders. We were four - me, shamail, aaveg & bharat. And we bagged three prizes! So we really rocked!

Envision, the predesigned software contest. Shamail & Aaveg presenting JamP2P. There was no second thought about being the first.

Crack the Shell, the unix shell programming contest. Me & Shamail. And we won by a big margin! Who needs to say more?

Google Dance, the googling competition. Google and find answers to weird questions. Me & Bharat. And we got the second prize, again!

It felt somewhat deceiving, when we had to pay 500 bucks as the participation charges. It seemed a big risk! However, our confidence rose as the day passed. And the end, we bagged three trophies! And free food for the two days for all participants! So we made a good deal in the end.

Nerdz '07 was a success, and we enjoyed a lot! I wished we had more guys from our college. But there's always a "next time"! But now, its time to celebrate! TuxEnjoy!

Get rid of RAID!

RID. That is RAID without the A. I'll show you how you can experiment with RAID without having multiple drives, which are, not-so-inexpensive actually.



If you don't know what is RAID, a half-an-hour glance through this TLDP guide will get you going. RAID is actually the feature of Linux that helps you append multiple drives (or partitions on different drives) as one device, with different kinds of features. RAID can also mirror your drive data, and protect your data from crashes.

So, to get started using RAID, you need multiple drives. Wait! Is that really true? Do you know about loop devices? That is, devices within devices. Can I fool the RAID by asking it to make an array from loop devices?

The answer is yes! Its possible! Here's a short experiment you can do to believe me!

First, let us create two files, that will act as two devices to RAID.
$ dd if=/dev/zero of=disk0 bs=1M count=256
$ cp disk0 disk1
The first command takes some time. Now you've two zero'd files, ready to be used as loop devices.

Using losetup, we'll setup the loop devices. If you're not already using any loops, you'll get loop0 and loop1 for the two files.
$ sudo losetup -f disk0
$ sudo losetup -f disk1
If they are successful, loop devices have been created. Just to check, you can run
$ sudo losetup /dev/loop0
$ sudo losetup /dev/loop1

Two loop devices ready! Its time to ask RAID to make an array out of them. This is simple, with a self-explanatory command, as
$ sudo mdadm --create --verbose /dev/md0 --level=raid0 --raid-devices=2 /dev/loop0 /dev/loop1
Take a deep breath, and read that command again. It's actually simple. If you're going along with me, you should see something like this,
mdadm: chunk size defaults to 64K
mdadm: array /dev/md0 started.

Great! We've made the RAID array device. Now its time we make it meaningful. Lets format it using ext3 filesystem!
$ sudo mkfs.ext3 /dev/md0
This writes inode tables, superblocks and creates journal. ( If you're just curious, let me explain that the "actual" writing was done on our files disk0 and disk1 )

Now we can just mount this device. Simply type,
$ mkdir myraid
$ sudo mount /dev/md0 myraid
If its successful, try moving inside.
$ cd myraid; ls
You'll see the lost+found folder there. Copy here a few big files to see it works actually. Unmount and mount again. It works!

To see that whether RAID has really done the job, you can
$ df -h | grep md0
/dev/md0 496M 11M 460M 3% /home/kazim/myraid
There we are, a drive with 496M!

Now you can do everything you wanted to do with RAID. For example, try to make the disk1 crash! Since we used raid0 level, a crash means no recovery! If we used raid4 with spare disks, we could even see recovery process going on!

How do you make disk1 'crash' ? Well, you should find a good answer yourself. For data corruption, you can destroy its superblock, or shred it, or do anything bad you can think of! RAID won't recover from a data corruption. But you can simulate hardware crash by using the --set-faulty option of mdadm. (Try crashing a mirror device on a RAID1 with a spare device and see how RAID recovers data.)

Specifically, however, if you try to delete the drives (while md0 running),
$ sudo rm disk0
$ sudo rm disk1
There'll be no data loss. You would ask, "why?". Well, go and revise your OS concepts. And you'll know why 'removing' is not 'crashing'.

Happy RAIDing!

Web 2.0 - the hype?

If you still don't understand why humans have made so much hype about the Web 2.0, and you think that its a crap, you should watch

Web 2.0 - The Machine is Us/ing Us (YouTube Video)

Web is like growth of a civilization. The civilization of the netizens. And when we say 2.0, we appreciate our growth. We are more organised. We are teaching the machine.

Read Prof Wesch's blog, on what he has to say about the video he made.

We've seen revolutionary changes in the way we work on internet. We think the change deserves a name, Web 2.0.

unlikely() & likely()

This week wasn't really productive. No preparations for sessionals yet. Stayed up last night so missed the college today. And some idiot classmates must be thinking now that I was preparing hard for sessionals. Poor kids, ;-)

Actually, I tried my hands with Linux Kernel Development. And its such an amazing thing!!

For starting out, I downloaded the source of latest kernel as linux-2.6.22.5.tar.bz2.
I unpacked it, and started reading code. And as some of you must have guessed it, it was beyond my limits of understanding.

Its quite difficult to start actually. Including the comments, the whole kernel code (latest) is about 7.3 million lines. Too much, na? The first version, 0.0.1 had about 10000 lines. I read somewhere that lately they've not been adding much to this code, but refining it and fixing bugs.

The Linux kernel is one of the best (or the best as they say) kernels around. It has got a great mix of good algorithms, and it has refrained from bad policies and poor design. Its monolithic and still quite modular.

Its well commented, and the code gives me a feeling that it can be understood, given time & patience. Well I've got some books too, notably Linux Kernel Development 2nd Edition, which will help me as I go.

There are some interesting things when writing kernel code. First, you can do everything and anything. Nobody will trap you or accuse you or kill you. No memory protection (as userspace has got), no faults and error trapping. You need to behave responsibly.

Second, you don't have the standard C library. No printf (though a printk is there). No great time saving functions. But fortunately, most useful ones are there in kernel headers (like string comparison etc.)

Third, you have a small, fixed-size stack. Unlike userspace applications, you just cannot allocate tons of variables! So every byte counts here.

Fourth, no floats in kernel code. No floating point arithmetic here. Because kernel has a do-it-yourself overhead. So, just integers and chars.

Finally, the one I found most interesting is about predictability. Conditional branches can be optimized as either very likely or very unlikely by GCC. The macros likely() and unlikely() do this work.

/* file is almost never NULL */
if (unlikely(file == NULL)) { .. }

/* the two sizes are almost always equal */
if (likely(new_size == old_size)) { .. }

Compiler will optimize these branches as the programmer has predicted. If prediction was good, it will improve performance. Otherwise it can degrade performance, too. You'll find unlikely() a lot in kernel code.

Its an intelligent piece of code. And so it requires a lot of time to study. Well, I am prepared for anything! Let's tame the beast!