How to make a PDF from a series of images

I was compiling a supplication extracted from a book of supplications. The problem at hand was to make a PDF out of various A4 images I had i.e. page01.xcf, page02.xcf, .., and so on. (XCF is a GIMP image)

A friend suggested ps2pdf, a command-line tool to convert from PostScript files to PDF.
Since GIMP can save to PS format, I saved all my images into PS format first.
However, the tool would accept only one ps file as input. So, I could get PDF file for each image, separately. Clearly this is not desired.

However, ps2pdf can read input from stdin by using -.
The solution was straightforward now:
$ cat page*.ps | ps2pdf - supplication.pdf

Works out of the box! (though takes some time).

The day Merb joined Rails

It's a historic day in history of Rails (and Merb). Eyepoppingly shocking, Merb 2 has a new name -- Rails 3. Yes! Merb is being merged into Rails 3!

DHH and Yehuda Katz announced it almost simultaneously. 96% people are positive about the merger. If you belong to 4%, David has an answer for you.

Ezra explains how Merb is undergoing a rebirth. This merger would hopefully break the myth that Rails is a monolith. Interesting to see community putting an end to drama and coming together.

Programming has evolved

When I look at the recent advancements in Behaviour Driven Development in Rails, I tend to think that programming has really evolved. Not in terms of advanced data structures and superior algorithms, but as a focus shift of intention.

Earlier the focus was the machine. Days are gone when we thought so much about saving an extra byte, trading code simplicity for speed. A compromise here, a saving there, ad infinitum.

The focus has definitely shifted to human understandability. Beauty of code now lies in its simplicity (some folks would argue that it always did, and they're right, but its more so apparent now).

First it was Ruby, then Rails, and now BDD frameworks like rSpec and Cucumber prove the point. Just have a look at these executable code examples.

Ruby

animals = [ "Tiger", "Cow", "Python", "Horse" ]
animals.each do |animal|
  puts animal + "is cool!"
end
#=> Tiger is cool! Cow is cool! ...

Does a C for loop or C++ iterator look more elegant?

Rails

class Project
  belongs_to              :portfolio
  has_one                 :project_manager
  has_many                :milestones
  has_and_belongs_to_many :categories
end

Code that you can just read, isn't it?

rSpec

lambda {
  employee.develop_great_new_social_networking_app
}.should change(employee, :title).from("Mail Clerk").to("CEO")

Yes, that's a real executable example.

And now, Cucumber

Scenario: Search by topic
   Given there are 240 courses where neither has the topic "biology"
   And there are 3 courses A,B,C that each have "biology" as one of the topics
   When I search for "biology"
   Then I should see a the following courses:
     | title |
     | A     |
     | B     |
     | C     |

More beautiful, more succinct, more concise. That's what programming has evolved to be.

Git your branch name into your shell

I use git on my Ubuntu for my Rails projects, and very frequently I need to know what branch I'm working on right now.
If I can just get my shell to display the current local branch all the time in the prompt, it'd be a lot easier. I just found I could do so.

You need to modify the environment variable PS1 to show your current local git branch.

For me, it looks like this now.

PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w [$(git branch 2>/dev/null| grep "\*" | cut -f 2 -d " ")]\$ '

And it renders out like

kazim@invincible:~/projects/aprioriate [refactoring]$


In short, you need to append the following to the PS1 string.

[$(git branch 2>/dev/null| grep "\*" | cut -f 2 -d " ")]\$


The redirection of stderr to /dev/null is required since it suppresses git error messages in case you're not inside a git directory.

Ugly though it may be, it works. And to make the PS1 persist, add the line into your ~/.bashrc file.

PS: I use bash. For zsh and others, its even easier to do this.

Finally, a solution to file sharing woes

File sharing was never easy, especially for big files. I don't trust GMail, even when it says it allows attachment size of 20M. I often had my GMail frozen attaching just a 7M file, after some 20 minutes (which is way too much for just a 7M file!).

Enter DropBox. 20GB of free online storage. And a desktop interface which sticks to your Windows Explorer/Mac Finder/GNOME Nautilus seamlessly. I won't say this is a new solution, since we had GMail drive once, and many others kept popping up.

What makes DropBox special is ease of use with friendliness. And you can keep working while it synchronizes your files quietly in the background. You can also set maximum download/upload limits, so that your browsing experience remains okay.

Go ahead. Tour Dropbox. See the screencast. Download it. Link your computer and start sharing. And be thankful to me forever!

Shortcuts in GNOME

There are more than one ways to set Keyboard Shortcuts in GNOME. In my Ubuntu 7.04, you can either go to System -> Keyboard Shortucts and change/add keybndings there. This way, you can change keybindings for most common operations like "Launch Web Browser" etc.

There's another way, a very versatile one, provided by GTK. In almost any GTK application, you can give shortcuts to menu items. Just open the menu, and point with your mouse on the menu item you want to give a shortcut to, and press the shortcut key. GNOME will automatically assign that shortcut to that menu item, and remember it.

If the procedure outlined above doesn't work for you for some reason, launch gconf-editor from a terminal. Make sure that desktop/gnome/interface/can_change_accels is true. Happy shortcuts!

Recovering a (Lost) ReiserFS Partition

The partition table on my sister's laptop always seemed fishy to me. It just didn't have the right feel about it. The cylinder numbers and sizes and all, but I never cared. As the laptop went through a series of installations, Ubuntu or Windows, the abnormality showed up.

After installing Windows, I found hard to install GRUB on MBR. The grub-install would refuse, and so would the grub. I was troubled when the output of cfdisk said that 30G is unusable space, out of my 60G disk.

But, I thought its just another routine partition table corruption, and I had a tool in my swiss-army knife - testdisk (http://www.cgsecurity.org/wiki/TestDisk). Testdisk analyses your disk for lost partitions and tries to guess the correct partition table. Then it shows the guess to the user, who can then choose to write the table or not.

In my case, though testdisk correctly guessed the partitions (it found more than actual ones), it could not guess a proper partition table. Instead, it asked me to specify which partition is primary, logical or bootable primary. Now that was difficult for me. And I didn't want to take risks.

One good thing about testdisk is that it lets you peek inside partitions and read filenames. I was particularly interested in recovering a ReiserFS partition, which was 20G in size. Fortunately, testdisk was able to see this partition, and the filenames too. It was found lying from cylinders 3856 to 6287.

Anybody who knows a bit about recovery knows that if you know the cylinder numbers, you've solved the problem. Recovering a partition becomes an easy job. I started by dd'ing out from cylinder 3856 about 1MB. But the result was not as expected. It wasn't a superblock. It was just 'data'. Just data? How could it be that testdisk was lying?

$ file mydump
mydump: data

I checked the calculations again, yet to no avail. It was data there. One thing was for sure -- the superblock must begin somewhere near cylinder 3856. I got a weird idea. I quickly made a dummy file of 200M using dd. I made a loop device out of it, and did a reiserfs format over it.

$ dd if=/dev/zero of=check-rfs bs=1M count=200
$ sudo losetup -f check-rfs
$ sudo mkfs.reiserfs /dev/loop0
$ hd check-rfs | less

I intended to find out the pattern with which a reiserfs superblock starts. The output of hexdump showed something like this:

00000000 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
*
00010000 00 32 00 00 ec 11 00 00 13 20 00 00 12 00 00 00 |.2....... ......|
00010010 00 00 00 00 00 20 00 00 00 04 00 00 93 98 2f 43 |..... ......../C|
00010020 84 03 00 00 1e 00 00 00 00 00 00 00 00 10 cc 03 |................|
00010030 02 00 01 00 52 65 49 73 45 72 32 46 73 00 00 00 |....ReIsEr2Fs...|
00010040 03 00 00 00 02 00 01 00 02 00 00 00 00 00 00 00 |................|

Look how smartly the filesystem developers have written ReIsEr2Fs there! The third line, which has 2 and a space, actually contains disk label.

With this amount of information, I tried to find the superblock in the 10MB dump.

$ hd mydump | grep -i reiser2fs

And fortunately, it was there! At an offset of about 1d200h from cylinder 3856.

0001d200 02 00 01 00 52 65 49 73 45 72 32 46 73 00 00 00 |....ReIsEr2Fs...|

Looking a few 100 bytes before this pattern, I found exactly the same fingerprint of ReiserFS superblock! After calculating and adding, I did a dd dump from the correct byes, and here it was -- the superblock!

$ file dump-again
dump-again: ReiserFS V3.6 block size 4096 num blocks 4883760 r5 hash


I took a bigger dump (2G), and tried to mount it. But mount refused, simply because there was not enough space on the loop device than the partition required (20G).
I looked for some IRC help, and enouf on ##linux suggested the offset option of the mount command. It can do just what I wanted -- to mount a range of bytes of a disk as a device.

$ sudo mount /dev/sda recovered/ -o ro,loop,offset=31716679680
$ ls recovered/
bin cdrom etc initrd initrd.img.old lost+found mnt proc sbin sys tools var vmlinuz.old
boot dev home initrd.img lib media opt root srv tmp usr vmlinuz

And there it was! Recovered intact. The next thing I did was to backup the 15G data it had. And I got a treat from my sister, too. :-P

Simplicity is often wonderful. And so is Linux.

Reinventing Rails

These holidays, I was busy developing spendwrite.com. The idea behind spendwrite is a community driven website where users share their experiences about the various places they visit in their town, like restaurants, coffee shops, dentists or shopping malls etc. Everyone reviews the outlets and rates them, so one can easily find the best of places around them.

In the process of development, I learned that it requires a lot of courage and patience to develop a web application. It also requires foresight.

To start with, I chose PHP as the language. Though I love the Ruby on Rails framework, I didn't choose it because its painfully slow. What I intended was, a functionality similar to the one offered by RoR framework, without a real framework in place.

MVC is the leading architecture used by modern frameworks. Rails is no exception, and so many other frameworks use it. So I decided to have an MVC architecture for my website.

I'd learned a lot about MVC and wrote little applications using Rails as well. But the advantage of using MVC was not entirely clear to me before I developed spendwrite.com.

To begin with, I had vague ideas only. I created four directories, like Rails does, called 'models', 'views', 'controllers' and 'helpers'. To be true to MVC, a templating engine was required to play the role of 'views'. Enter Smarty, the templating engine we all know and love. With its caching abilities, it was clearly better than any other templating engines available for php.

Initially, I was creating a model for each table in my database. But as I wrote more code, I found out that much of the code to the models are common. Thats where ActiveRecord component of RoR clicked to my mind. ActiveRecord has a basic CRUD code same for all models. On top of that, individual models define their own methods.

So, I did the same. I carefully wrote CRUD functions, and made this code common to all models. This modular approach was fruitful. Models were tidy now, and the code was easier to manage and debug. Once the CRUD is solid, the models work flawlessly.

Controllers. I must confess, that the habit of writing all-code-in-one-file doesn't go overnight. I wrote a lot of code in controllers, which I shouldn't have. For example, the code for validating user input. Though I had defined helpers like validate_numericality_of and validate_minlength_of etc. in a helpers file, yet writing validation logic in controllers didn't make sense. It took some time for me to realize that validation actually belongs to models, and not to controllers.

Code rearrangement followed. By having all the validation done in models, you relieve controllers of the repetitious job.

For authorization, I'd love to implement something like before_filter in Rails. Since I really couldn't find equivalent, I wrote a model for authorization, and every controller must call a method must_authorize(); if it wants so.

As I write more and more code, I come to realise the beauty of MVC. If you follow the MVC principles strictly, you write less code and error free code. And I also realize how beautifully Rails has implemented MVC architecture.

Rails is no doubt the most productive framework available today for web development. The more you don't use Rails, the more you realize it. :-P

Learning by experimenting is the best way to learn. And while I do, I find myself reinventing Rails. Any coincidence?

Linux is fun for me

Do you've a SysRQ button on your keyboard? No. You must've the PrintScreen button, next to Scroll Lock? Yes. Then you can try out kernel keyboard shortcuts.

The fastest (and most unsafe) way to reboot your computer is
to press these these combinations, one after one.
Alt+SysRQ+s - Sync data to disks for all mounted disks immediately.
Alt+SysRQ+u - Remount all filesystems as readonly.
Alt+SysRQ+b - Reboot immediately.

There're a lot of kernel functions availible, depending upon your kernel version.
You can find them in kernelsource/Documentation/sysrq.txt file.

You can enable all of kernel functions by,

echo 1 >/proc/sys/kernel/sysrq

Pressing Alt+SysRQ+h should tell you about various sysRQ functions availible (in dmesg output). For example, on my system (Feisty Fawn), they are

SysRq : HELP : loglevel0-8 reBoot Crashdump tErm Full kIll saK showMem Nice powerOff showPc unRaw Sync showTasks Unmount shoW-blocked-tasks

The saK (secure access key) can be used to allow secure login. The output of kernel functions, if any, goes to /var/log/messages (or dmesg).

Google will free the mobiles

Google has unpacked the gPhone using Android. Its the next biggest thing in mobile industry. Having your phone run open source code and hacking it the way we hack Linux. Well, it looks promising now.

The Open Handset Alliance founded by Google includes big names from the mobile world - Nvidia, Intel, Motorola, Samsung, LG, Qualcomm etc. Together these companies shall come forward to manufacture and design gPhones, that will run the open system called Android.

Now, gPhone ain't a phone like iPhone. Its not even a phone for that matter. Its just a prototype. Their idea is to have thousands of gPhones, just like we've thousands of Linux distros! The handsets shall be coming to market next year.

Well, some big players are still missing. Nokia, Sony Ericcson, Blackberry, Apple, Verizon and AT&T. And yes, Microsoft.

A video by the developers introducing Android
Another video by kids, who want their phone to make coookies for them.
Open Handset Alliance
Official Google Blog: Where's my Gphone?

Google had acquired Android Inc. two years ago, in a very quiet manner. Android is being developed as a mobile platform to be released under Apache v2 license. The complete documentation and Android SDK shall be released on November 12, 2007.

We know there've been other initiatives as well. OpenMoko, Qutopia, and now Android. The mobile shall be finally freed!

TuxCoders made it at Nerdz!

We're excited. We're happy! And we're the winners. At Nerdz 2007, Jamia Hamdard!

For those who don't know, Nerdz is the annual technical fest organized by Department of Computer Sciences at Jamia Hamdard. They host a variety of contests, including on the spot programming, overnight coding, debugging etc.

So we were the team known as TuxCoders. We were four - me, shamail, aaveg & bharat. And we bagged three prizes! So we really rocked!

Envision, the predesigned software contest. Shamail & Aaveg presenting JamP2P. There was no second thought about being the first.

Crack the Shell, the unix shell programming contest. Me & Shamail. And we won by a big margin! Who needs to say more?

Google Dance, the googling competition. Google and find answers to weird questions. Me & Bharat. And we got the second prize, again!

It felt somewhat deceiving, when we had to pay 500 bucks as the participation charges. It seemed a big risk! However, our confidence rose as the day passed. And the end, we bagged three trophies! And free food for the two days for all participants! So we made a good deal in the end.

Nerdz '07 was a success, and we enjoyed a lot! I wished we had more guys from our college. But there's always a "next time"! But now, its time to celebrate! TuxEnjoy!

Get rid of RAID!

RID. That is RAID without the A. I'll show you how you can experiment with RAID without having multiple drives, which are, not-so-inexpensive actually.



If you don't know what is RAID, a half-an-hour glance through this TLDP guide will get you going. RAID is actually the feature of Linux that helps you append multiple drives (or partitions on different drives) as one device, with different kinds of features. RAID can also mirror your drive data, and protect your data from crashes.

So, to get started using RAID, you need multiple drives. Wait! Is that really true? Do you know about loop devices? That is, devices within devices. Can I fool the RAID by asking it to make an array from loop devices?

The answer is yes! Its possible! Here's a short experiment you can do to believe me!

First, let us create two files, that will act as two devices to RAID.
$ dd if=/dev/zero of=disk0 bs=1M count=256
$ cp disk0 disk1
The first command takes some time. Now you've two zero'd files, ready to be used as loop devices.

Using losetup, we'll setup the loop devices. If you're not already using any loops, you'll get loop0 and loop1 for the two files.
$ sudo losetup -f disk0
$ sudo losetup -f disk1
If they are successful, loop devices have been created. Just to check, you can run
$ sudo losetup /dev/loop0
$ sudo losetup /dev/loop1

Two loop devices ready! Its time to ask RAID to make an array out of them. This is simple, with a self-explanatory command, as
$ sudo mdadm --create --verbose /dev/md0 --level=raid0 --raid-devices=2 /dev/loop0 /dev/loop1
Take a deep breath, and read that command again. It's actually simple. If you're going along with me, you should see something like this,
mdadm: chunk size defaults to 64K
mdadm: array /dev/md0 started.

Great! We've made the RAID array device. Now its time we make it meaningful. Lets format it using ext3 filesystem!
$ sudo mkfs.ext3 /dev/md0
This writes inode tables, superblocks and creates journal. ( If you're just curious, let me explain that the "actual" writing was done on our files disk0 and disk1 )

Now we can just mount this device. Simply type,
$ mkdir myraid
$ sudo mount /dev/md0 myraid
If its successful, try moving inside.
$ cd myraid; ls
You'll see the lost+found folder there. Copy here a few big files to see it works actually. Unmount and mount again. It works!

To see that whether RAID has really done the job, you can
$ df -h | grep md0
/dev/md0 496M 11M 460M 3% /home/kazim/myraid
There we are, a drive with 496M!

Now you can do everything you wanted to do with RAID. For example, try to make the disk1 crash! Since we used raid0 level, a crash means no recovery! If we used raid4 with spare disks, we could even see recovery process going on!

How do you make disk1 'crash' ? Well, you should find a good answer yourself. For data corruption, you can destroy its superblock, or shred it, or do anything bad you can think of! RAID won't recover from a data corruption. But you can simulate hardware crash by using the --set-faulty option of mdadm. (Try crashing a mirror device on a RAID1 with a spare device and see how RAID recovers data.)

Specifically, however, if you try to delete the drives (while md0 running),
$ sudo rm disk0
$ sudo rm disk1
There'll be no data loss. You would ask, "why?". Well, go and revise your OS concepts. And you'll know why 'removing' is not 'crashing'.

Happy RAIDing!

Web 2.0 - the hype?

If you still don't understand why humans have made so much hype about the Web 2.0, and you think that its a crap, you should watch

Web 2.0 - The Machine is Us/ing Us (YouTube Video)

Web is like growth of a civilization. The civilization of the netizens. And when we say 2.0, we appreciate our growth. We are more organised. We are teaching the machine.

Read Prof Wesch's blog, on what he has to say about the video he made.

We've seen revolutionary changes in the way we work on internet. We think the change deserves a name, Web 2.0.

unlikely() & likely()

This week wasn't really productive. No preparations for sessionals yet. Stayed up last night so missed the college today. And some idiot classmates must be thinking now that I was preparing hard for sessionals. Poor kids, ;-)

Actually, I tried my hands with Linux Kernel Development. And its such an amazing thing!!

For starting out, I downloaded the source of latest kernel as linux-2.6.22.5.tar.bz2.
I unpacked it, and started reading code. And as some of you must have guessed it, it was beyond my limits of understanding.

Its quite difficult to start actually. Including the comments, the whole kernel code (latest) is about 7.3 million lines. Too much, na? The first version, 0.0.1 had about 10000 lines. I read somewhere that lately they've not been adding much to this code, but refining it and fixing bugs.

The Linux kernel is one of the best (or the best as they say) kernels around. It has got a great mix of good algorithms, and it has refrained from bad policies and poor design. Its monolithic and still quite modular.

Its well commented, and the code gives me a feeling that it can be understood, given time & patience. Well I've got some books too, notably Linux Kernel Development 2nd Edition, which will help me as I go.

There are some interesting things when writing kernel code. First, you can do everything and anything. Nobody will trap you or accuse you or kill you. No memory protection (as userspace has got), no faults and error trapping. You need to behave responsibly.

Second, you don't have the standard C library. No printf (though a printk is there). No great time saving functions. But fortunately, most useful ones are there in kernel headers (like string comparison etc.)

Third, you have a small, fixed-size stack. Unlike userspace applications, you just cannot allocate tons of variables! So every byte counts here.

Fourth, no floats in kernel code. No floating point arithmetic here. Because kernel has a do-it-yourself overhead. So, just integers and chars.

Finally, the one I found most interesting is about predictability. Conditional branches can be optimized as either very likely or very unlikely by GCC. The macros likely() and unlikely() do this work.

/* file is almost never NULL */
if (unlikely(file == NULL)) { .. }

/* the two sizes are almost always equal */
if (likely(new_size == old_size)) { .. }

Compiler will optimize these branches as the programmer has predicted. If prediction was good, it will improve performance. Otherwise it can degrade performance, too. You'll find unlikely() a lot in kernel code.

Its an intelligent piece of code. And so it requires a lot of time to study. Well, I am prepared for anything! Let's tame the beast!

Get organised!

This one goes out to all my friends and foes who are a part of FLOSS community. Now is the right moment for all of us to get more organised. We don't have much time. Its now or its never. The FLOSS community is calling upon us. Get involved!

You are a free programmer or designer or developer. You are free to do whatever you want. You are free to develop software for yourself. You are free to make it big and better. But not for very long. Because as soon as you get a job, your freedom is lost. The fun for programming is over. Your spirit dies when you do it for money and position.

Imagine, you are a software engineer. Working for 9 hours a day. You just got a 100 page specification from your boss. You got a deadline. And now, painful days of working like a machine. 'Machine' because you are to do what you are told. You cannot innovate. You are not allowed to think. You ain't free. Your freedom sold in dollars.

Again, imagine you are an open source developer. The scenario is different now. You are working on a project you chose for yourself. You are enthusiastic about your work. You've new and great ideas and you are gonna implement them. The fun of programming is alive and everyday is a holiday. You innovate new things, and you find appreciation. You find greatness.

As Niyam (at ILUGD) puts it, "You are born unknown. You'll die unknown. You won't be great if you don't do great things. Join the free software community and get known. Then only you'll be called great. Find that greatness!"

As a student, you've taken the freedom you enjoy for granted. But once you've a boss, you are no longer free. You are a slave of your boss, and your boss is a slave of his boss.

You don't need to be a Linux geek to join us. All you need is a belief in software freedom. There's no second to waste. Rise against ignorance. Make everyone aware of free software. Participate in development! Get known. Get your friends involved.

Freed.in is near. Fasten your seat belts! We've to make it bigger than ever. Free your friends from proprietary software. Involve as many people as you can! Tell everyone what is free software. Tell them to use it. Invite them to Freed.

Gear up! The world is yours!

And don't hesitate to ask me if you don't know what to do.

To blog or not to blog

But that's not the question, since I've decided finally, after so many requests from so many people (read exaggeration), to blog. This is intended to be my technical blog, about all the funny and stupid and tragic things that I do with technology.

Why Tux Playground? Tux is the name of the penguin that first chased Linus Torvalds, and since then, the whole world has been chasing him, including me. Playground because its all about hacks and tweaks! I think thats all about it. See you again :-)