The Pensieve

Of late, I have been following up with data analytics and learnt R and Pandas (using Python). Somehow, most of the work that I now do is crunching data and representing them in a more fancier way. In a lighter vein, when I am at home, I try to see how my non-work data can be represented as charts and graphs. Here is the first attempt to categorize the list of programmers editors that I had used throughout my life time... This was generated using R and ggplot (ggplot is a great library that allows you to generate different types of charts). Why not excel? Sure, I can do the same in excel as well. But once the data becomes huge to manage, it becomes difficult to monitor each of the cells to see if the formula is right. Sometimes, they get complicated enough that I have to wait for a few seconds every time I make a change in a cell or consider turning off auto-calculation in excel. It is much more simpler to have the processing logic as a source file that you can modify separated from t...

On my daily commute to my office, my only companion are the various FM radio channels. Most of them were hyping the World Tobacco Day on the 31st of May which included how much a person would have spent if he smokes a few packets to the various health issues. All this was nice until on that same day, where there were a few ads on the same channels sponsored from a tobacco company enticing listeners on their new offerings! Oh what irony!

On the way to Sholinganallur via the IT highway, a flock of Pelicans have made the backwaters their 'in-transit stay' much to the delight of the commuters. Most of the commuters slow down and some (including myself) visited over the weekends to take a snap of the feathered friends.

I had to do some shopping and was lured into a some of these shops by their big bold 'off' posters. But alas, once inside, anything that I touch, the salesman utters "Sorry sir, but that does not have the discount". Not to be out-done, I ask "So what do you have on the discount sale?". He points to a table where there are a 'few' set of shirts that no one would pick up, in violet and pink and fluorescent green. But their prices seems to be barely justifiable even 'after' you apply the discount. Note size of the '50%' and 'Sale' on the poster, where I think the entire wares on discount would comfortably fit in and you will still be able to make out what the poster says. Perhaps there should be some legalization on the size of the '50%' that you can have on the poster to depend upon the number of articles you have on a discount compared to the remaining items. In the end, you end up buying the product at the same price y...

Been installing various desktop search tools like the Google Desktop, Copernic (once free), Microsoft Search etc and after a few weeks / months of usage, eventually felt happier to uninstall them. One of the primary reason I need a desktop search was I needed a tool to quickly search for mails. I would seldom search for files on my machine. But one assumption that we need to keep in mind is that, these search engines are not exact. That is, the engines fetch you results, but they may not be comprehensive. The web crawling spiders that scour the web for data and index them, it is possible thet the server's index may be out of sync with respect to the actual web site as new data would have been posted after the spider index the content. In the Internet, sure, if you have a few sites missed out, it does not matter much. But desktop search apps seem to work in very similar lines. If at some time, for want of speed or to free up available memory, you stop indexing your mails or document...

With the onset of interactive web pages that update dynamically the moment you move your mouse over a control, development of an AJAX (Asynchronous Javascript And XML) based web site adds to the complexity. For one, they mess up with your browser history and limit your ability to bookmark a particular page as almost all contents are loaded "in-page". The other disadvantage I see is the in-ability of custom scripts to access the site and extract information. For example, I can write a perl script that can access the Indian Railways page and query the arrival of a particular train. If the site used AJAX, it's going to be more difficult. Cellphones that bundle with micro-browsers either lack javascripting or access is usually blocked by service providers leaving you grapping with an empty framework with no actual data in it. Development wise, you introduce a sense of uncertainty into your web-page where when your page looks like it has completed loading, more data is still b...

The other day, I was thinking of consolidating all my music collection from various machines and burn them into a DVD as a backup. There were various ways that I could have done the transfer: Extract the music back from the disk originals Copy them using a rewritable media like a pen drive or a writable media like a CD Connect over the network and copy the files I can use Samba as some of my machines are Windows based Setup a FTP server to copy files I can setup a ssh server and securely copy the files to my Linux box That was how I discovered ssh, the secure shell. I had originally been looking at setting up Samba on my Linux box, but was frightened with the possibility of it suffering a virus attach as most of the viruses / botnets would be scanning for a windows PC and this would become a backdoor for entry. The other option was ssh which was simple to setup and quite secure. It basically provides a secure way to connect to a computer that enables the ssh service. You are presented ...

The other day, I saw news item at slashdot where people mentioned that Linux seemed to perform better that Windows on network throughput. Some attributed it to the ISP throttling the bandwidth for the first few minutes where as others mentioned that the Linux network drivers seemed to be configured to maximize network performance where as the Windows settings are sub-par. An indepth article depicting based on actual data is provided which eventually leads to thinking perhaps, you should tweak your windows network settings... Though people had complained that both the machines tested had slightly different specs and hence the performance results could be skewed, I have been browsing on both a windows PC and a linux box and it does look like (no systematic tests conducted!) the throughput that I get on a Linux PC is much higher than as experienced on the windows PC so far. The windows PC or laptop has 512MB RAM and is an Intel Centreno Duo 1.66GHz where as the Linux Box is a older AMD...

Aadi Sale, New Year Sale, Year End Clearence Sale and now the Global Meltdown sale! Happened to see this posted at a window of a garment shop at Pondy Bazzar, T-Nager. An innovative approach to woo in customers. Apart from the PR, this shop sells really good (limited collection) and cheap (3 times cheaper) clothes for kids compared to the branded cos like the Levi's or Lee. London Stores, Pondy Bazar, T-Nagar.

It's unfortunate to see such a big corporation fall down in light of the scam announced yesterday. Share prices of the listed script dropped down from Rs.170 to Rs.35 in the NSE in a matter of hours and the entire IT industry was watching the news unfurl dumbstruck. The fate of 52,000 employees and their families hang and hopefully, the company must either build back the trust of the share holders and its employees or the employees should be is a position to secure jobs in other companies. Only time will tell...

One more year has passed by and the time has come to wish all of you a happy, healthy, safe, peaceful and prosperous New Year! This is the time when one looks past on what has happened and makes plans on what to achieve and focus going forward. And the best of luck in making up your New Year resolutions. As one of my mentor mentioned long back, you would usually be bound and emotionally attached to your resolutions if you commit yourself in paper by signing on the paper what you plan to achieve. Though most of my resolutions are broken in the next few days / weeks, I am planning to be different this time and commit my pain points to paper (maybe I can get a bond paper too, to make it really legal ;-)). Give it a try. You might be able to hold on to your resolutions a little bit longer until it becomes a habit. And no... I'm not going to publish my resolutions here ;-)

Merry Christmas wishes to one and all! In other news, the partial Internet free days make me ponder on reliant we have become on these series of 'tubes' that send and receive data from various servers. This brings in light of the hassles that you may face when you become fully attached to online applications. My e-mail provider provides LDAP and is online-only. You have a plethora of online-apps that provide basic word-processing, do your taxes, view pictures and store presentations etc. If you have the cable cut, you loose access to everything. Telephone and credit card companies are urging their users to move online, to receive bills and statement alerts via e-mail rather than the traditional mail in the pretext of accessibility and saving the environment. I'm not against saving the environment, but most of us are moving towards a system where there is only one way to get informed about the various activities in life and if this one way gets disrupted, I wonder what woul...

Been there, done that. That's what you would hear if you are talking about SPAM filters for your e-mail. The other new area that I am constantly getting irritated about is the mobile SMS SPAM that make their way into my cell phone. One way or the others, we inadvertently give away our mobile phone nos to the various dealers who pass on the same to bulk SMS generators. They in turn make it a point to enlighten you of the various offers that you should be thankful to spend your money on! Thank God I am not charged for incoming messages! I know most providers like banks or the telecos provide a Do Not Call registry usually hidden away in the small-print. They abide by your request, but it is the other sources where you are forced to give away our number from where the main problem arises. One thing that I noticed that most mobile phone lack is an active SPAM filter for your mobile phones that can scan your incoming SMSes and over a course of time, learn from you, on what you regard as...

After being tortured to sit through one too many series of never ending serials, here are a few rules or patterns that I had observed: Design patterns on serials or movies The-Not-Good-as-it-seems Pattern: If it is too good to be true, it is a dream and in real-world, it will happen exactly the opposite The-One-is-not-enough Pattern: One of the primary character will have two wives and will struggle between the two (or more?). Medical-Miracle Pattern: Someone will get amnesia and later will be always be cured. Double-Trouble Pattern: If twins are born, they will get separated to meet again 20 years later. Of-course, there will be scope for impersonation. The-Cryptic-Prophet Pattern: Swamijis, gypsies, sadhus will usually be able to give cryptic clues on what is about to happen next The-Cycle-of-Life Pattern: If a primary actor's family is portrayed as being very rich, chances are they will become poor. Conversely, the family opposing the hero / heroin will have the lifestyle op...

The new Honday City comes with an audio player that takes your portable player, collections of memory or USB sticks. Though a good option that gives flexibility allowing you to carry your music and listen to all of them, the minus point is, they now lack a CD player. I see this as a disadvantage. Guess where I will play a newly purchased CD? In my car, on my way back home! The same goes for CDs purchased during a long trip. I might not have my laptop handy to rip and transfer songs on the go. The music download scenario currently in India is not good either. Amazon and iTunes do not offer song downloads here. So, purchasing online music which is likely to happen during your home or office are not it atleast in India. With these audio system, I will have to hold on until I reach home, rip the songs, copy them to my USB stick and only then, I will be able to listen to them in my car. Though most cars offer to add a CD as an option, for the cost, I feel this should be standard.

Honda recently offered a slash in their new Civic Hybrid model. Though not all are sure if it was a publicity move or an inventory clearing sale, I believe with good mileage figures and a growing concern on how green everyone should be, hybrid cars will see an increase in prominence in the Indian roads. The Indian market is price sensitive. Be it an industrialist who is buying a Rolls Royce or a car enthusiast pining for a BMW, the inevitable question remains: "What is the mileage?". This has led to a slew of Diesel cars in the market . Though economic to us, it still fails against the 'green' parameter. The other competitor to hybrids are the all-electric vehicles which may be termed greener (assuming that the cost of production, battery is far less harmful than the actual emissions), they lack in range, ability to quickly recharge (which seems to be a 5 minute affair for it's counterparts) and the lack of accessible recharge locations (do you think at the curren...

If you have kids who are into schooling, now would be the time when you are revising their knowledge for the upcoming mid-term and half-yearly exams. A good way to check up on your kid's (and your) math skills is to setup up a question paper. Unfortunately, kids have good memory power that you can't reuse the same maths question paper. Otherwise, you would see them breeze through the QP as they tend to 'remember' most of the answers. The alternative is to have multiple QPs. But that is a pain and I had been thinking of coming up with a simple script or software (TeX) to generate the QP dynamically. The final solution was much simpler. Use Excel or Open-office Calc (in my case) to dynamically generate the question paper. The key is, for each of the question, have a bunch of random numbers (to your specifications) generated so that you can construct the parts required for your sum and generate multiple versions of the question paper by recalculating the workbook.

With the advent of e-books, software readers, hand-held devices and the Project Gutenberg , you literally have access to more than a book shelf. A good e-book reader for your mobile when you are standard at some place with nothing to do is Mobipocket reader . It comes along with a good PC based e-book creator that can take simple HTML or .txt files and convert into a format usable from a mobile device. With the text files combined with Emacs and a few regexps, I was able to convert most of the files to readable html files in a few minutes and feed them back to the e-book format converter for hours of readable entertainment. So, my current bookshelf contains: Arabian Nights Robinson Crusoe by Defoe Daniel The Devil's Dictionary by Ambrose Bierce Dracula by Bram Stoker With these books with me, who would mind waiting on a long queue?

Ever since I had seen GCompris , a children activity application, I always wanted to add modules that caters to the need of getting my son to learn while he plays games. GCompris supports a python interface for rapid board development. But one quip that I had was, GCompris is a Linux only application (though they have a commercial win32 port). Now, if I can find a framework that works both on the Linux as well as the windows platform, it would make great sense. Gcompris as of now only will restrict schools to the Linux OS (which is good, but most schools would probably be using Windows and getting them to change would prove difficult) would be good. An alternative framework would be Java with it's great 2D API (no fancy 3D here to confuse the kids). Others would by SDL with C/C++ or pygame (which I read has performance issues on the windows front. Read childsplay ). A new contender that I noticed was the Mono framework which sports a 2D API identical to the DotNet framework. My im...

After numerous hours of tweaking, I resolved the problem with my WG311v3 NetGear PCI card. The problem? The router was too far away! Now that I am online using my Linux box, here are a few screen-shots of my desktop! The next time your supported WiFi card acts up, you know what to do... For those inquisitive lot, I am currently using the XP drivers with ndiswrapper.