Problems with Large Database

Data growth is one of the problems that the DBA of today’s age faces. As the world moves toward the information age the amount of data is increasing significantly everyday. With massive amount of data the problems that pop up are space issues, performance issues and management issues. Not only data is increasing but it is also expanding its dimensions because people tend to know more and more now a days. Databases are designed for one solo purpose and that is data storage and manipulation. So no matter what, data is increasing or not the DBA had to tackle it somehow.

I will not talk much about the problems that the DBA faces with large databases because I have no practical experience of that. However I have the experience of working as a developer on large tables. Well the bottom line is experience counts and it counts more then anything. You have read all the stuff related to SQL and PL/SQL and you think that you can merely do anything with it. Most probably you are thinking wrong because knowing and working with tools available are both different things. I am not saying that both SQL and PL/SQL are something really difficult but when you are using them against large databases, problems will occur and you will be wondering that am I doing something wrong?

I am quoting an example of my own here to explain what I am trying to say. A few days back I got an ad-hoc query to answer. The query was to find all those Mobile numbers which are in table “Test1” suppose and are not in “Test2”. I wrote the following query for that:

SQL>select a.mobileno from test1 a where a.mobileno not in (

2 select b.mobileno from test2);

There is nothing wrong with the logic or query. The only thing wrong is that “Not In” operator is too slow to be used in the above query because I have 44 million records in table “Test1” and 10 million records in “Test2”. This query ran well over 10 hours without results. How can I do this? This question made me almost punching my head down. One of my senior colleagues from whom I have learnt a lot of SQL tricks told me the master blaster query which was as under.

SQL>select a.mobileno from test1 a, test2 b

2 where a.mobileno=b.mobileno (+) and b.mobileno is null;

I copied it pasted it and in under 5 minutes or so I got the required results. Now I use this query in so many ways to enhance performance in my applications and I rarely and I mean rarely use the “Not In” operator. The point I want to make is that you will not find this in any documentation or book because they are not meant for that. You can only learn this by experience.

Cricket! A Love Lost

Since my childhood I used to love cricket. I used to watch it and I used to play as well. That love is long lost. If I had to figure out then it would be the World Cup ’99 that made me allergic of it. The cricket of Pakistan now a day is shameful. We have not an even a single world class player. Take a look at our team and you will have no desire to sit in front of your TV watching your heroes mugging around. Gone are the days when we had players like Wasim Akram, Waqar Younis, Saeed Anwar, Aamer Sohail etc who used to win matches for us. I even have some strange feelings while talking about cricket of Pakistan right now.

Bad time comes on everything. It’s not at all shameful but I think this is not simply a bad time for Pakistani cricket. When you had a president who does not know heck about cricket then what else you can expect. The problems are in wide range and the most basic of them all is that we in Pakistan have undeserving people at the top in each and every field of life and PCB is no exception. PCB has never done anything useful for the cricket. They mishandled each and every issue. The recent issue regarding Shoaib Akhtar is one prime example of that. Don’t even think for a second that I am a fan of this so called cricketer. I think the problem with him is that he made a wrong decision when he decided to be a cricketer. He should have been a model or an actor but who knows what he had done there.

This may seem a bit harsh especially considering the fact that we have cleaned sweep a series. I will not mention the team because it will generate another conspiracy. May be we should invite Holland or Zimbabwe again and then again Bangladesh so Shoaib Malik could have an unbreakable winning streak as a captain.