Sunday, October 26, 2008

Openoffice getting better and better,

A new version of OpenOffice is available for download.
There are a lot of reasons for upgrade, checkout the release notes here.
it has new artwork new glassy look which may attract most of the users.

The reason I upgraded is that being in IT sector I have to create/edit/review docs, I am always happy to use openoffice(2.4) for all the tasks except replying to review comments(MSFT).
Comments are called notes in openoffice, although you can see and edit the comments created by the junk office suite(which most of the people use) but the comments were not presented in a very nice format, a different window was opened for each note where have to put a reply. Doing this was a bit tiring and I had to make sure that the format of the note is correct. While the junk office suite provides a more fancy way of editing notes/comments every comment is presented in the margin at the right of the page.

Open office3 does the same but in a slightly better way it put the notes in the margin and also provides a scrollbar for big comments.

I use ubuntu and could not find a repository for open office 3 so I had to download the tarball and install it manually.
The installation is very easy just download the tarball and run setup. You will need rpm2cpio for the installer to work. You can chose to install the bins in a private location if you do not want to install it in a system wide location for all users.

HTH

Saturday, October 18, 2008

Keep the Spark - Inaugural Speech for the new batch at the Symbiosis BBA program 2008 by Chetan Bhagat

Good Morning everyone and thank you for giving me this chance to speak to you. This day is about you. You, who have come to this college, leaving the comfort of your homes (or in some cases discomfort), to become something in your life. I am sure you are excited. There are few days in human life when one is truly elated. The first day in college is one of them. When you were getting ready today, you felt a tingling in your stomach. What would the auditorium be like, what would the teachers be like, who are my new classmates - there is so much to be curious about. I call this excitement, the spark within you that makes you feel truly alive today. Today I am going to talk about keeping the spark shining. Or to put it another way, how to be happy most, if not all the time.

Where do these sparks start? I think we are born with them. My 3-year old twin boys have a million sparks. A little Spiderman toy can make them jump on the bed. They get thrills from creaky swings in the park. A story from daddy gets them excited. They do a daily countdown for birthday party – several months in advance – just for the day they will cut their own birthday cake.

I see students like you, and I still see some sparks. But when I see older people, the spark is difficult to find. That means as we age, the spark fades. People whose spark has faded too much are dull, dejected, aimless and bitter. Remember Kareena in the first half of Jab We Met vs the second half? That is what happens when the spark is lost. So how to save the spark?

Imagine the spark to be a lamp's flame. The first aspect is nurturing - to give your spark the fuel, continuously. The second is to guard against storms.

To nurture, always have goals. It is human nature to strive, improve and achieve full potential. In fact, that is success. It is what is possible for you. It isn't any external measure - a certain cost to company pay package, a particular car or house.

Most of us are from middle class families. To us, having material landmarks is success and rightly so. When you have grown up where money constraints force everyday choices, financial freedom is a big achievement.

But it isn't the purpose of life. If that was the case, Mr Ambani would not show up for work. Shah Rukh Khan would stay at home and not dance anymore. Steve Jobs won't be working hard to make a better iPhone, as he sold Pixar for billions of dollars already. Why do they do it? What makes them come to work everyday?

They do it because it makes them happy. They do it because it makes them feel alive. Just getting better from current levels feels good. If you study hard, you can improve your rank. If you make an effort to interact with people, you will do better in interviews. If you practice, your cricket will get better. You may also know that you cannot become Tendulkar, yet. But you can get to the next level. Striving for that next level is important.

Nature designed with a random set of genes and circumstances in which we were born. To be happy, we have to accept it and make the most of nature's design. Are you? Goals will help you do that.

I must add, don't just have career or academic goals. Set goals to give you a balanced, successful life. I use the word balanced before successful. Balanced means ensuring your health, relationships, mental peace are all in good order.

There is no point of getting a promotion on the day of your breakup. There is no fun in driving a car if your back hurts. Shopping is not enjoyable if your mind is full of tensions.

You must have read some quotes - Life is a tough race, it is a marathon or whatever. No, from what I have seen so far, life is one of those races in nursery school. Where you have to run with a marble in a spoon kept in your mouth. If the marble falls, there is no point coming first. Same with life, where health and relationships are the marble. Your striving is only worth it if there is harmony in your life. Else, you may achieve the success, but this spark, this feeling of being excited and alive, will start to die.

One last thing about nurturing the spark - don't take life seriously. One of my yoga teachers used to make students laugh during classes. One student asked him if these jokes would take away something from the yoga practice. The teacher said - don't be serious, be sincere. This quote has defined my work ever since. Whether its my writing, my job, my relationships or any of my goals. I get thousands of opinions on my writing everyday. There is heaps of praise, there is intense criticism. If I take it all seriously, how will I write? Or rather, how will I live? Life is not to be taken seriously, as we are really temporary here. We are like a pre-paid card with limited validity. If we are lucky, we may last another 50 years. And 50 years is just 2,500 weekends. Do we really need to get so worked up? It's ok, bunk a few classes, goof up a few interviews, fall in love. We are people, not programmed devices.

I've told you three things - reasonable goals, balance and not taking it too seriously that will nurture the spark. However, there are four storms in life that will threaten to completely put out the flame. These must be guarded against. These are disappointment, frustration, unfairness and loneliness of purpose.

Disappointment will come when your effort does not give you the expected return. If things don't go as planned or if you face failure. Failure is extremely difficult to handle, but those that do come out stronger. What did this failure teach me? is the question you will need to ask. You will feel miserable. You will want to quit, like I wanted to when nine publishers rejected my first book. Some IITians kill themselves over low grades – how silly is that? But that is how much failure can hurt you.

But it's life. If challenges could always be overcome, they would cease to be a challenge. And remember - if you are failing at something, that means you are at your limit or potential. And that's where you want to be.

Disappointment's cousin is frustration, the second storm. Have you ever been frustrated? It happens when things are stuck. This is especially relevant in India. From traffic jams to getting that job you deserve, sometimes things take so long that you don't know if you chose the right goal. After books, I set the goal of writing for Bollywood, as I thought they needed writers. I am called extremely lucky, but it took me five years to get close to a release.

Frustration saps excitement, and turns your initial energy into something negative, making you a bitter person. How did I deal with it? A realistic assessment of the time involved – movies take a long time to make even though they are watched quickly, seeking a certain enjoyment in the process rather than the end result – at least I was learning how to write scripts , having a side plan – I had my third book to write and even something as simple as pleasurable distractions in your life - friends, food, travel can help you overcome it. Remember, nothing is to be taken seriously. Frustration is a sign somewhere, you took it too seriously.

Unfairness - this is hardest to deal with, but unfortunately that is how our country works. People with connections, rich dads, beautiful faces, pedigree find it easier to make it – not just in Bollywood, but everywhere. And sometimes it is just plain luck. There are so few opportunities in India, so many stars need to be aligned for you to make it happen. Merit and hard work is not always linked to achievement in the short term, but the long term correlation is high, and ultimately things do work out. But realize, there will be some people luckier than you.

In fact, to have an opportunity to go to college and understand this speech in English means you are pretty darn lucky by Indian standards. Let's be grateful for what we have and get the strength to accept what we don't. I have so much love from my readers that other writers cannot even imagine it. However, I don't get literary praise. It's ok. I don't look like Aishwarya Rai, but I have two boys who I think are more beautiful than her. It's ok. Don't let unfairness kill your spark.

Finally, the last point that can kill your spark is isolation. As you grow older you will realize you are unique. When you are little, all kids want Ice cream and Spiderman. As you grow older to college, you still are a lot like your friends. But ten years later and you realize you are unique. What you want, what you believe in, what makes you feel, may be different from even the people closest to you. This can create conflict as your goals may not match with others. . And you may drop some of them. Basketball captains in college invariably stop playing basketball by the time they have their second child. They give up something that meant so much to them. They do it for their family. But in doing that, the spark dies. Never, ever make that compromise. Love yourself first, and then others.

There you go. I've told you the four thunderstorms - disappointment, frustration, unfairness and isolation. You cannot avoid them, as like the monsoon they will come into your life at regular intervals. You just need to keep the raincoat handy to not let the spark die.

I welcome you again to the most wonderful years of your life. If someone gave me the choice to go back in time, I will surely choose college. But I also hope that ten years later as well, your eyes will shine the same way as they do today. That you will Keep the Spark alive, not only through college, but through the next 2,500 weekends. And I hope not just you, but my whole country will keep that spark alive, as we really need it now more than any moment in history. And there is something cool about saying - I come from the land of a billion sparks.

Wednesday, June 18, 2008

*Yet Another gcc optimization*

Have a look at these codes.

1)
/*example1.c*/
#include
int main()
{
char *p=NULL;
printf("%s\n", p);
return 0;
}
Output is,

Segmentation fault

2) Then I slightly changed the code,
/*example2.c*/
#include
int main()
{
char *p=NULL;
printf("%s \n", p); //add a space
return 0;
}

Now, output is

(null)


Why??
Mostly we assume that our GCC compiled source code is always calling the functions we wrote in the source code. Wrong!!!

Case 1)
>gcc example1.c
>nm --undefined-only a.out
w _Jv_RegisterClasses
w __gmon_start__
U __libc_start_main@@GLIBC_2.0
U puts@@GLIBC_2.0

>

Case 2)

>gcc example2.c
>nm --undefined-only a.out
w _Jv_RegisterClasses
w __gmon_start__
U __libc_start_main@@GLIBC_2.0
U printf@@GLIBC_2.0

>

So in first case, it is calling puts which gives segmentation fault and in second case printf is called which prints “(null)” in case of NULL pointer.

The optimization is that “A printf call with a simple format string ending with '\n' is converted to a puts() call.” So in first case its converted into puts call but in second case it remains as printf.

To get the original printf call with GCC-4.0.2 you must provide -fno-builtin or
-fno-builtin-printf as command line argument

>gcc example1.c -fno-builtin -fno-builtin-printf
>nm --undefined-only a.out
w _Jv_RegisterClasses
w __gmon_start__
U __libc_start_main@@GLIBC_2.0
U printf@@GLIBC_2.0
>


Happy???

Tuesday, February 12, 2008

Performance Engineering of Stock Exchange

(Disclaimer : this is written by me, sometime back I had sent this in a mail, but many of my friends thought this is just a Forwarded mail & didn't read it :D )

Again this about a guest lecture I had attended way back on 30-10-2006 in IIT Bombay during one of my courses. It was by Dr. Rajesh Mansharamani, who is Principal Consultant & Head,
Performance Engineering Research Centre, TRDDC (Tata Research, Design
and Development Centre).

(Disclaimer: all the information in this post was written according data on 30th oct. 2006. Specifically many numbers may be outdated now. Also I may have mistakenly written something while recalling the lecture.)
First half part of the post is very small intro. to the domain knowledge & then the technical stuff follows.

Title of the talk was Performance Engineering of Stock Exchange of India:
10,000 orders/day to 10,000 orders/second.

This lecture was really nice one. Speaker talked about the the practical application. He explained about the evolution of one of the very vital Software(actually this can be a very narrow term. Rather I can say vital Automated System or something...as it included evrything from s/w, processor, in-memory database, to network issues & many more....).
It was about Bombay Stock Exchange i.e. BSE's software system.

Its obivously very crucial....as BSE is world's 3rd largest Capital Market & Largest Stock Market....(can't recall exact Commerce terms..)
turnover in BSE is Rs. 8000 crore/day. means Rs. 24crore / minute... so now u can imagin how important is the s/w running for such system..

He started of with introduction to Stock Market terms..
Buy Order Book - contains orders from the buyers..entries are no. of shares they want to buy & MAXIMUM price at which they want o buy.
Sell Order Book - contains offers from sellers..entries - no. of shares offered & price
(I think this was in the case of normal stocks... after this comes Concept of DERIVATIVES which is different from this )

Derivatives - its basically a estimation of future value & signing a contract in advance..
means lets say I am (a buyer/customer) signing a FUTURE CONTRACT for buying 100 shares of a COMPANY "X" in JAN 2007 with price Rs 200 per share....
these derivatives are declared by BSE 2-3 months in advance..so any one interested in buying can then register for this, by paying some amount (like token amount which we pay while renting/ buying home).. e.g. Rs. 1500 for 100 share.
Suppose in JAN 2007 - share price goes to Rs. 300, but as my Contract signed for Rs. 200, i will get it at Rs 200. So my profit is Rs. 100 per share.. AND if share price goes down to Rs. 150, so now as buyer I am getting loss of Rs. 50..so instead of this I just cancel buying this stock. PENAULTY here is the TOKEN amount... I will not get that back...
(here many interesting financial concepts are related....will talk on this more after some time, I wanna get more info. abt that..)

Again there are some more terms about Derivatives..
Call, put ~ similar to our http calls...
option - do u wanna keep this deal or wanna sell to someone...
holder , writer - its something about ownership of share..

Strike Value - value at the time of contract...
Spot Value - peak value..

That's the end of financial topics for a while.. People might have got bored by reading all this. Thank you for your patiences. ------------------------------

when BSE started of with s/w in 1994, there expectation was 10,000 trades per day & 50,000 transaction / day (it includes inquiry etc..)
& now they handle3 million trade/day.. 600 transactions per second..
see how much big these figures are.. & all these affect flow of MONEY....so S/W MUST (remember MUST) work correctly , efficiently..

They are using Stratus Fault Tolerant CPU (they were using 2 CPUs at that time, now using 6 CPUs...they divided them in pair of 2.. each pair is doing some specilized task such as STOCK , CAPITAL or....something.. )

They use Appletree in-memory database. They DONT use HARD disks to store the transaction data. As a day's transaction in all don't take much space as such.

Now hav a look at the problems BSE had with this s/w...And how they evolved...
PROBLEM1 - in 1994-97 evrything was going smoothly..so nobody really cared about s/w maintence or future growth.
in 1997 the s/w started degrading with QUEUE sizes at individual modules in the system increased upto 30,000! that means people were getting very high RESPONE TIME (generally which we can see nowadays also in cyber cafes in small cities)

( & in those days memory size supported were 32k. so Project Managers were really praying that this many people should not come at a time.)
SOLUTION 1 - they found one module which was sending reports to some Managers (dont recall who they are exactly, but may of BSE and related authorities.) AND INTERESTING THING was - It was SENDING REPORT for EVERY TRANSACTION.
so sending that report involved WRITING SOME INDEX IN THE DISK & doing some PROCESSING etc. so thing was 1% (manager ) activities were delaying 99 % (customer) activities.
So DECISION they took was instead of sending frequent reports, send a daily report at the end of the DAY.This helped in REDUCING that WRITING 8 indices to 4 indices (don't ask me why this is so ? it was reporting activity's internal task)

so this solution really worked. Performance increased DRASTICALLY. & then people waked up and decided to work for future needs.
(MORALE : while developing S/W, understand importance of BUSINESS NEEDS...& try to prioritize them..)

while designing ARCHITECTURE they followed approach - dont try for efficient algo.s in CODING only...instead KEEP THE GOAL of PERFORMANCE from the DESIGN itself....while ANALYSING BUSINESS NEEDS, while DESIGNING ARCHITECTURE TAKE CARE of PERFORMANCE ENGG....

BENCHMARKING PROCESS -
while testing the product, they used to start testing in the night at 10.30 upto 5 am...they were trying to simulate the WHOLE DAY...
(BSE scenario ---> it starts at 9/10 am closes 4/5 pm... & LOAD is HIGH in starting & ending half hours... for remaining day its less)
so instead of wasting so much time in simulating whole day, they decided to manage for only PEAK VALUES of LOAD....this implicitly takes care of low load which is present throughout the day...
(MORALE: while doing some work, there exists shortcuts which DO NOT involve RISK at all..!)

BROADCAST -
what we generally see on TV - share prices are shown continously throughout the day.
so BSE provides these agencies with values of stock.
previously they used to send all the stock prices at rate 10 or 100 /sec. if we take simple example of human eye which have capability of 10 images /sec , then how somebody can see 10s of prices getting updates per second ?
So BSE decreased rate of broadcast. all they noticed that - top 30 companies are involved 80% of TOTAL TRANSACTIONS, so send prices of these IMPORTANT stocks & frequently value changing STOCKs more frequntly than other companies.
so here came PRIORITIZED treatment to COMPANY data broadcast..


PROBLEM 2 - LEGACY S/W MAINTENCE...
they got troubled by some STRANGE thing because of which RESPONSE time significantly increased . so transactions failed..
so they tried to find out the cause of this slow speed .
& what was it ? GUESS. hint : it was a unnecssory overhead.

SOLUTION 2 - DEBUG STATEMENTS which were getting printed on console, (we PROGRAMMERs have habit of DEBUGGING by PRINT stmt.)
so becuase of such carelessness, there was big loss in share market.
(MORALE - ALWAYS check out for such small-small things which may cause BIG harm to you in future.)

with advent of new techonlogies they moved to MULTITHREADING at user level..

PROBLEM 3 - for sending data on networks the existing (at that it was something 16/ 64kbps... ) n/w was not so adaquate. but how to increase the capacity ? as BSE will not provide money for this.& no customers as well will upgrade their lines by investing some money.

SOLUTION - use some compression.
some guy from Pune developed compression technique customized to this application. but soon it failed, as packet structure always get changed drastically.
so instead they now went for Lempel Ziv Oberhumer (LZO) compression technique.

PROBLEM4 - PROBLEM 3 did not end ---
they compressed the packets. but they were sending it over TDMA.
so actual goal should also be to minimize the no. of TDMA frames used to send these packets.
now problem is u hav to fit these packets with heterogeneous sizes in frames, so as to minimize no. of frames.

SOLUTION 4 - its BEAN - PACKING problem which NP COMPLETE (we cant find answer in polynomial time...) so choose one optimal solution - their heuristic was Largest packet first..


PIPELINING - thoughput depends only on BOTTLENECK node / module in pipe...
so they tried to FINE TUNE BOTTLENECK modules...

NOW from hereafter comes much more TECHNICAL rather MATHEMATICAL stuff...
for the queues in the systems they came up with equations for response time etc.. (like what we did in that subject..)
herein its totally about queues & their performance. skipping that mathematical part.

-------------------------------------------------------------

They divided database on no. of SERVERS for parallel work...
only limitation becuase of this was ---> while calling for derivative contract, ppl can't have stocks from 2 different servers in one CONTRACT; but this doesn't matter, as BSE itself announces CONTRACTS.

There are some dedicated lines between BSE & stock brokers.(there is whole business story behind this as well..)
Now their goal is to support 10,000 transcations per second in 2015..
so they did SHANGHAI stock market case study.
its supporting big market. Architecture of that is really superb !
data structures that can be used in each of the submodules according to computing need -
heap - but can't find top 3/5 at a time.
binary search trees.
AVL / RED BLACK tree
message queues - for messaging between modules, uses FLUSH MODEL... rate of flush to disk... (like "sync" command)

PROBLEM 5 - HETEROGENIETY
as broker firms are using some third party s/w. (& also usual examples heterogeneity(nw/ etc.) that's always present in this computer world.)
they faced a problem when some firm always used to get TRANSACTION FAILED error..
generally when N/W connection breaks down..when it comes up, client must establish connection again.
if not then also Customer request for some transaction will be taken, but while giving response back to customer the SYSTEM doesn't know the customer - id BECAUSE there is no connection established between them
SOLUTION - third party s/w vendors didn't considered the case when n/w can break down. they must Re-Establish connection.
(MORALE - while interacting with others, there is always possibility of mismatch...be aware! )

PROBLEM 6 - COMMUNICATION problem for S/W ppl
when reporting some problem to non-techie / non-s/w person , it should be said in way that is understandable to that person.
IF S/W ppl SAY - We want to reduce 4 out of 8 index in database to improve performance, then who will understand ? instead tell them - if we send ONLY daily report to managers then it will help in better performance for all users.

in case of figures - 10,000 transaction per sec....etc...
tell them that we can do all the transactions of the day in 5 minutes..


so in all speaker talked about many of the practical aspects of what we learn in books....
here I skipped with mathematical part which he explained.
in all it was a great experience to have a lecture like this as a part of a course...!

thanks,
-Rahul

Saturday, February 2, 2008

Douglas Comer & lessons learned from the Internet project

Recently we had a interesting talk by Prof. Douglas Comer at IIT Bombay. We have referred some of the networking books authored by him, though there is big list of books in front of his name. I must say his books as well as his talk was very nice. You really learn many good things out of them. Prof. Comer is a professor at Purdue University and currently Vice President at Cisco research.

His talk was on "Lessons learned from the Internet project." The talk covered issues they faced while designing internet protocols. His presentation was very precise and highly informative. (you can find that at -
xys.ccert.edu.cn/reference/Comer_Internet_Lessons_Talk.pdf )

Some of the interesting things he mentioned -
1) how research idea is assessed from commercial perspective. As the students of networking we learn a lot about routing, shortest path algorithms. But in reality, "Internet routing is not based on shortest paths." :)
Its all about the Money.

2) debate with telephone companies regarding connectionless vs. connection oriented infrastructure design. (finally Internet guys won ! )

3) one good lesson for people starting with new project idea - start with specific goal, rather than trying for solution for all possible cases. In short - "Don't try to save the world" :)

He also talked about some open research problems -
1) understanding the behavior of traffic in the internet.
2) routing issues. Still there are problems of route flapping & black holes.

Good thing about his talk was his technical elegance, and his sense of humor. I didn't sleep in a 1hour lecture, that too at 2 in the afternoon :)

Also please go through some of the essays on his homepage. I read through this funny essay regarding OSI design - which depicts the 7 dwarf types from which 7 layers of OSI came up :D

regards,
-Rahooooool

Friday, December 21, 2007

chmod "+s"

Purpose of this blog is to just break this loooog Sannata mode. Many people might be aware of this, but for those who dont know can read.

We all know the basic file access permissions on linux. Access permissions can be set per file for owner, group and others on the basis of read (r), write (w) and execute permissions (x).

Linux processes run under a user-ID. The effective user-ID is the one that determines the access to files. So we can set user or group ID on execution using chmod command with 's' bit


>chmod 4755 suidtest
or
>chmod u+s suidtest


This causes the file to be executed under the user-ID of the user that owns the file rather than the user that executes the file. Same thing is applicable for group ID.


As you can see this is a very powerful feature especially if root owns the file with s-bit set. Any user can then do things that normally only root can do. A few words on security. When you write a SUID program then you must make sure that it can only be used for the purpose that you intended it to be used. Always set the path to a hard-coded value. Never rely on environment variables or functions that use environment variables. Never trust user input (config files, command line arguments....). Check user input byte for byte and compare it with values that you consider valid.


Here is the sample program with output.

1)

#!/bin/sh

#suid.sh: Print user information

echo " effective user-ID:"

id -un

echo " real user-ID:"

id -unr

echo " group ID:"

id -gn


2)

/*suid.c*/

#include

#include

int main(){

/*secure SUID programs MUST

*not trust any user input or environment variable!! */


char *env[]={"PATH=/bin:/usr/bin",NULL};

char prog[]="/tmp/suid.sh";

if (access(prog,X_OK)){

fprintf(stderr,"ERROR: %s not executable\n",prog);

exit(1);

}

printf("running now %s ...\n",prog);

setreuid(geteuid(),geteuid());

execle(prog,(const char*)NULL,env);

perror("suid");


return(1);

}


Now, with root user do

'gcc -Wall suid.c'

'chmod u+s a.out'


Here is the output

vinitd@orchid:/tmp> whoami

vinitd

vinitd@orchid:/tmp> ./a.out

running now /tmp/suid.sh ...

effective user-ID:

root

real user-ID:

root

group ID:

users

vinitd@orchid:/tmp>


--Note: It is possible to switch off Suid when mounting a file system. If you find the option "nosuid" in /etc/fstab then this Suid feature is switched off. For details have a look at the man-page of mount.

Friday, November 23, 2007

Nothing Technical About IT !!


As the caption suggests, there's nothing technical about this post. But could not help but put this up over here!

Received the image alongside in a mail, and loved it for no apparent reason ;-) !!

MS enthusiasts (Lele chill, I do not mean your MS), please excuse me, and do not hold this against me :-) !!






P.S.: Why has this blogspace gone into sannaTa mode ?