yea, it's no fun... but I used to specialize in dealing with Google penalties.... so bad mistakes become my profit ;o)
It can be a serious problem and take many months to rectify. In the end one has to add plenty of value into the site and high quality links to balance out the link profile.... and ask real nice
Many times it can be as simple as folks that start a new sites and;
Add 300+ Directory links
Add 150+ Article marketing links
And given the query space, that alone over a 3 month period can tank a site....
...and on that topic I did a recent post called; the
Link Builders Guide to Historical Ranking factors
... it is one of the more important factors (along with diversity) when crafting a strong link profile.