Wall Street saw a brutal sell-off today, with the major market indexes - Dow Jones, S&P 500 and NASDAQ - dropping respectively 2.64% (367 points), 2.56% (39 points) and 2.65% (74 points). The slide reflects, among other things, a continued concern about credit and housing issues, rising oil prices, falling dollar value, and uncertainly on Fed’s next move.

This is still only a scratch on the surface compared to what happened this day 20 years ago. In what is now termed the infamous Black Monday, US stock market recorded the largest one-day crash in its history on October 19, 1987. That fateful day, Dow Jones crashed 23%, and S&P 500 dropped 21%.

Dow would have had to lose almost 3200 points today to match a 23% loss. That is a huge number for a single day drop. Can it happen again? Theoretically yes, but unlikely. There is still controversy on what exactly went wrong on black Monday. But there are several safety measures in place today to prevent a similar catastrophe. This includes improved monitoring of stock trading, and also computerized access that allows investors to execute transactions by themselves.

I generally avoid reading predictions of the government’s move every time the market hits a rough patch, as is happening now. Guessing the mind of Fed chief Ben Bernanke - if he will cut the interest rate again to soothe investor sentiment - has become as much a suspense as predicting the stock market.

Donald Luskin writes that with S&P 500 index only <6% below its all-time high (a 10% drop counts as a mere market “correction”), the market is still bullish, and this is not the time yet for government intervention. He also reminds us of Bernanke’s 2002 speech, where he said that the interest rate is a tool for regulating economic growth in the long term, not for giving short-term relief to the market every time it slips.

Many do not share Luskin’s optimism on market conditions today, and they do not need to look up charts. The current credit crunch and housing crisis have stressed the investing environment, and it feels particularly hard coming right after the boom over the preceding year.

But this is not a recession yet, and until the overall economy is threatened, the Fed has no reason to step in. I agree with the conservative sentiment that the investors must own up responsibility for their action, and it is not the government’s (and hence the taxpayer’s) job to bail them out. But it is the government’s job to ensure the investors are not duped by reckless actions of private enterprises, as the subprime mortgage mess today shows.

Since July 19 this year, when Dow Jones Index rose to a record 14,000 points, it is down to 12,845 at yesterday’s closing (even falling below 12,600 at one point). That is almost 10% drop in less than a month, nearing the official definition of a market correction. Is this a precursor to a bear market? Here are CNN Money’s 5 ways to know:

1. Rising oil prices. High energy cost triggers fear of inflation, and Federal Bank typically counters by raising short-term interest rates, which deflates stock prices. Oil price has soared 20% so far this year.
2. Rising Treasury yields. As the yield of 10-year Treasury notes rises sharply, investors often turn away from stocks in favor of the security of notes and bonds. The yield has not changed much this year yet.
3. Falling number of growing stocks. If the broad indexes are being pulled up by only a few large company stocks, while others are falling around them like ninepins, that cannot be a good sign. Over the last few weeks, almost four times as many stocks hit 52-week lows as highs.
4. Falling consumer spending. If we buy less, it stands to reason that the product-oriented industry will suffer. Retail sales dropped almost 1% in June, and the continuing housing market crunch can force consumers tighten their money-belt even further.
5. Falling corporate earning growth. If corporate earning slows, the market will slow down as well. Low productivity growth, inflation fears, high long-term interest - all of them dissuade companies from borrowing money. We won’t know till 2007 earning reports are out.

In short, we do not know. Even though this type of analysis is better than those based on charts alone (the so called “trend analysis”), predicting the future based on past events is still notoriously difficult, if not impossible. The only certainty about the market is in its past, a classic example of “hindsight being the perfect science”.

If we look at the data for a broad stock market index, such as the Dow Jones Industrial Average or S&P 500 Index, three things jump out of the page. The first one is good news - there is a strong upward trend in market movement over the entire recorded history. This means that holding onto a diversified portfolio should fetch significant gain over long term - a popular retirement strategy for many who still have several years left before hanging up their boots.

The second feature is the big ups and downs - the so called “boom-bust” - that tend to happen every few years. Two recent examples are the technology bubble of 1980’s and the (in)famous “dotcom” bubble of 90’s, both of which burst with a loud bang catching many investors off-guard (see the picture here for the S&P Index). For those with a relatively short term investment horizon, getting hit by a recession and losing much of the asset within a few days can hurt.

The third (less explored) feature is the day-to-day fluctuations, the sawtooth pattern of little ups and downs. A great thing about them is the market’s surprising resilience - a fall is almost always followed by a rise, as if being constantly pulled to the underlying trend by an unseen force. To see an example of this daily gain/loss pattern, let us again look at the S&P data (even though Dow Jones Index is the favored child of Wall Street, many prefer S&P 500 because of its broader representation of the market).

The graph here shows the difference of today’s and yesterday’s closing values relative to yesterday’s value (in %). An interesting thing is the near symmetry about the horizontal 0-line. There are as many ups as downs, and they often appear in pairs. A striking example is the largest single-day crash of 21% registered on October 19, 1987, followed within days by the biggest (in the recorded period) one-day gain of 9%. (If we plot the difference between today’s closing value and the average value of the entire preceding month, the result still remains largely the same.)

Another way to look at this alternating sequence of fluctuations is by plotting the distribution of the gain/loss values (a “histogram” plot). The picture here shows this histogram, which is tightly bunched around zero. The large peak near zero indicates that most fluctuations are relatively tiny, and the rare big events lie on the outer “tails” of the distribution. The second histogram on the right shows how many times a loss (or gain) is followed by another loss (gain) on successive days. Most of the time an up occurs right after a down, and rarely does a loss (or gain) continues for as long as a week. Some people prefer using a statistical concept known as the “1-lag autocorrelation coefficient” to quantify such random fluctuations. A value of this coefficient close to zero means one cannot predict with any degree of certainty that a loss today will be followed by another loss tomorrow. For our data, this value is 0.077, which is as near zero as one can get.

To wrap up (this little self-tutorial), there are two take-home messages from this analysis. First, there is no reason to lose nerve after a crash, no matter how precipitous. The market inevitably recovers, often on the very next day, and in fact it registers strong gains over long term. Those poor souls who cleared out soon after the 1987 crash did the worst thing imaginable: selling off when the market has bottomed out. By contrast, the lucky (or prudent) few who held on through the turmoil recovered their asset (and then some) within the next decade. The second bit of wisdom is that no one can consistently time the market. It is easy to get lucky once in a while, but a next-day predictability quotient of 0.077 means that this is also a surefire way of getting burned.

Now that the stock market has gotten into another volatile phase - swinging up and down in sync with jittery investor sentiment - the dooms-day forecasters and soothsayers alike have started coming out of the woodwork. There are as many advisers warning us “sell-off time is NOW”, as others saying “the bull market is still on, even though bull run may be over”.

This is also the time when a few market timers catch their lucky break. If stock prices fluctuate without any apparent regularity, by pure chance anyone can succeed once in a while in offloading just before a crash. But market almost always recovers as rapidly as it falls. Kiplinger recently noted that a timer has to be lucky not once, but twice - to get out at a high, and also to get in at the next low. Even that kind of luck is occasionally possible, but consistent market timing would be a rare feat indeed!