If we look at the data for a broad stock market index, such as the Dow Jones Industrial Average or S&P 500 Index, three things jump out of the page. The first one is good news - there is a strong upward trend in market movement over the entire recorded history. This means that holding onto a diversified portfolio should fetch significant gain over long term - a popular retirement strategy for many who still have several years left before hanging up their boots.

The second feature is the big ups and downs - the so called “boom-bust” - that tend to happen every few years. Two recent examples are the technology bubble of 1980’s and the (in)famous “dotcom” bubble of 90’s, both of which burst with a loud bang catching many investors off-guard (see the picture here for the S&P Index). For those with a relatively short term investment horizon, getting hit by a recession and losing much of the asset within a few days can hurt.

The third (less explored) feature is the day-to-day fluctuations, the sawtooth pattern of little ups and downs. A great thing about them is the market’s surprising resilience - a fall is almost always followed by a rise, as if being constantly pulled to the underlying trend by an unseen force. To see an example of this daily gain/loss pattern, let us again look at the S&P data (even though Dow Jones Index is the favored child of Wall Street, many prefer S&P 500 because of its broader representation of the market).

The graph here shows the difference of today’s and yesterday’s closing values relative to yesterday’s value (in %). An interesting thing is the near symmetry about the horizontal 0-line. There are as many ups as downs, and they often appear in pairs. A striking example is the largest single-day crash of 21% registered on October 19, 1987, followed within days by the biggest (in the recorded period) one-day gain of 9%. (If we plot the difference between today’s closing value and the average value of the entire preceding month, the result still remains largely the same.)

Another way to look at this alternating sequence of fluctuations is by plotting the distribution of the gain/loss values (a “histogram” plot). The picture here shows this histogram, which is tightly bunched around zero. The large peak near zero indicates that most fluctuations are relatively tiny, and the rare big events lie on the outer “tails” of the distribution. The second histogram on the right shows how many times a loss (or gain) is followed by another loss (gain) on successive days. Most of the time an up occurs right after a down, and rarely does a loss (or gain) continues for as long as a week. Some people prefer using a statistical concept known as the “1-lag autocorrelation coefficient” to quantify such random fluctuations. A value of this coefficient close to zero means one cannot predict with any degree of certainty that a loss today will be followed by another loss tomorrow. For our data, this value is 0.077, which is as near zero as one can get.

To wrap up (this little self-tutorial), there are two take-home messages from this analysis. First, there is no reason to lose nerve after a crash, no matter how precipitous. The market inevitably recovers, often on the very next day, and in fact it registers strong gains over long term. Those poor souls who cleared out soon after the 1987 crash did the worst thing imaginable: selling off when the market has bottomed out. By contrast, the lucky (or prudent) few who held on through the turmoil recovered their asset (and then some) within the next decade. The second bit of wisdom is that no one can consistently time the market. It is easy to get lucky once in a while, but a next-day predictability quotient of 0.077 means that this is also a surefire way of getting burned.