Aggregate excess demand on wall street.

The rational investor behavior and news triggered price change assumed by the Efficient Market Hypothesis (EMH) could not explain most of asset price variances suggesting the need for an alternative theory. The Behavioral Finance Theory (BFT) advocates those economic judgments and decisions in markets are often irrational because of systematic and predictable psychological bias. However, due to the lack of measurable investment behaviors, proponents of the efficient market hypothesis argue that irrational behavior could not be reliably identified and predicted. Here we show that the price-takers behavior gauged by the normalized excess demand (NED) can be measured and the results explain most of the variances of SP500 daily returns over eight years of available data, the remaining variances are due to price-makers behavior, an influence abstracted out by the Walrasian general equilibrium theory. The interactions between behaviors of price-takers and price-makers drive market price fluctuations. For short-term prediction, we demonstrate that detected market makers' inventory positions often lead to intraday and daily market reversals. For long-term forecasting, feedback analyses of NED and SP500 data reveal signals of looming plunges and recovery processes in 2000, 2008, and 2020 market crises.