From hot hands to declining effects: the risks of small numbers.

About 25 years ago, a group of researchers demonstrated that there is no such thing as the "hot hand" in professional basketball. When a player hits 5 or 7 shots in a row (or misses 10 in a row), what's at work is random variation, nothing more. However, random causes do not stop players, coaches, fans, and media from talking about and acting on "hot hands," telling stories and making choices that ultimately are based on randomness. The same phenomenon is true in medicine. Some clinical trials with small numbers of events yielded positive findings, which in turn led clinicians, academics, and government officials to talk, telling stories and sometimes making choices that were later shown to be based on randomness. I provide some cardiovascular examples, such as the use of angiotensin receptor blockers for chronic heart failure, nesiritide for acute heart failure, and cytochrome P-450 (CYP) 2C19 genotyping for the acute coronary syndromes. I also review the more general "decline effect," by which drugs appear to yield a lower effect size over time. The decline effect is due at least in part to over interpretation of small studies, which are more likely to be noticed because of publication bias. As funders of research, we at the National Heart, Lung, and Blood Institute seek to support projects that will yield robust, credible evidence that will affect practice and policy in the right way. We must be alert to the risks of small numbers.