Researcher Degrees of Freedom Manufacture False Positives

Researchers commonly make on-the-fly analytical decisions—checking significance levels during data collection and stopping when significant, not pre-specifying sample size or statistical tests, and discarding outliers—that virtually guarantee positive results; Joe Simmons demonstrated these 'researcher degrees of freedom' could 'show' that listening to the Beatles' 'When I'm Sixty-Four' physically made people younger.