Last November, I posted about a study by Cornell psychologist Daryl Bem that purported to show that study participants displayed extrasensory perception (ESP) – specifically, the ability (within the context of the study) to accurately predict future events.
The [Bem study] has inflamed one of the longest-running debates in science. For decades, some statisticians have argued that the standard technique used to analyze data in much of social science and medicine overstates many study findings — often by a lot. As a result, these experts say, the literature is littered with positive findings that do not pan out: “effective” therapies that are no better than a placebo; slight biases that do not affect behavior; brain-imaging correlations that are meaningless.
The ‘standard technique’ referenced above is significance testing, and the (relatively) new approach common outside of social science is Bayesian probability. These are two Big Ideas in quantitative research but the NY Times article does a good job explaining them in a clear, accessible way.