Down with P-values!

One of the few bloggers I follow just posted on the decision by Basic and Applied Social Psychology to prohibit the use of P-values and significance testing in articles submitted to their journal. While this isn’t necessarily earth-shattering (note that the editorial announcing the new policy has been viewed 50k times while the issue’s most popular actual article had 65 views as of this writing), it is definitely an interesting change in how science is practiced.

BASP‘s alternative is to embrace descriptive statistics, emphasizing effect sizes and distributions. This might enrich the literature for phenomena with meaningful effects. I think we’ve all heard Randy bring up the importance of large effect sizes. What will be interesting, however, is what happens if “large” becomes a qualitative term. At the moment, blind adherence to P-values serves a gatekeeping function: unless the effect size is large relative to the variability of the property being measured, it is not considered “significant”. Removing that automated check shifts the burden downstream, ideally to the reviewers, but likely to the readers, to determine whether a study provides sufficient evidence to be interpreted qualitatively.

To keep this from getting too long, I would direct readers to an excellent piece by Regina Nuzzo that analyzes both the genesis and failings of the P-value. It will be interesting to see how this evolves.