Hearsay Statistics: Myths, Misinformation, and Remedies.
In the course of teaching statistics to university students over more than 30 years I have encountered many persistent myths and misinterpretations. I once believed many of these. Examples of myths include the "Damn Lies" quote, Bayes' theorem, Bayes' rule, Bayes factor, and even Bayes' portrait. Examples of misinterpretation include Bayes' probability model, pseudoreplication, and randomization as a surrogate for Type I error. Examples of mistaken belief inlude the law of small numbers, the frequentist-priorist wars, and p-values as evidence. Examples of mythical practices include rank-based non-parametric tests, statistical tests of assumptions, retrospective power analysis, multiple comparison tests, and square root and arcsin transformations. Some of these examples result in malpractice-- harm to the evidence (loss of information), harm to subjects, harm to the environment, or hidden risk to a stakeholder in a public setting. Remedies are offered for discussion.