"Significance", in 1885 and today
There's an ongoing argument about the interpretation of Katherine Baicker et al., "The Oregon Experiment — Effects of Medicaid on Clinical Outcomes", NEJM 5/2/2013, and one aspect of this debate has focused on the technical meaning of the word significant. Thus Kevin Drum, "A Small Rant About the Meaning of Significant vs. 'Significant'", Mother Jones 5/13/2013:
Many of the results of the Oregon study failed to meet the 95 percent standard, and I think it's wrong to describe this as showing that "Medicaid coverage generated no significant improvements in measured physical health outcomes in the first 2 years."
To be clear: it's fine for the authors of the study to describe it that way. They're writing for fellow professionals in an academic journal. But when you're writing for a lay audience, it's seriously misleading. Most lay readers will interpret "significant" in its ordinary English sense, not as a term of art used by statisticians, and therefore conclude that the study positively demonstrated that there were no results large enough to care about.
Read the rest of this entry »