Grading political comments

« previous post | next post »

A dozen people have sent me links to this blog post — "Presidential Debate Grammar Power Rankings", Grammarly Blog 10/6/2015 — or to various commentaries on it, e.g. Justin Moyer, "Trump supporters have the worst Facebook grammar, study finds", WaPo 10/7/2015; Emily Atkin, "New Analysis Ranks Presidential Candidates By Their Supporters’ Grammar", ThinkProgress 10/6/2015; Paul Singer, "Democrats crush Republicans in grammar; Chafee on top", USA Today; "Trump First in the Polls, But His Supporters Are Last in Grammar", Yahoo! Health 10/7/2015; etc.

I don't have time this afternoon to write anything more about this, so feel free to talk among yourselves…



  1. rcalmy said,

    October 7, 2015 @ 3:47 pm

    Hoo boy.

    I haven't gone back to confirm this, but I feel like the majority of the time I have seen Grammarly mentioned on any of the language blogs I follow, it is a lament about how poor their grammar advice is.

    That's on top of the fact that I find the concept of the ranking depressingly irrelevant and elitist.

  2. ThomasH said,

    October 7, 2015 @ 3:53 pm

    It could be useful information for candidates to know which population subgroups to target. No candidate targets the "elite" but some go to greater lengths to eschew "elites" than others. I assume Mr Trump speaks differently when he's negotiating deals with Mayors and Governors than on the stump.

  3. david donnell said,

    October 7, 2015 @ 3:57 pm

    My favorite defense of Republicans from the Think Progress comments: Dan Orlovsky wrote, "What did we learn? Republicans are less likely to proofread. Probably stems from years of not being pretentious."

  4. J. W. Brewer said,

    October 7, 2015 @ 4:15 pm

    Their methodology sounds like they're at least trying in terms of having an objective notion of what counts as an "error" that doesn't just dock points for slang or informality of register. But obviously only a tiny and unrepresentative subset of any candidate's supporters are going to post neutral-to-positive comments at least 15 words long on the campaign's official facebook page, and there seems no a priori reason to believe that the way in which or degree to which the sample is unrepresentative is going to be consistent from candidate to candidate. E.g. even if you could somehow figure out that the median actual Trump supporter is X% more cogent and Y% less bellicose than the median Trump facebook commenter, there's no reason to think you could use the same adjustments to the online expression of the median Clinton facebook commenter to accurately reverse- engineer the characteristics of the median Clinton supporter.

    It also strikes me as maybe a bit odd to use the same number (even if randomly-selected) of comments for analysis for each candidate, given that the total universe of Trump-supporting comments one could choose from must be several orders of magnitude higher than the total universe of e.g. Pataki-supporting candidates (and the same for e.g. Sanders-supporting comments versus Chafee-supporting comments), although I will confess I'm not enough of a stats guy to know if that's likely in context to distort the data and if so in what direction.

  5. J. W. Brewer said,

    October 7, 2015 @ 4:22 pm

    One other point: it might be useful (if feasible) to see a distribution, not just averages. I'm pretty sure the standard claim by those who study the demographics of political affiliation is that in 21st century America the median level of formal education of Republican-leaning voters and Democratic-leaning voters is quite similar, but the distributions are markedly different. Specifically, the Democratic constituencies are more of a "barbell" in that the extremes (voters who never finished high school and voters with graduate degrees) lean more Democratic while the groups in the middle (at least a high school diploma but no more than a bachelor's degree) lean more Republican. Obviously, it is hazardous to treat level of formal education as a proxy for e.g. intelligence or capacity for pragmatic judgment rather than social class, career preference etc, but the same is obviously true of treating rate-of-errors-in-online-prose as a proxy for anything of great significance.

  6. Mai Kuha said,

    October 7, 2015 @ 8:16 pm

    Nic Subtirelu just blogged insightfully about this.

  7. Lazar said,

    October 7, 2015 @ 11:48 pm

    One issue might be that there have been multiple Republican debates so far, but no Democratic ones: maybe debates cause people to write more heated, mistake-ridden comments.

  8. CD said,

    October 8, 2015 @ 12:06 am

    It's despair-inducing that "grammar" apparently means "anything that can be called a mistake in your writing."

  9. Jerry Friedman said,

    October 8, 2015 @ 10:08 am

    CD: Not to add to your despair, but that's been going on for a long time. The OED is rather prescriptive about it:

    "As above defined, grammar is a body of statements of fact—a ‘science’; but a large portion of it may be viewed as consisting of rules for practice, and so as forming an ‘art’. The old-fashioned definition of grammar as ‘the art of speaking and writing a language correctly’ is from the modern point of view in one respect too narrow, because it applies only to a portion of this branch of study; in another respect, it is too wide, and was so even from the older point of view, because many questions of ‘correctness’ in language were recognized as outside the province of grammar: e.g. the use of a word in a wrong sense, or a bad pronunciation or spelling, would not have been called a grammatical mistake. At the same time, it was and is customary, on grounds of convenience, for books professedly treating of grammar to include more or less information on points not strictly belonging to the subject.

    "Until a not very distant date, Grammar was divided by English writers (following the precedent of Latin grammarians) into Orthography, Etymology, Syntax, and Prosody, to which Orthoëpy was added by some authors. All these terms (except Syntax) were used more or less inaccurately (see the several words). The division now usual is that into Phonology, treating of the sounds used in the language, Accidence, of the inflexional forms or equivalent combinations, and Syntax, of the structure of sentences; the branch of grammar dealing with the functions of the alphabetic letters is usually treated along with the phonology."

    It also says, "This entry has not yet been fully updated (first published 1900)."

    I looked up "orthography" and didn't see anything about inaccurate uses.

  10. popegrutch said,

    October 8, 2015 @ 11:57 am

    Snopes has now posted a takedown on the methodology: "By their own admission, the data was collected and interpreted by employees of Grammarly (not data analysts or dedicated researchers) with the ultimate goal of promoting their grammar-checking app."

  11. Theo Vosse said,

    October 8, 2015 @ 12:55 pm

    There's not much on the methodology, but the examples show that they count missing quotation marks as an error. Suppose that's the only thing you measure. Then the group that cites most often will come out worst (c.p.). For other errors the same holds. So a difference between the groups is not necessarily representative of their grammatical competence.

  12. Lai Ka Yau said,

    October 8, 2015 @ 7:36 pm

    @Theo Vosse: Although representative of their grammatical performance…

  13. Lane said,

    October 9, 2015 @ 11:12 am

    When I tested Grammarly a few years ago it was close to unerringly wrong.

  14. joglio said,

    October 13, 2015 @ 2:13 pm

    Are Trump supporters less educated than, say, Bernie supporters? I'd bet on it.

    Doesn't change the fact that bragging about the average intelligence of your voting base is foolish and elitist in the worst way.

RSS feed for comments on this post