AI counting again
« previous post | next post »
Following up on "AIs on Rs in 'strawberry'" (8/24/2024), "Can Google AI count?" (9/21/2024), "The 'Letter Equity Task Force'" (12/5/2024), etc., I thought I'd try some of the great new AI systems accessible online. The conclusion: they still can't count, though they can do lots of other clever things.
Google Gemini 2.0 Experimental Advanced still thinks there are 2 Rs in "strawberry":
But Google Gemini 2.0 Flash Experimental gets it right!
Hooray! And it knows (and can tell me) the first sentence of the Declaration of Independence. But it confidently miscounts the number of instances of "e" in that sentence:
(Actually there are 50…)
Grok 2 also does the right thing with "strawberry", and even shows its work:
And it also knows the first sentence of the Declaration of Independence, and is happy to show it to me. But its count of the instances of "e" in that sentence is spectacularly wrong:
To support my contention that there are actually 50 instances of "e" in that sentence, I wrote a trivial 6-line program that not only counts them, but outputs the text with incremented superscript boldface numbers after each "e":
Whe1n in the2 Course3 of human e4ve5nts, it be6come7s ne8ce9ssary for one10 pe11ople12 to dissolve13 the14 political bands which have15 conne16cte17d the18m with anothe19r, and to assume20 among the21 powe22rs of the23 e24arth, the25 se26parate27 and e28qual station to which the29 Laws of Nature30 and of Nature31's God e32ntitle33 the34m, a de35ce36nt re37spe38ct to the39 opinions of mankind re40quire41s that the42y should de43clare44 the45 cause46s which impe47l the48m to the49 se50paration.
I also asked Grok 2 to "plot sin(3x) between x=pi/2 and x=3pi/2". Here's the result, which has a lot of fancy algebra talk, but…
For your reference, here's what I was hoping to see:
David Morris said,
December 20, 2024 @ 5:14 pm
This comment more naturally goes with the first counting-the-rs-in-strawberry post, but the comment button isn't showing there at the moment. I am writing music for a poem with the word 'single' in it. I had to hyphenate it under two notes. I decided on 'sin-gle' because 'si-ngle' and 'sing-le' are obviously wrong. Google's AI Overview's answer: No, you should not hyphenate the word "single" because it is a one-syllable word.
(Possibly it's a 'single'-syllable word!)
Michael Vnuk said,
December 20, 2024 @ 5:23 pm
It's odd that Grok 2 gave the first word of the first sentence of the Declaration of Independence as "Wen" rather than "When".
A Google search for "When in the Course of human events, it becomes necessary" gives me about 52000 hits, while "Wen in the Course of human events, it becomes necessary" gives me only 3 hits.
I understand why many typos occur: humans type. But machines are copying electronically, so typos should not occur.
Mark Liberman said,
December 20, 2024 @ 5:27 pm
@Michael Vnuk: "It's odd that Grok 2 gave the first word of the first sentence of the Declaration of Independence as "Wen" rather than "When"."
Wow, I didn't even notice that. One more piece of evidence that I'm the world's worst proofreader.
MattF said,
December 20, 2024 @ 8:28 pm
@Michael Vnuk:
Given the misspelling of ‘When’, I wonder how many ‘h’s the software would find in that sentence.