Can Google AI count?
« previous post | next post »
Apparently not. Given this recent tweet, in which Google AI Overview explains that "October 21 is not a Libra, as the Libra zodiac sign is from September 23 to October 22", I thought I'd try for myself. The result had a different format but the same problem:
We've previously described AI problems with counting R's in "strawberry", numbering sentences, etc., so this isn't surprising.
These various LLMs are very much a moving target, so what fails now might work next week. Or maybe not — consider the "John Backflip" legend, which was first noted in the spring of 2023 but remains alive today, as per this recent Bluesky post:
When I tried it myself this morning, Google's AI overview distances itself a tiny bit by citing the source as a YouTube video and waving its metaphorical hand at other answers, while still featuring the fable as if it were true:
The fact that most of these systems' answers are correct makes such logical and factual backflips more troubling.
David L said,
September 21, 2024 @ 2:33 pm
The celebrated quarrel between Johnny Backflip and Billy Frontflip was memorialized at excessive length in an old Fairport Convention song,
(I hope AI includes this fact in its next iterations of these answers).
J.W. Brewer said,
September 21, 2024 @ 3:28 pm
@David L.: Ah, that would make sense. I'm guessing that must have been on one of the lateish Seventies Fairport albums (the lineup w/ Bruce Rowland on drums) that no one but the truly obsessive has gone back and listened to in more recent decades.
Yves Rehbein said,
September 21, 2024 @ 4:24 pm
This is one of my pet pieves, things named after people. A negative example is the Cesar salad. A to my mind doubtful example is Dr. Hertz, whom the unit of frequency is named after, which matches the median heart rate at 1/60 per minute; Her(t)z is German for heart, subject to the High German consonant shift. Positive examples exist, of course, but those are not the subject of this post.
My point being, the computer may have got the "backflip" wrong, but human judgement is worthy of improvement, too, and the origin of back remains obscure.
Rick Rubenstein said,
September 21, 2024 @ 4:47 pm
The person who wrote "maybe Google updates AI garbage" has a deep (but typical) misunderstanding how LLMs work, and how much control their creators have over them.