Computational linguistics in three acts
« previous post | next post »
Towards the end of April, I gave a short presentation at the Penn Science Café in a session on "The past, present, and future of AI". I mentioned this in a comment on an xkcd cartoon in "Machine Learning", 5/17/2017, where I also reproduced my opening Science Café slide:
Over the weekend, Fernando Pereira posted a wonderful account of these three eras, with some thoughts about the nature of the underlying problems and possible directions for the future: "A (computational) linguistic farce in three acts", Earning My Turns 6/10/2017.
MattF said,
June 12, 2017 @ 10:24 am
I remember, back in the late '60s, I attended a lecture on AI. The speaker strode up to the front of the room, drew a big 'N' on the blackboard, then drew a box around the 'N'. And said 'We start with Nature'.
Those were the good old days.
Mara K said,
June 12, 2017 @ 11:47 am
Can the computational linguistic research of the 2020s please please start with the linguistic rules humans already know? Sincerely, a linguist with computational aspirations
J.W. Brewer said,
June 12, 2017 @ 1:39 pm
I learned a bit about the not-very-impressive-seeming state of the art, as it then was, in an intro AI-for-non-CS-majors class I took back in, let's see, probably during the '85-'86 academic year. It was in hindsight kind of a review of how little Act I had actually delivered after several decades of lavish funding during which the big breakthrough was always just a few years away from whenever a given grant proposal was being written but never got much closer. Perhaps the instructor plus various grad students etc floating around campus were already aware that Act II was being prepared backstage but it was not yet ready to be included in an intro undergrad syllabus? If I wanted to reacquaint myself with the state of the art, is this a good time, or should I wait another few years so that Act III is fully up and running and Act II of only historical interest?
J.W. Brewer said,
June 12, 2017 @ 2:06 pm
And of course Dr. Pereira's narrative suggests that while Act I was still playing itself out in the university world, researchers in non-university settings like Bell Labs (including the young Mark Liberman) were doing a disproportionate share of the early work for Act II. Dr. Pereira's "northern barbarians" narrative likewise suggests that perhaps the early developers of Act III were working in (and perhaps thus empowered to chart a new course by?) different institutional settings than those that had become dominated by the stars of Act II, but his analogies are (for good or for ill) written for readers who already know the story better than I do and thus remain a bit opaque to me.
Bill Benzon said,
June 12, 2017 @ 2:35 pm
In the discussion after Pereira's post Peter Norvig posts a useful link, A Pendulum Swung Too Far (2007), about the rationalism/empiricism "pendulum". I'm in the process of reading it and recommend it.
Joe Brewer: I'm further from this business than others who read, post, and comment here, but I wouldn't hold my breath. It's not going to be solved within a decade – whatever it is and whatever solved might mean – but I'm sure interesting things are coming down the pike. If you go poking around now I'm sure you'll find some interesting stuff. Keep in mind, though, that the "neurons" in these neural network machines resemble real neurons about as much as a smiley face emoji resembles the Mona Lisa.
Andrew said,
June 12, 2017 @ 2:57 pm
The last one is still applied statistics — everything neural nets do can be explained in statistical terms. And in some sense, I have to think that "intelligence is applied statistics" comes very close to being true by definition.
Rubrick said,
June 12, 2017 @ 4:13 pm
@Andrew: "The universe is applied statistics" is also probably spot on, but it may not be enlightening. (Then again, it may!)
peter said,
June 12, 2017 @ 5:13 pm
@Andrew:
If intelligence is merely applied statistics, why do AI people spend so much time and effort on Search algorithms? Logic? Argumentation? Communications languages? Multi-agent systems? AI Planning? Game theoretic reasoning? Mechanism design? Qualitative decision-making?
Dan Lufkin said,
June 12, 2017 @ 10:03 pm
I wonder whether phenomena such as the Dunning-Kruger effect (confidence in one's opinion increases with ignorance of the subject) will turn out to be emergent properties of intelligence, both natural and artificial.
Idran said,
June 13, 2017 @ 11:21 am
@Peter: Because from a broad enough perspective, those are also all applied statistics? I think that's what Andrew was getting at.
peterv said,
June 13, 2017 @ 11:44 pm
@Idran: That is precisely the claim I am contesting. I cannot see, for example, how the formal representation of permissions and obligations by means of deontic modal logic, a topic of great importance currently for the design of autonomous vehicles, can be construed as being an application of statistics.