The syntax of texture and the texture of syntax
« previous post | next post »
W. Tecumseh Fitch, Angela D. Friederici and Peter Hagoort, Eds., "Pattern perception and computational complexity", Philosophical Transactions of the Royal Society B, July 2012:
Humans around the world are fascinated by visual and auditory patterns, and the perception of complex iterative, hierarchical and recursive patterns, by humans and animals, has become an important research focus. Current hypotheses highlight key cognitive mechanisms that may underlie unusual human abilities such as language, music, and the visual arts. Research addressing this overarching theme has been hindered by the variety of disciplines involved, and a diversity of theoretical frameworks. Recently, an overarching framework has been sought in formal language theory, a component of the mathematical theory of computation which focuses on abstract patterns of varying complexity, and the computational algorithms that generate or process them. Formal language theory provides a comprehensive and explicit system for describing patterns, and combined with artificial grammar learning, has fueled an explosion of work on the biology and neuroscience of pattern perception in both language and other domains. This special issue combines comprehensive reviews, tutorials and opinion papers with new research on humans and animals, providing a summary of the current state of the art. With a focus on neuroscientific and comparative animal work, it provides a unique overview of this new and exciting area of cognitive science, and a glimpse of things to come.
I'm glad to see this special issue. Ever since I heard about the first "artificial grammar learning" experiments (e.g. Jenny Saffran, Richard Aslin, Elissa Newport, "Statistical Learning by 8-Month-Old Infants", Science 1996), I've been trying to persuade my colleagues that this work is really about auditory texture discrimination rather than about language learning. As I explained in "The texture of time: Even educated fleas do it", 11/24/2009:
[I]t seems to me that this experiment (and many others like it) are really not about syntax learning at all. Rather, they're extensions, into the dimension of time, of the research pioneered by Bela Julesz on pre-attentive texture discrimination in static visual displays. See e.g. Bela Julesz, "Textons, the elements of texture perception, and their interactions", Nature 290: 91-97, 1981:
The study of pre-attentive (also called effortless or instantaneous) texture discrimination can serve as a model system with which to distinguish the role of local texture element detection from global (statistical) computation in visual perception. […] Without using the sophisticated techniques described in this article, it is not ovious, even in the case of pre-attentive texture discrimination, whether local differences between the texture elements directly contribute to discrimination or whether these differences are sensed in a global way only through differences in the statistics of the texture.
One of things that emerges from the earlier research that Endressa & Hauser cite (and much more that they didn't) is that repetition in time is probably a sort of temporal texton for most animals — that is, adjacent elements that are identical in terms of a salient feature form a local temporal pattern that "directly contribute(s) to pre-attentive texture discrimination", rather than constituting a "[difference] sensed in a global way only through differences in the statistics of the [temporal] texture". Endressa & Hauser come close to saying this, but they don't get there, and their bibliography fails to cite the texture-perception literature at all.
E & H show that (at least under the circumstances of their experiments) part-of-speech is not a feature whose repetition is tracked by human pre-attentive perception. This is interesting, but by no means a novel type of discovery. The texture-perception literature is full of contrasts among local features that "directly contribute" to texture discrimination, local features that contribute via their statistical distribution, and local features that are not accessible at all to pre-attentive texture discrimination.
Or in "Rhyme Schemes, Texture Discrimination and Monkey Syntax", 2/9/2006, writing about some of Tecumseh Fitch's earlier work:
Short patterns of this type — strings characterized in terms of position-wise equivalence classes of their elements — are clearly very salient to humans. (And note that the equivalence-classes can be defined by any salient shared properties, like "starts with [k]" or "is an odd integer".) Given two random schemes from among the 15 possible patterns of length 4, or the 52 possible patterns of length 5, I suspect that after being familiarized to the first pattern, subjects will easily discrimate it from instances of the second, even if none of the local elements used in the experiment ever occurs more than once.
As the patterns' length increases, this task will clearly become harder and harder — unless the patterns to be discriminated happen to have rather different local properties. For example, if one length-12 pattern happens to start with AAABBB while the other one starts with ABCABC, the discrimination task will be trivial.
One way to model this would be to assume that subjects are sensitive to the statistics of equivalence-class properties of local substrings — what we might call schematic n-grams — just as they are sensitive to the statistics of conventional n-grams. This might be as simple as noting when adjacent symbol pairs are the same vs. different, or it might be based on progressively more complicated sorts of calculations, organized in the ways familiar to formal language theorists.
If this is on the right track, then formal language theory will help us understand this sort of auditory texture discrimination after all — but we'll need to to take a broader view of the vocabulary of the "language", and how it's related to the particular sequences that we use as stimuli.
I haven't had a chance yet to study the articles in the recent special issue (table of contents here), but I'm looking forward to it. More later, as I work my way through them.
Some other perhaps-relevant LLOG posts: "Hi ho, hi ho, it's off to formal language theory we go", 1/17/2004; "Learnable and unlearnable patterns — of what?", 02/25/2006; "Starlings", 4/27/2006;; "More on pitch and time patterns in speech", 10/15/2006; "Ask a baboon", 4/19/2012.
Andy Averill said,
September 10, 2012 @ 7:18 am
I can't help thinking of analogous processes that take place while listening to music, especially classical music. Composers seem to have understood, instinctively or consciously, the importance of controlling their listeners' attention through devices such as exact repetition, repetition with variation, and contrast. And certainly the (innate? or learned?) ability to detect meaningful small units within a larger stream of sound is crucial. I wonder if they say anything about this in this journal.
Daniel Barkalow said,
September 10, 2012 @ 1:42 pm
Andy: I don't know if this journal says anything on the subject, but there's a fascinating book about that: "A Generative Theory of Tonal Music" (Lerdahl and Jackendoff, 1983). It probably ought to be required reading for anyone who want to call the auditory processing they're studying "language". It should probably also be required reading for anyone who wants to argue for a theory of language with a small number of components. But it seems to be more influential in music theory than linguistics or cognitive science, unfortunately.
R A Chatwin said,
September 10, 2012 @ 11:01 pm
"… with new research on humans and animals …" Ain't there an "other" missing somewhere there. (Purism maybe, but not exactly not-germaine to the issue, although even Father Darwin did it — and "The Expression of the Emotions in Man and Animals" is a fascinatingly before-its-time book).
Jim Hurford: What is wrong, and what is right, about current theories of language, in the light of evolution? (2) | Replicated Typo said,
September 11, 2012 @ 2:33 pm
[…] area of research (see e.g. this paper published a couple of days ago in a special issue on "Pattern perception and computational complexity") that plays a fundamental role in answering the question of what is wrong, and what is right […]