Tone collections and affective reactions
« previous post | next post »
A few days ago, I pointed to a recent paper arguing that "major and minor tone collections elicit different affective reactions because their spectra are similar to the spectra of voiced speech uttered in different emotional states" ( Daniel L. Bowling et al., "Major and minor music compared to excited and subdued speech", Journal of the Acoustical Society of America, 127(1): 491–503, January 2010).
The argument in this paper has a nice rhetorical shape: the authors use a new form of quantitative analysis to explain the psycho-physiological substrate of a generally-accepted cultural association. But in this case, both sides of the explanation strike me as having some very odd properties. In this post, I'll try to explain what struck me as strange in their characterization of the cultural association between "tone collections" and "affective reactions". At some point in the future, I'll return to their quantitative analysis of music and speech.
They express the scale-affect association this way:
Other things being equal (e.g., intensity, tempo, and rhythm), music using the intervals of the major scale tends to be perceived as relatively excited, happy, bright, or martial, whereas music using minor scale intervals tends to be perceived as more subdued, sad, dark, or wistful.
And they cite authorities from Zarlino 1571 to Burkholder et al. 2005 in support of this view. But their emotional terminology involves a wide range of different psychological dimensions, which they collapse, without discussion, into just one.
Consider their first two oppositions: excited vs. subdued and happy vs. sad. One of these is a dimension of arousal, while the other is a dimension of emotional valence or polarity — and both in music and in life, these two dimensions are independent if not orthogonal.
Someone who's happy can be in a subdued state of calm relaxation, rather than being excited and ebullient; and someone who's unhappy can be in an excited state of panic, fear, grief, or rage, as opposed to being subdued and depressed.
And in the kinds of music under discussion, melody and harmony are used congruently with other musical dimensions to achieve the composer's affective goals. One of the least "subdued" and "wistful" pieces of music ever written is this one:
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Mozart here sets to music a hymn about the day of judgment, the day of wrath, the day the earth will be burnt to ashes. The music's relatively "excited" and even "martial" character surely does depend on "other things" like "intensity, tempo, and rhythm" — but the fact that it's in D minor doesn't make it even a little bit more "subdued" or "wistful" than it would have been if Mozart had written it in D major instead.
It's fair to say that the mood of this music is "dark" — but that's a matter of emotional valence or polarity, not physiological arousal.
A bit later in the same work comes some music that is genuinely subdued and even a little wistful:
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
And again, the fact that it's in B-flat major doesn't make it any more "excited" or "martial" than it would have been if it were in a minor key.
I don't know any systematic survey, but it seems to me that this is typical. To the extent that "major and minor tone collections elicit different affective reactions", it's not in terms of arousal-dimensions like excited/subdued, but rather in terms of psychological dimensions like those sometimes called "valence" or "polarity". And crucially, these are independent of level of arousal.
As a result, it seems odd to me that Bowling et al. chose to look at the differences between "excited" and "subdued" speech as the putative source of the affective associations of major and minor scales.
Most descriptions of emotional states — whether by psychologists, drama coaches, or novelists — involve a much richer ontology of affect. For example, Rainer Banse & Klaus Scherer ("Acoustic profiles in vocal emotion expression", Journal of Personality and Social Psychology 70:614-636 ,1996) distinguish 14 emotional categories: anxiety, boredom, cold anger, contempt, despair, disgust, elation, happiness, hot anger, interest, panic, pride, sadness, and shame.
I mention this paper because it's in Bowling et al.'s bibliography — and it was also the model that I used in 2001 to design a corpus of acted emotional speech (published as Emotional Prosody Speech and Transcripts). We also used a range of dominant-to-submissive attitudes, crossed with a range of degrees of vocal effort created by distance to the interlocutor. I'll be curious to see how Bowling et al.'s quantitative measures fare when applied to material like this.
[Update — for another example, in response to Brett's comment below, listen to the start of the second and third movements of Brahms' piano trio in B:
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Or the largo from L'inverno vs. the presto from L'estate in Vivaldi's Four Seasons:
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
Audio clip: Adobe Flash Player (version 9 or above) is required to play this audio clip. Download the latest version here. You also need to have JavaScript enabled in your browser.
The point is not that minor-key pieces are always vigorous and major-key pieces always languorous, just that's no reliable association in the other direction.]
Dan Lufkin said,
January 24, 2010 @ 1:13 pm
Mozart could write in any key he wanted to and achieve any desired emotional effect. Lesser lights, though, may well think, "This part should sound wistful, so I'd better write it in a minor key, the way my teachers taught me." The result over the years is that there's a high correlation between wistfulness and minority. Nothing psycho-acoustically inherent about it; it just reflects received opinion among musicians and their teachers.
[(myl) That's a theory — any evidence? Think of all the wistful and poignant major-key popular songs, and fiercely excited minor-key ones — roughly as many as vice-versa, I'd predict… of course, perhaps their popularity demonstrates the genius of their composers, who were thereby freed from the bounds of mere cultural convention… Unfortunately I don't know of any way to survey a balanced sample of unpopular or marginally-competent music (nor do I think that's the right way to test the idea, really). Anyhow, it's not just Mozart.]
Another point, and perhaps the basis for the above: for a long time, pre-Boehm, playing a woodwind instrument in a remote key required a lot of technique, cross-fingering, half-covering holes. Your average village orchestra stuck to as few accidentals as possible so the cheerful popular pieces for weddings and dances favored the easy major keys. (Here I speak as a mediocre recorder player who does not sound good in E-flat minor.) Mahler and Beethoven knew how to make the most of that key and you also hear it occasionally in older jazz, but most pickup orchestras are well-advised to skip it. There's no problem, of course, with keyboards and strings, which is where Bach struts his well-tempered stuff in any old key.
Geoffrey K. Pullum said,
January 24, 2010 @ 1:37 pm
I'm definitely with Mark on this one. Khachaturian's "Sabre Dance" is a wild piece of joyous fun, but in a minor key. Some traditional songs like "Amazing Grace" and "The Battle Hymn of the Republic" strike me as extraordinarily sad — I get chills, even tears in the eyes — but they're in a major key. That old line about minor keys being sad and wistful is something our piano teacher told us. Teachers get away with repeating other tired old nonsense too. The truth is out there (as Scully and Mulder used to say), but it's much deeper and more complex than people think.
Justin said,
January 24, 2010 @ 1:49 pm
Your principal concern seems to be that the comparison is made between the "arousal" of speech and the "valence" of music. I agree that this seems like an odd and unjustified comparison. When ones considers both (as you noted) to be single-dimensional representations of a high dimensional "affect-space," the correlation is trivial. Assuming that music and speech represent two distributions in affect-space, by collapsing each into a single-dimensional representation then comparing their correlations, one would expect that they would show *some* correlation (unless the dimensions being compared are perfectly orthogonal).
[(myl) Exactly.]
Ian Tindale said,
January 24, 2010 @ 3:19 pm
Why is it that when people terminate a phone conversation, they depart from normal speech and instead sing the parting words (the greetoutg?) in the form of a pretty little song.
[(myl) There are many circumstances in which stylized or quasi-chanted intontation may be used: calling, warning, complaining, etc. This reduces your question to a larger one without answering it, but that's all I have time for now, except to observe that Bob Ladd has a theory about this.]
uberVU - social comments said,
January 24, 2010 @ 4:02 pm
Social comments and analytics for this post…
This post was mentioned on Twitter by PhilosophyFeeds: Language Log: Tone collections and affective reactions http://goo.gl/fb/L65D…
Breffni said,
January 24, 2010 @ 5:25 pm
GKP:
Teachers are in the difficult position of having to introduce abstract terms that defy definition even by specialists. When the facts are complex, disputed and difficult to pin down, they often reach for first approximations that capture some salient dimension of the concept or distinction, maybe one involved in prototypical cases. That gives students some preliminary foothold: with experience, there's a fair chance that they'll develop a more nuanced understanding of the phenomenon, even if they can no more make it explicit than their teachers can. Kids are taught that major / minor maps onto happy / sad. They may or may not come to realise consciously that that's at best only roughly the case – but it's a usable heuristic for distinguishing the two, and being able to distinguish them is all that's required for most purposes.
[(myl) Happy/sad, though still problematic, would be a step forward from excited/subdued, which is the dimension whose correlates in speech this paper actually tests.]
So I'm inclined to cut teachers some slack in these matters. A more precise characterisation of tonality – or, say, part of speech – would be unteachable to kids, and a broad-strokes characterisation doesn't necessarily set them on the wrong track forever. I bet a large proportion of linguists initially learned that verbs were action words, and I would likewise bet that of those that did, the great majority were nonetheless reliably able to identify verbs long before they started studying linguistics.
Brett said,
January 24, 2010 @ 6:49 pm
Apart from the opening bars, I'm not sure that Mozart Dies Ire is really a good example. Most of the rest of the movement sounds downright jolly in comparison, desspite of the fiery lyrics. Having performed the Requiem a few times with more experienced musicians, I know that this is a sort of running joke about the piece. It think it's kind of a pity, since the first few bars giving such an amazing impression of the wrath of God, but the rest of it doesn't really deliver.
[(myl) Well, it's the thought that counts. And of course, for some people, the day of judgment is happy though solemn occasion, once a suitable atmosphere of awe is established, so maybe Mozart did this on purpose. Anyhow, another quasi-random example, among the hundreds on my ipod, consider the second and third movements of Brahms' piano trio in B (which I've added at the end of the post above, since the audio function doesn't work in WordPress comments).]
More on topic, I think the problem with the rest of the piece is definitely related to its use of major chords. I think there's definitely something to the notion that the primary major triad sounds more upbeat that the minor. It's not universal, but it's definitely a preference. I think the pieces that most capably avoid the conventional emotional valances associated with the various keys make effective use of other parts of the scales.
[(myl) But what the paper tested was not exactly "the conventional emotional valences", it seems to me:
What about "relaxed and happy", or any of the high-arousal negative emotions? And I did a quick check of a sample of the baroque, classical, and romantic-era music on my ipod, and by my counts, major and minor keys have roughly the same relative frequency in low-arousal movements as in high-arousal movements movements. ]
Bob Ladd said,
January 24, 2010 @ 7:24 pm
Thanks for the reference to my paper, Mark. But in case North American readers are puzzled by Ian Tindale's query about switching into singsong at the end of of phone conversation [I'm assuming from the name and the comment that Ian is British], it's probably worth pointing out that what I called "stylized intonation" is normally used on 'Bye at the end of a phone conversation in Britain but not normally used in North America.
Bill Benzon said,
January 24, 2010 @ 7:32 pm
Thanks for that analysis, Mark. Though I've not tried to think it through "to the bottom," it makes sense to me.
I don't know quite how to put this, but it's not as though music has the major and the minor scale and that's it. Emphasis on that distinction seems to be a simplification of the harmonic practice that evolved in European art music from, say, the 16th through the 19th centuries. But other scales were used and are used, not to mention musical practice in other cultures.
Consider modal jazz, which makes extensive use of the so-called dorian mode, which is neither a major nor a minor scale. The root triad is minor, but jazz musicians have often used quartal harmony (built on fourths rather than thirds) with the dorian mode (think of the classic John Coltrane quartet with Jimmy Garrison on bass, McCoy Tyner on piano, and Elvin Jones on drums). And then there is the blues, which tends to be notated in major keys — though there is such a thing as a minor-key blues. The blues scale will be played over a major chord, but admits of flatted thirds and sevenths, which are, however, microtones rather than minor thirds and sevenths. So you can't play the true tones on the piano, but vocalists can sing them and other instrumentalists can get them (guitarists bend strings, brass and saxes can "lip" them, etc.). A blues can be mournful or joyful depending on tempo and pulse, etc. And the saddest of sad blues can be played over major harmonies.
[(myl) Though I never thought of it before, maybe one of the attractions of "power chords" is that they're ambiguous as to modality. The traditional goals of avoiding excess fuzz and using easy fingerings are probably primary, but still… ]
Bill Benzon said,
January 24, 2010 @ 8:56 pm
Yes, power chords are modally ambiguous. & it's not at all obvious to me that those traditional goals are necessarily primary. Remember that rock has a lot of modally ambiguous blues in its roots.
We've also got to consider how the chord is voiced and whether not we're using only triads, or whether we're adding upper extensions (7ths, 9ths, etc.). A major triad in root position consists of a major third (root to third) and a minor third (third to fifth. Put it in first inversion, with the third at the bottom of the chord, and you've got a minor third (third to fifth) and a fourth (fifth to octave). A minor triad in root position consists of a minor third (root to third) and a major third (third to fifth). Put it in first inversion and you've got a major third (third to fifth) and a fourth (fifth to octave). And you've got a second inversion for each chord, where the fifth is on the bottom. Of course, you can also play the inversion game with chords having upper extensions. So, between voicing and extensions you can do quite a bit to vary the quality of a chord.
Jerry Friedman said,
January 24, 2010 @ 9:07 pm
I take it the excerpt from the second movement of the Brahms piano trio is in a minor key and the one from the third is in a major key? And I like the idea that I could think of, or for that matter listen to, a jazz piece and know what kind of harmony was being used.
I wonder whether musicologists have asked people to describe the emotions of musical extracts and whether they've tried to correlate that with modality and other factors.
FM said,
January 24, 2010 @ 9:24 pm
Irrelevantly, because I can't find a relevant place to report this, a friend showed me what seems to be an automated snowclone generator:
http://www.manateecomics.com/
Mark F. said,
January 24, 2010 @ 9:50 pm
It seems to me that a lot of the comments are still conflating the happy/sad valence with the excited/subdued one, even while thinking they are agreeing with Mark's post.
Personally, I buy the happy/sad mapping for major/minor, if you accept the "all other things being equal" caveat. An alternate mapping is to safe/scary. But excited/subdued isn't one I would have included at all.
Ian Tindale said,
January 25, 2010 @ 3:21 am
…or also, concluded / seeking closure ?
(Bob Ladd, – you’re right, I am British).
Rubrick said,
January 25, 2010 @ 3:42 am
It strikes me that "music using the intervals of the [major/minor] scale" represents such a messy and complex a data set— even an individual piece does— that I wouldn't expect to get very far using that as a starting place.
Rather than saying "all other things being equal", why don't they make the other things be equal by using a study of the effect of major and minor intervals, played at consistent tempo and volume, on emotional response? Surely many studies must have been done. (Is it in fact conclusively established that the intervals themselves, played in sequence or as a chord, elicit consistent emotional responses? Does it hold for places with non-Western musical traditions?)
[(myl) There are dozens if not hundreds of studies of this general sort, some quite old, e.g. Christian Paul Heinlein, "The affective characteristics of the major and minor modes in music", Comparative Psychology 8(2):101-142, 1928; or (more recently) Emery Schubert, "Modeling Perceived Emotion With Continuous Musical Features", Music Perception 21(4):561-585 2004; Laura-Lee Balkwill & William Forde Thompson, "A Cross-Cultural Investigation of the Perception of Emotion in Music: Psychophysical and Cultural Cues", Music Perception 17(1):43-64 1999.]
If they've found that minor chords share some fingerprint with "sad" speech and major chords with "happy" speech, they're really onto something; but it doesn't sound as though that's what they've done.
[(myl) They take the mode/affect correlation to be established by the literature they cite, and focus on how to explain it by comparing the empirical distribution of musical intervals (relative to the tonic) with the relative strength of harmonics in speech, as measured in a particular (and new) way. As I said in the body of the post, this is an investigation of exactly the right sort. But they follow some (though by no means all) previous researchers in projecting all dimensions of affect and arousal into a single line, and as a result of this step, I'm skeptical of their interpretation of the rest of the enterprise.]
Szwagier said,
January 25, 2010 @ 12:36 pm
As someone who's interested in both language and music, although with little formal education in the latter, I'm always interested in research in this area.
One thing troubles me, however; the apparent insistence on tonal music as though the 20th century never happened. It's all very well to claim, and even show, that major keys by and large induce these emotions, and minor keys induce those, but in my case – purely anecdotal, clearly – major and minor keys mostly induce irritation.
What about the people who actively dislike western tonal music? When all's said and done, this is likely to be the majority of humanity…
[(myl) Surely there's music on earth that's ever been as popular as the various varieties of 20th-century Angophone popular music, all of which is resolutely tonal.]
Assorted links | India News Blog, Latest News From India, Latest Blogs From India said,
January 25, 2010 @ 1:12 pm
[…] 2. Happy and sad. […]
Matt McIrvin said,
January 25, 2010 @ 1:20 pm
I was just thinking about what consistent associations I have with major/minor keys, given that happy/sad seems too simplistic, and eventually decided that I think of major keys as sweet and minor keys as salty/savory.
(Which leads to fanciful extensions: power chords made from octaves and fifths are citrus-sour, blue notes and diminished chords add bitterness, dominant and major seventh chords are spiced in different ways…)
Rick S said,
January 25, 2010 @ 3:23 pm
Has anybody ever studied the effects of music on an EEG? I've always felt (intuited) that minor keys have more complex harmonic ratios (many more non-simple fractions), and wondered if that correlates with a more disturbed/conflicted mental state. That same mental state also, if I'm not mistaken, affects vocalization (the basis of some claims about detecting dishonesty via vocal qualities), which could explain the major/minor-happy/sad pairing as well as connect vocal quality to musical modality. And it certainly seems a more objective metric than the excited/subdued scale.
[(myl) See Anders Green et al., "Music in minor activates limbic structures: a relationship with dissonance?", NeuroReport 19(7):711-715, 2008:
Warning: NeuroReport describes itself as "a channel for rapid communication of new findings in neuroscience". In this particular case, the stimuli used are so vaguely described that the experiment is impossible to replicate and thus impossible (in my opinion) to interpret.]
DCBob said,
January 25, 2010 @ 3:36 pm
I think 'blue notes,' by moving between major and minor thirds, explore in minute detail the geography between 'happy/excited/sweet' and 'sad/muted/savory' moods and express a great deal of ambiguity of mood through that movement in sound. Note by the way that blue notes are found both in African and in European folk traditions, and the American blues tradition simply expands the repertoire by combining elements of different traditions.
Tom Recht said,
January 25, 2010 @ 3:49 pm
The second and third movements of the Moonlight Sonata are another good counterexample.
Szwagier said,
January 25, 2010 @ 4:58 pm
Please note that I very carefully did not say Anglophone – you also have to include the all the European-originated (in the sense that the populations who live there are in the majority cultural, as opposed to 'racial', descendants of Europe) forms, which would include Central and South America, the Caribbean, and Southern Africa, not to mention a reasonable amount (being academically cautious) even of Asian popular music (that which is clearly based on western models).
Despite this, there are undoubtedly large areas of Asia, Africa and the Indian sub-continent where all this music is either not heard or not important. While it's certainly true that these other musical traditions have their own scales, it's also true that these bear little or no relation to what we would call 'tonal' music. The terms 'major' or 'minor' are meaningless in relation to these musics – they just do not work that way.
The assumption in the research I've read seems to be that western-style major/minor/whatever is either the only formulation there is or, perhaps even worse, the 'best' formulation there is. It isn't, and it isn't. Major/minor (as an example only) can say nothing definitive about human reactions when it's a distinction that is meaningless for a large proportion of humanity. I believe this is a bias that needs to be addressed before this sort of research can be taken seriously.
Charles Minus said,
January 25, 2010 @ 7:59 pm
I find this article to be stunningly ehnocentric. As has already been pointed out, major/minor distinctions are meaningless in most of the world's music. Making these overbroad statements on the relationship between music and emotion is without base and really insulting to other musical cultures. Most of the music in non-european (and much ethnic European) cultures is based on the pentatonic scale. The pentatonic scale is based on the physical properties of vibrating objects, so all pentatonic scales are pretty much the same. But the "in between" notes are all over the place from one culture to another.
As to the "blue" notes in jazz and blues. I am not an expert on this obviously, but one of the explanations I have heard is that this note comes from an attempt to bend the third of the European scale to fit the third of the scale heard in much African music.
I would contend that all the emotional content attached to certain chords and intervals is cultural and learned. Listen to the various ragas in classical Indian music and see if you can tell what mood or time of day they are supposed to represent. I can't, but someone raised in that culture has no problem making the connections.
Rich said,
January 27, 2010 @ 4:25 pm
The main thing (not the only thing) that irritates me about these kinds of claims is that they usually function as just-so stories explaining the "rightness" of Western Classical music. We start with a prejudice — in this case, as someone said above, the thing your piano teacher told you about minor keys when you were nine — and then go and find some data to create a justification for it.
Personally I'd put this in the same box as those evolutionary explanations for boys preferring blue that pop up every now and again.
affective acoustics « THOUGHT FOR FOOD said,
February 9, 2010 @ 9:52 pm
[…] liberman, m. – tone collections and affective reactions […]