Pitch sequence animation
« previous post | next post »
A neat animation by Jack Stratton of James Jamerson's bass line in the 1967 hit song "Ain't no mountain high enough":
The best way to watch it is full screen, in my opinon.
It would be nice to have a program that creates a similar dynamic highlighting of syllable-scale pitches and rhythms in speech, maybe based on Gentle and Drift.
Philip Taylor said,
April 5, 2020 @ 6:07 am
I find it odd (?disturbing?) that the transitions between the bass notes are shewn as continuous; might it not perhaps be better if they were omitted completely, as they are not (IMHO) representing anything present in the music.
[(myl) For me it's an effective representation of the flow of time.]
Philip Taylor said,
April 5, 2020 @ 6:51 am
OK, I've tried looking at it from that perspective, and I see what you mean, but for me a single level baseline (mapping time) from which the bass notes rise instantaneously to their final pitch would seem a viable (and to my mind, preferable) option.
[(myl) You should try it and post the results for comparison.]
Krogerfoot said,
April 5, 2020 @ 9:20 am
That is a truly inventive way to illustrate the genius of Jamerson's bassline, especially the thing he's doing in the choruses.
KevinM said,
April 5, 2020 @ 9:34 am
Although fetishized, traditional notation is a limited system for reminding a musician what to do next; as to both rhythm and pitch, it is an approximation. This animation–and thanks for posting it–is fascinating in that it captures many things that music notation cannot. Can we locate where the soul in Jamerson's playing resided? Never.
Michael Watts said,
April 5, 2020 @ 10:27 am
In terms of visualization, this kind of video is pretty common. Here's a MIDI visualization for an interpretation of Yellow Submarine: https://www.youtube.com/watch?v=fICgZ0MVvSU .
The video here is doing one thing better (for general public consumption) and one thing much, much worse:
The good thing it's doing is that it represents time going from left to right, instead of from down to up. These videos are conceived of as instructions to a pianist, so they lay out the keyboard left to right, as you would find it on a piano. But for the pure purpose of enjoying the video, having pitch vertical and time in reading direction makes more sense.
The bad thing it's doing is that in addition to representing time going from left to right, it also represents time as the radius of the circle representing the start of each note. This is really, really weird. There's already a time axis. The problem with drawing a straight line from the center of each circle to the center of the next is that the y-axis represents pitch, and many notes are held. When a note is held, the line shows pitch dropping steadily while — in fact — it's staying constant.
M. Paul Shore said,
April 5, 2020 @ 10:36 am
I usually pay a certain amount of attention to the bass line in popular music of whatever kind, making at least a rudimentary assessment of it; and so many bass lines come up short, the usual flaw being that they duplicate or nearly duplicate something that's already going on in the melody or harmony above, rather than providing a genuine counterpoint. Granted, sometimes the duplication-or-near-duplication option can be musically effective, but usually it seems fairly clear that the use of that option is just the result of musical apathy on the part of the bassist, and/or a failure of the bassist to really think through how he or she can best function in the overall musical texture.
In the case of "Ain't No Mountain High Enough", I never paid any attention to the bass line before now, because the melody, harmony, and lyrics of that particular song are so hackneyed that any time I hear a second or two of it I go into partial mental shutdown to protect myself from brain damage. But now that I've been prodded into really listening to its bass line, I have to say it's one of the best ones (i.e., bass lines, not songs) that I've ever heard. Brilliant, brilliant counterpoint. I wonder whether James Jamerson wrote any of it down as a memory aid for the recording session–based on what I've just read on line about his musical development, it seems pretty clear that he did know music notation–or whether he just did it all in his head, most likely in a short amount of time. If the latter, it's of course even more impressive.
Philip Taylor said,
April 5, 2020 @ 11:08 am
OK, there is nothing like a challenge to motivate a man ! At first sight it seems that HTML 5's <canvas> element might be the ideal way in which to present such an animation; I have not used <canvas> previously, but "nothing ventured, nothing gained".
I am reasonably confident that I won't be able to use JavaScript within the Language Log infrastructure, so it will have to be hosted off-site, but if I succeed in my aim I will add a link.
A <canvas> element follows, and in JS Bin it can be animated with a mouseover, but I suspect that there is little chance of that working here !
Daniel said,
April 5, 2020 @ 3:18 pm
Agreed, Michael Watts; it is very common. My favorite music animation is this one of Liszt's Bohemian Rhapsody No. 2.
The part about using the radius of the circle to indicate note duration is weird, but something related is done with Dance Dance Revolution (or Stepmania) games that is more effective, in my opinion: notes are colored by where they fall in the beat structure. That is, notes on beat are red; notes on the half beat are blue; notes halfway between red and blue are green; and so on. Triplets off beat have their own color. This kind of thing helps to show syncopation, for example, as a train of blue notes rather than red.
Philip Taylor said,
April 5, 2020 @ 3:30 pm
That is a fascinating animation, Daniel — thank you so much for drawing it to our attention. I assume that the green represents left hand and the red right, but what I like particularly is that long before the notes are played, the radial "stave lines" that will eventually carry them have already started their journey from the mysterious and invisible "centre of the musical universe".
Ben Zimmer said,
April 5, 2020 @ 3:36 pm
Stratton did a couple of good visualizations for James Jamerson basslines on Stevie Wonder songs. For the second one ("For Once in My Life") he transcribed the bassline and played it in isolation.
John Shutt said,
April 5, 2020 @ 4:48 pm
Complementary to the problem of visually facilitating perception of a single element (bass line) within a more complex musical setting, is the problem of visually facilitating perception of many elements as they fit together into a complex musical setting. (Which seems like it ought to have linguistic applications as well.) Here's an animation of a full orchestra performing The Blue Danube (I hope this link comes out right): link.
Michael Watts said,
April 5, 2020 @ 5:13 pm
Huh. I was trying to use Gentle's online demo, and it just crashes, attempting to load an audio file that isn't available and printing the graceful error message "undefined".
It's slowly compiling in a virtual machine; I kind of hope the same problem doesn't happen in the local install.
Michael Watts said,
April 5, 2020 @ 10:33 pm
Speech pitch seems a lot more continuous than musical notes, but I guess by focusing on highlighting at syllable scale you're intentionally trying to get rid of that aspect of it. If I'm wrong about this, then I don't understand why Drift by itself isn't what you want.
Gentle and Drift both do their own alignment, and Drift seems to have some problems. Gentle already produces timestamps, so it wouldn't seem to be necessary to have Drift do the same work again — is there something that would be more convenient than Drift for answering the simpler question "what is the speech pitch in this audio file at this time?"?
[(myl) I mentioned Gentle and Drift because they've been used to solve related problems for digital humanists — but there are plenty of other forced-alignment and pitch-tracking programs Out There, and widespread understanding of how to roll your own if you want to.
As for the idea of syllable-level visualizations, back in the 1970s some speech scientists showed that people can't distinguish original pitch contours from versions replaced by a piece-wise linear approximation, where each syllable either has a level pitch or a linear ramp, depending on duration and amount of change. I'll post some links when I have time to dig them up.
Update — a more recent replication of this idea can be found in Ravuri, Suman, and Daniel PW Ellis. "Stylization of pitch with syllable-based linear segments." In 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 3985-3988. IEEE, 2008.]
maidhc said,
April 7, 2020 @ 5:52 pm
Speaking as a former bass player, who put in quite a bit of time trying to learn James Jamerson's bass lines, there are two important things lacking in these visualizations:
-where the onset of the note comes in relation to the beat
-there is a poor indication of how long to sustain the note, i.e. whether there are gaps of silence between notes
It would not be hard to modify the presentation to address these, by adding a rhythm grid, and representing a note as a bar instead of a circle.
Connecting the notes with lines is misleading, because it makes it look as though there is a glissando.
Krogerfoot said,
April 7, 2020 @ 7:14 pm
"the melody, harmony, and lyrics of that particular song are so hackneyed that any time I hear a second or two of it I go into partial mental shutdown to protect myself from brain damage"
I never imagined I would encounter a musical sophistication so lofty and refined that it is actually injured by Marvin Gaye and Tammi Terrell's "Ain't No Mountain High Enough."
Viseguy said,
April 7, 2020 @ 7:16 pm
This video made my day — bless you, Jamerson-Stratton via Liberman! My only wish was for an F-clef transcription as a closed-caption option — one that addressed, inter alia, @maidhc's bullet points.
By chance, there was a Handel walking bass playing in the background while I watched. This made for an interesting comparison/contrast — and a good exam question for Musicianship 101.
M. Paul Shore said,
April 8, 2020 @ 9:13 am
KevinM: The posted animation doesn't even capture *one* thing that traditional music notation couldn't capture, let alone "many things". James Jamerson's bass line is played in a straightforward, unadorned way, with no pitch or rhythm nuances (i.e., at the sub-note level) to be notated–and contrary to your implied assertion, such nuances *can* be notated in the traditional system: you just have to take the trouble to do it. This bass line consists simply of eighth notes, quarter notes, dotted quarter notes, etc., without even much dynamic variation; its "soul" resides entirely in its skillful selection of those notes–which is nothing to look down on! There are plenty of other bass lines in the world, particularly in jazz, whose soulfulness involves detailed nuances of pitch and rhythm at the sub-note level, but this isn't one of them, and there's nothing wrong with that.
As maidhc mentioned, the animation presented here is actually considerably vaguer for practical purposes than traditional music notation would be. It fails to convey in an easily, instantly apprehensible way what pitch is supposed to be sounded and how long it should be sounded for. No bassist could play this animation accurately: he or she would have to decipher it, either by using some software (but why would anyone write such software?), or by printing the whole animation out in scroll form and then using a large ruler and a pencil to painstakingly draw horizontal and vertical lines as reference marks for ascertaining what the pitches and durations are supposed to be. Furthermore, as Philip Taylor has pointed out above, the animation's diagonal lines between the note-attacks are almost entirely meaningless: after all, it's not as if Jamerson was constantly sliding from each note to the next.
Your accusation that "traditional notation is a limited system for reminding a musician what to do next; as to both rhythm and pitch, it is an approximation" is almost totally off the mark. For one thing, it assumes that all music performance is primarily memory-based, with the notation providing nothing but reminders. In fact, any ensemble of good professional musicians can be presented with written music they've never seen before, and instantly bring it to life, with at least 90 to 95% of the quality they'll bring to their final, rehearsed rendition. I suspect that your view of traditional notation is based on your almost never having seen any beyond that in an introductory piano book, or in the beginning pages of some barely-ever-opened music theory textbook. Or maybe, more understandably, it's based on your having repeatedly encountered the typical inaccuracies of piano-vocal sheet music of various popular songs, and concluded that the discrepancies between what's on the page and what's on the recording are the result of some fatal flaw in traditional notation rather than what they really are, namely the result of the regrettable hackishness of the people who write and publish such piano-vocal sheet music. The fact is that traditional musical notation, even though it has a few features that are suboptimal (but that nevertheless can be overcome reasonably easily), has huge capacities for flexibility and expansion, with the main limitation being people's willingness to take the trouble to use those capacities, rather than take the easy way out and write down something that's simplified or oversimplified. If you want to get an idea of the rich resources of modern music notation, go to the music section of a good university or big-city library and take a look at the notation reference books of Gardner Read or Kurt Stone or various other writers; and then look at some scores by Brian Ferneyhough and/or Matthias Pintscher and/or some of the other "New Complexity" composers, or if the library doesn't have those, then scores by the late composers Luciano Berio and Pierre Boulez. The notational subtleties you'll see in those scores are potentially applicable to music of all kinds: they could even be used to capture the performance nuances of, say, a Billie Holiday or a John Coltrane or a Robert Plant. Again, it's just a matter of the notator bothering to do it.
M. Paul Shore said,
April 8, 2020 @ 10:27 am
Krogerfoot: I know I'll probably ruffle some feathers by saying this, but it's not difficult to be too musically sophisticated for "Ain't No Mountain High Enough" (other than appreciating the bass line). I have two basic objections to the song. One is that it belongs to the "call me and I'll be there" sub-sub-genre of popular song, which in my view should've been subjected to a fifty-year ban starting some time in the late sixties. (I often think of the "there" in question as possibly being some particular exit on the New Jersey Turnpike.) Second, the hook is founded on a harmonic alternation between the submediant or subdominant on the one hand and the minor dominant on the other (within the context of a minor key), a type of musical gesture that's been overused in R-and-B music to the point of absurdity. My platinum standard for popular song is "Glamour Profession" by Steely Dan, a song where the chords are so complex and unusual that I suspect it'd take the average reasonably talented professional composer/arranger a couple of hours or more at a piano or synthesizer to figure out exactly what they are.
Regarding the question of musical sophistication in general, might I suggest–in all sincerity and with no insult intended–that you might find it interesting to listen to Pierre Boulez's Le marteau sans maître or Luciano Berio's Sinfonia, with a view towards calibrating your mental scale of musical sophistication and non-sophistication and contemplating where the boundary between them might lie.
Philip Taylor said,
April 8, 2020 @ 12:13 pm
Well, if I may add a brief peraonal perspective — whilst "Ain't no mountain high" did little for me, at least I could recognise that it was music, if not the sort of music to which I would instinctively be drawn (I am more an Allegri's Miserere person). With Pierre Boulez's Le marteau sans maître or Luciano Berio's Sinfonia, I am afraid that to my uncultured ear they were the aural equivalent of much modern "art" (where "art" = "painting" or similar) — just splotches of paint scattered on a canvas, seemingly at random, yet praised by the cognoscenti as works of such ineffable meaning and quality that mere mortals should not be allowed to gaze on them, let alone own them.
M. Paul Shore said,
April 8, 2020 @ 12:34 pm
Perhaps for my above-mentioned calibration exercise I could instead suggest Stravinsky’s Le chant du rossignol and Ives’s Three Places in New England.
David P said,
April 8, 2020 @ 2:56 pm
Another way to illustrate a bass line: just play it:
https://www.youtube.com/watch?v=WejJ1RPKXjo&fbclid=IwAR2adlECu6y2DOHT2ndzyoALKFSTJWzwabs12hX-L-oqCTefSknU6OMBm5s
Krogerfoot said,
April 9, 2020 @ 9:28 am
M. Paul Shore: It's been a long time since someone offered to recalibrate my musical tastes, and it's really kind of charming to suggest that Pierre Boulez would ruffle my feathers. I didn't know much about Boulez until the 1980s, when he was in charge of IRCAM and collaborated daringly with some artists I admired. It wasn't until more recently that I began appreciating Marvin Gaye, so maybe my musical sophistication has deteriorated somewhat. Anyway, "Ain't No Mountain High Enough" came out in the late sixties, so it might have escaped your proposed ban on not-complicated-enough music.
Michele Sharik Pituley said,
April 9, 2020 @ 1:20 pm
M. Paul Shore wrote:
> Or maybe, more understandably, it's based on your having repeatedly encountered the typical inaccuracies of piano-vocal sheet music of various popular songs, and concluded that the discrepancies between what's on the page and what's on the recording are the result of some fatal flaw in traditional notation rather than what they really are, namely the result of the regrettable hackishness of the people who write and publish such piano-vocal sheet music.
It depends on whether the piano-vocal sheet music in question is intended to be a faithful transcription of a particular singer's performance (faithfully notating every melisma down to the 64th note), or a general outline of the tune. The former is interesting for study by those who wish to learn how to embellish a melody in a particular style (and for aural skills & transcription training for the transcriber), while the latter is useful to those for whom a more bare-bones notation allows them to quickly see a tune and put their own style on it (for example, "real books" in the jazz world).
I would also submit George Crumb's music or even Schoenberg's Sprechstiimme for examples of how traditional notation can be extremely flexible.
Michele Sharik Pituley said,
April 9, 2020 @ 1:44 pm
@Daniel: thanks for posting the Liszt visualization — it's gorgeous!
M. Paul Shore said,
April 9, 2020 @ 8:02 pm
Krogerfoot: It wasn't that I thought that Boulez's Le marteau sans maître would necessarily ruffle your feathers, just that it might give you a clarifying reference point; but it sounds as it maybe you had that reference point already, at least to some extent. I admit I was humorously exaggerating about the "brain damage" part, so maybe you were humorously exaggerating when you said you "never imagined [you] would encounter" someone who felt the way I did.
I actually like some Marvin Gaye songs, and I'd probably like more of them if I were to get to know more of them; it's just "Ain't No Mountain High Enough" in particular that I have a problem with. And please note that my retroactively proposed ban was not on uncomplicated music–there's actually a certain amount of uncomplicated music that I cherish–but rather on "call me and I'll be there" songs. Regarding that proposed ban, I'm thinking now that it should've been one taking effect in the early seventies, not the late sixties, and I wish I could revise my earlier comment. And yes, I realize that "Ain't No Mountain High Enough" would've escaped either ban: to my mind it was the later entries in that sub-sub-genre that really pushed things over the edge of tolerability. I also realize that, because "Ain't No Mountain High Enough" was one of the earlier entries in the sub-sub-genre, my accusation that its lyrics are "hackneyed" is arguably a bit unfair, taking that song to task for what are more the sins of later, similar songs; but I think that even by the standards of 1966/67 the hackneyed quality was already there.
M. Paul Shore said,
April 9, 2020 @ 8:10 pm
Michele Sharik Pituley: Yes, there are occasional admirable exceptions to the typically deficient quality of published piano-vocal versions of popular music.
I've often wondered how many popular musicians personally oversee, or even have a trusted representative oversee, the piano-vocal or other written versions of their music that are put on the market. Probably very few.
Krogerfoot said,
April 9, 2020 @ 8:31 pm
M. Paul Shore: I won't keep defending ANMHE, as it's far from my favorite Marvin Gaye performance, and I don't disagree with you about how original it is—it's a serviceable Ashford & Simpson tune, but probably wouldn't be remembered without Marvin Gaye/Tammi Terrell/Diana Ross's performances of it. I've been an aficionado of precious, unlikeable music for a long time and have learned that no matter how avant-garde, insanely complex, or downright weird some favorite artist of mine is, I will invariably meet up with someone who thinks they're a little too pedestrian.
I've often encountered the thing you noted, where people will dismiss something as unoriginal when it in fact is the very thing that subsequent imitators are aping, making you rant along the lines of "this stuff wasn't a cliché at the time—this guy invented the entire genre!" But I agree that ANMHE is not exactly a groundbreaking work.