<\/p>\n
By \"pause group\" I mean simply the stretch of speech between silent pauses, as described in \"The shape of a spoken phrase<\/a>\", 4\/12\/2006. As a source of data, I used the\u00a0Mississippi State word alignments<\/a> for Switchboard. (In particular, the version that I used is\u00a0here<\/a>, and\u00a0the word counts for the\u00a0509,242\u00a0pause groups in the corpus, as I extracted them from the *word.text files, are\u00a0here<\/a>.)<\/p>\n The minimum length is one, and the maximum is 63:<\/p>\n The mode is 1 — as it would have to be for a two-state markov process to be responsible for the data — and the mean is 6.03. However, the empirical probability of continuing after N words is by no means constant:<\/p>\n (The histogram counts are\u00a0here<\/a>, and the calculated empirical probabilities of continuation are\u00a0here<\/a>.)<\/p>\n I conjecture that the special behavior of very short pause-groups reflects the fact that these conversational pause groups are a mixture of at least two quite different processes, one process generating quasi-independent contributions like<\/p>\n yeah i mean and the other generating short backchannel feedback like \"right\", \"I know\", \"yeah really\", \"oh I see\", …<\/p>\n The larger-scale process seems to gradually run out of steam, in the sense that beyond 3, the probability of continuing falls gradually and steadily up to and beyond 25 words. This decline is systematic and statistically significant, and it means that the length distribution of even the longer pause groups can't reflect a simple markov process (for the reasons explained here<\/a>). However, the fall is quite gradual, and the distribution is quite a smooth one, so that (especially if we treat length one as special) it would be pretty well approximated by an exponential decay:<\/p>\n What about sequential effects? At least taking each conversational side by itself, there is a statistically significant but small (r = 0.1) positive correlation between the lengths of adjacent pause groups. Linear regression yields<\/p>\n Gn+1<\/sub> = 5.4 + 0.1*Gn<\/sub><\/p>\n That is, the length in words of pause group n+1 is predicted to be 5.4 plus one tenth of the length of pause group n. There's enough data that the slope of this relationship is clearly different from zero — \u00a0p < 2*10^(-16), according to R — but only around 1% of the variance in length is being accounted for.<\/p>\n In order to look at line lengths in Whitman's poetry, I downloaded the 1881-1882 edition of Leaves of Grass from the Whitman Archive<\/a>, and ran the .html file through a little script to eliminate (I hope) everything but poem text and titles, with the titles used to delimit poems but otherwise ignored. Run-on lines (marked with line-initial spaces in this text) were joined. The histogram of line lengths in the result is as follows:<\/p>\n Here we can see some evidence that a different process is involved in creating the counts for very short lines; and in this case, we can tell more exactly what it is.\u00a0I didn't take the time to distinguish section numbers and internal section names from lines of poetry, and most of these — there are about 200 of them in the 10,202 lines — are of length 1. And in fact, most of the the \"lines\" of length 1 are such section numbers and internal section names.<\/p>\n The modal line length is 10 words — with or without omitting the lines of length 1 — and the mean is 11.1. Omitting \"lines\" of length 1, the mean is 11.3.<\/p>\n This seems to be a distinctly different pattern from Conrad's paragraphs and Switchboard pause groups. But a plot of the empirical probability of continuation has some qualitative similarities to the Switchboard plot:<\/p>\n In particular, there's an initial rise (here from 1 to 2) reflecting the fact that most very short \"lines\" are the output of a quite different process; and then a steady fall (here from 2 to 15) reflecting the same non-markovian \"running out of steam\" phenomenon that we saw in the Zebra Finch song bouts (\"Finch linguistics<\/a>\", 7\/13\/2011) as well as in the Switchboard pause groups.<\/p>\n The patterns are quantitatively quite different — but perhaps a wider range of free-verse authors, and a wider range of speech styles, would yield more quantitative overlap. In particular, it seems possible that skilled extemporaneous narrative might have a modal pause-group length more like Whitman's modal line length.<\/p>\n What about sequential effects in the Whitman line-length data? There's also a positive correlation between the lengths of adjacent lines, and it's a bit larger than in Switchboard pause groups: r=0.3, 9% of variance accounted for. The coefficients of a linear model are<\/p>\n![\"\"](\"http:\/\/languagelog.ldc.upenn.edu\/myl\/SWBPauseGroups1.png\" "\\"Click")
<\/a><\/p>\n![\"\"](\"http:\/\/languagelog.ldc.upenn.edu\/myl\/SWBPauseGroups3.png\" "\\"Click")
<\/a><\/p>\n
\nfor somebody who is
\nyou know for most of their life has has
\nuh
\nnot just merely had a farm but had ten children
\nhad a farm
\nran everything because her husband was away in the coal mines and
\nand you know facing that situation it it's quite a dilemma
\ni think<\/span><\/p>\n![\"\"](\"http:\/\/languagelog.ldc.upenn.edu\/myl\/SWBPauseGroups2.png\" "\\"Click")
<\/a><\/p>\n![\"\"](\"http:\/\/languagelog.ldc.upenn.edu\/myl\/WhitmanLinesX1.png\" "\\"Click")
<\/a><\/p>\n![\"\"](\"http:\/\/languagelog.ldc.upenn.edu\/myl\/WhitmanLinesX2.png\" "\\"Click")
<\/a><\/p>\n
![\"\"](\"http:\/\/languagelog.ldc.upenn.edu\/myl\/WhitmanLines4.png\" "\\"Click")
<\/a><\/p>\n