Language Log

« previous post | next post »

This headline in yesterday's NYT caught my attention: "Despite His Credentials, Nyquist Has Many Doubters". Of course the story is about Nyquist the Horse, who is named after Nyquist the Hockey Player. But for a moment, I thought it might be about Nyquist the Engineer.

Now, you may not have heard of Nyquist the Engineer, but if you're reading this, then you rely on his work many times a day — every time you use a computer or a phone or a (digital) camera or a monitor, or pretty much any other digital device that interacts with continuous signals in the real world.

At least, you rely on some work that bears his name, the Nyquist-Shannon Sampling Theorem. But like Nyquist the Horse, Nyquist the Engineer has some doubters.

The Wikipedia article on the Sampling Theorem notes that

The theorem was also discovered independently by E. T. Whittaker, by Vladimir Kotelnikov, and by others. It is thus also known by the names Nyquist–Shannon–Kotelnikov, Whittaker–Shannon–Kotelnikov, Whittaker–Nyquist–Kotelnikov–Shannon, and cardinal theorem of interpolation.

But the article ends with an unusual section under the heading Why Nyquist?, which starts "Exactly how, when, or why Harry Nyquist had his name attached to the sampling theorem remains obscure", and goes to discuss the history with comments like

Exactly what "Nyquist's result" they are referring to remains mysterious […] This explains Nyquist's name on the critical interval, but not on the theorem. […] it is not a sampling rate, but a signaling rate.

So, doubters.

Whatever Nyquist's role, the history of the sampling theorem is really interesting, as you can learn by reading Hans Lüke, "The origins of the sampling theorem" IEEE Communications Magazine 1999. He starts like this:

In 1948 and 1949, Claude E. Shannon published the two revolutionary papers in which he founded the information theory [1, 2]. In [1] the sampling theorem is formulated as “Theorem 13”:

Let f(t) contain no frequencies over W. Then

$$f(t) = \sum_{-\inf}^{\inf} X_n \frac{\sin \pi (2Wt – n)}{\pi(2Wt – n)}$$

where

$$X_n = f(\frac{n}{2W})$$

It was not until these papers were published that the theorem known as “Shannon’s sampling theorem” became common property among communication engineers, although Shannon himself writes in [2] that

This is a fact which is common knowledge in the communication art.

A few lines further on, however, he adds:

… but in spite of its evident importance [it] seems not to have appeared explicitly in the literature of communication theory.

The following analysis takes the above statement as its starting point. It will become apparent that mathematicians, practicians, and theoreticians in communication engineering came across the implications of the sampling theorem almost independent of one another, and that the links between them did not emerge until later stages of this development.

Lüke's reference [1] is Claude Shannon, "A Mathematical Theory of Communication", Bell System Technical Journal 1948, which has the extraordinary property of being accessible to any bright high-school student, despite being the founding document of the most important new area of mathematics in the 20th century.

Lüke ends his history this way:

The sampling theorem for lowpass functions plays an important role in communication engineering as a connecting link between continuous-time and discrete-time signals. The numerous different names to which the sampling theorem is attributed in the literature — Shannon, Nyquist, Kotelnikov, Whittaker, to Someya — gave rise to the above discussion of its origins. However, this history also reveals a process which is often apparent in theoretical problems in technology or physics: first the practicians put forward a rule of thumb, then the theoreticians develop the general solution, and finally someone discovers that the mathematicians have long since solved the mathematical problem which it contains, but in “splendid isolation.”

Some of the original references:

Edmund Taylor Whittaker, "On the functions which are represented by the expansions of the interpolation theory," Proc. Royal Soc. Edinburgh, Sec. A, vol.35, 1915.
Harry Nyquist,  "Certain topics in telegraph transmission theory",  Transactions of the A.I.E.E., 1928.
Karl Küpfmüller,"Utjämningsförlopp inom Telegraf- och Telefontekniken", ("Transients in telegraph and telephone engineering"), Teknisk Tidskrift 1931.
V. A. Kotelnikov, "On the carrying capacity of the ether and wire in telecommunications", Material for the First All-Union Conference on Questions of Communication, Izd. Red. Upr. Svyazi RKKA, 1933 (in Russian).
Claude Shannon,  "Communication in the presence of noise", Proceedings of the IRE , 1949.

More of the history can be found in A.J. Jerri, "The Shannon sampling theorem—Its various extensions and applications: A tutorial review", Proceedings of the IEEE, 1977.

Or for a broader and more digestible survey (with pictures!), in the form of a slide deck, see Radomir Stanovic et al., "Some Historical Remarks on Sampling Theorem", which follows the prehistory from Lagrange to Cauchy and Borel — and concludes that

Nyquist considered a different problem, the sampling theorem is a statement dual to that of Nyquist.

I'll take this as qualified support for the view that Nyquist deserves to have his name left on the theorem. And they do leave his name — along with 22 others — on their final slide:

It's a shame, anyhow, that none of these folks have ever had a race horse named after them.

Update — but whatever, Nyquist won the race.

 

May 7, 2016 @ 9:28 am · Filed by Mark Liberman under Awesomeness

Permalink