http://www.sciencedirect.com/science/article/pii/S0097848501000791 ]]>

At least about the chaos. Not sure about period-doubling]

]]>[(myl) Rashevsky's Wikipedia article says that "In the early 1930s, Rashevsky developed the first model of neural networks" — but it also says "Citation Needed". Can you provide one?]

]]>Rosenblatt's perceptron wasn't some amazing revolution that people just ignored, it flat out didn't work unless your dataset was just about trivially classifiable to begin with. Neural networks need to be multilayer to overcome that flaw.

[(myl) This is not quite right — as Minsky and Papert (*Perceptrons: An introduction to computational geometry*, 1969) noted, multiple layers of linear perceptrons reduce by simple algebra to an equivalent single layer.

The crucial innovation was the non-linearities. Minsky and Papert noted that you could overcome the limitations of the linear perceptron that way, but incorrectly asserted that no learning algorithms could be constructed for such systems. I believe that Stephen Grossberg's methods proving them wrong, at least in certain cases, were already around in the late 1960s (though not published until his "Contour Enhancement, Short Term Memory, and Constancies in Reverberating Neural Networks" in 1973), but certainly by the time of J.J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities", PNAS 1982, and David Rumelhart and James McClelland, *Parallel Distributed Processing: Explorations in the Microstructure of Cognition* (1986), methods such as "back-propagation" were available. There was a substantial blip of PDP enthusiasm in the 1980s, but the main accomplishments of such systems were conceptual rather than practical, and they were eclipsed by statistical machine learning methods, until the 00's, when progress came in the form of some new learning algorithms and (mainly) much more efficient linear-algebra engines, originally developed as graphics units for gaming.

The deep history starts with McCulloch and Pitts, "A logical calculus of the ideas immanent in nervous activity", *Bulletin of Mathematical Biophysics,* 1943.]