Nathaniel Rochester

Nathaniel Rochester (1919–2001) was an American computer engineer at IBM who helped design the IBM 701, one of the first commercially produced scientific computers, and who then used that machine to run some of the earliest computer simulations of neural networks. He was also one of the four signers of the 1955 proposal for the Dartmouth summer project, alongside John McCarthy, Marvin Minsky and Claude Shannon. Where those three are remembered as theorists, Rochester is the engineer in the founding group: the person who knew what the hardware of the day could and could not do, and who had the access to a working large computer that turned speculation about machine learning into running experiments.

Rochester's career placed him at the point where computer design met the first attempts to model the mind. He studied electrical engineering at MIT, worked on radar during the Second World War, and joined IBM in the late 1940s, where he became one of the principal logical designers of the IBM 701 (announced in 1952 as the Defense Calculator) and is credited with writing an early symbolic assembler for it, a program that let programmers write mnemonic names instead of raw numeric machine codes. That is itself a piece of symbol-processing history: using a computer to translate readable symbols into machine instructions is the same move that later made compilers and high-level languages possible. IBM's research group at Poughkeepsie, where the 701 and its successor the 704 were available, became one of the few places outside a university where a researcher could request hours on a serious machine for an unconventional experiment.

His best-known research result is the 1956 paper by Rochester, John H. Holland, Lawrence H. Haibt and William L. Duda, Tests on a cell assembly theory of the action of the brain, using a large digital computer, published in the IRE Transactions on Information Theory. The team set out to test Donald Hebb's 1949 proposal that groups of neurons strengthen their mutual connections when they fire together and thereby form cell assemblies that store patterns. They simulated a network of several hundred model neurons with randomly assigned connections on the 701, updating each unit's state and its connection strengths step by step according to a Hebb-style rule. The notable outcome was a negative one that taught something: the plain rule, as first stated, did not by itself produce stable, well-separated assemblies, and the simulated activity tended to spread across the whole network until the model was modified, for instance with inhibition and other additions. Computer simulation was showing a theory's hidden gaps, which was exactly the function the group hoped simulation would serve.

At Dartmouth, Rochester's contribution was less a single idea than the engineer's perspective, including interest in how a machine might be made to improve or originate behavior through controlled randomness. The 1955 proposal listed topics such as automatic computers, programming languages, neuron nets, the theory of the size of a calculation, self-improvement, abstractions, and randomness and creativity, and Rochester's practical experience colored the neural-network and machine-learning side of that list. His IBM colleagues at the time included Arthur Samuel, whose checkers-playing program also ran on IBM machines, so the company's research floor was an early home of what would later be called machine learning. Rochester is not the author of the term artificial intelligence; that choice was McCarthy's.

A common misconception is that the 1956 simulation was the first neural network on a computer, or that it was a failure. Other groups, including Belmont Farley and Wesley Clark at MIT in 1954, simulated network behavior on computers at about the same time, so the better claim is that it was among the earliest. Its disappointing first result was also the valuable part: it exposed what Hebb's verbal theory had left unspecified, which is a standard and respectable use of simulation. The same pitfall appears elsewhere in his story, where credit for the 701 and for early AI is spread across teams, and Rochester's role is easy to lose between the better-known names.

The modern relevance of his work is that the loop he helped open, state a learning rule, simulate it at scale, and let the behavior correct the theory, is how neural-network research is still done. Hebbian ideas persist in unsupervised and associative learning, in Hopfield networks and in a long line of biologically inspired learning rules, while the practical lesson that a simple local rule needs additional constraints before it yields useful structure echoes in normalization, regularization and inhibition-like mechanisms in contemporary models. The hardware side of his story also holds: it was general-purpose computers that gave early AI researchers a laboratory.

Read Nathaniel Rochester as the engineer of the founding group: the 701 designer who put a theory of the brain on a computer in 1956, found where it broke, and showed that simulation could be an experimental tool for cognitive science as well as an engineering method.