Â鶹´«Ã½

In memory of Robert Hecht-Nielsen, an influential neuroscientist, entrepreneur and Â鶹´«Ã½ professor

Robert Hecht-Nielsen

San Diego, Calif., July 1, 2019 -- Robert Hecht-Nielsen was an influential neuroscientist, entrepreneur, and electrical engineering professor at the University of California San Diego. He passed away in his sleep on May 26, 2019 in Del Mar, Calif. He was 71 years old.

A pioneer in the development of neural networks, Hecht-Nielsen authored the first textbook on the subject, Neurocomputing, in 1989. His contributions extend beyond neuroscience to the fields of cognitive science, artificial intelligence, and physics.

As a mathematics student, he wanted to find the underlying mathematical principles implemented by neural tissue to achieve cognition, he said in a Q&A article from the British Computer Society in 2007. This inspired him to make seminal contributions to the theory of neural networks during the “second great wave” of neural network research that transpired during the 1980s and 1990s, the first wave being the period immediately following the development of the perceptron in the 1960s and the third wave being the current era of the development and use of powerful deep neural network algorithms.

The understanding of the training and behavior of neural networks established in the 1980-90s is a crucial, foundational aspect of the application of deep neural networks and Hecht-Nielsen was front-and-center in that development. Hecht-Nielsen made significant contributions to the theory of neural network training using the back-propagation algorithm and was among the core group of researchers of the time who rigorously proved that neural networks are “universal approximators,” in principle capable of learning any statistically optimal decision making rule, pattern recognizer or situation predictor.

In 1986, he co-founded HNC Software, a neural networking startup in San Diego, based on his breakthrough work in predictive algorithms. HNC was perhaps the first start-up in the neural network/machine learning space to be significantly financially successful, and in many ways this is due to Hecht-Nielsen’s deep understanding that given enough training data and computational power, neural networks will nearly always find an optimal solution to a properly posed prediction problem.

“There were a number of startups, but only HNC achieved the success that it did. Robert deserves a lot of credit for that,” said Â鶹´«Ã½ computer science professor Gary Cottrell, who noted that many of his students worked for HNC part time. “HNC also developed the first specialized hardware for neural network computations, preceding by many years the development of GPUs,” said Cottrell.

The company went public in 1995 and was eventually purchased by Fair Isaac, where Hecht-Nielsen went on to serve as a vice president of R&D.

In the 2000s, when the machine learning community had moved away from neural networks as a center of research focus, Hecht-Nielsen told Â鶹´«Ã½ electrical engineering professor Ken Kreutz-Delgado that the secret to HNC’s success was that in the domain of financial prediction they were able to exploit the fortuitous availability of a large set of training data and the willingness to commit, for the time, a very large amount of computational resources.

According to Kreutz-Delgado, Hecht-Nielsen then went on to predict that once the availability of massive data sets and ready access to vast computational power were established, the return to the use of neural networks as a central predictive tool was inevitable.

“It is noteworthy that, with the addition of the use of deep neural networks, this has indeed turned out to be the case,” said Kreutz-Delgado.

Teaching at Â鶹´«Ã½

In 1986, Hecht-Nielsen joined Â鶹´«Ã½ as an adjunct professor in the Department of Electrical and Computer Engineering at the Jacobs School of Engineering. At that time, he developed and taught a pioneering three-quarter graduate course series on neurocomputing, a popular course which earned him a graduate teaching award and which inspired many graduate students to engage with neural networks both in research and in industry long before the current interest in neural networks.

“Robert always thought BIG and he had a tremendous impact on me through his unique ECE 270 course which blended neural networks, signal processing, biology, mathematics, entrepreneurship, and leadership,” said Dharmendra S. Modha, a Â鶹´«Ã½ alumnus who is now an IBM Fellow. “I wrote my first paper with Robert and he served as a guiding force throughout my career, always providing a truly uncommon perspective.”    

In 2006, , to further his investigations into the use of neural network models in computational neuroscience. In particular, he focused on developing a theory of human intelligence, which he referred to as “Confabulation Theory”, which postulates that the mind invents (“confabulates”) a variety of scenarios about the world which are then tested against current internal models and sensory data to arrive at a best fit.

Robert Hecht-Nielsen was a bigger-than-life scientist and entrepreneur who had a large impact on the world in many domains. He will be greatly missed and always remembered.

Biography

Hecht-Nielsen received his Ph.D. in Mathematics from Arizona State University. Over the next 40 years, he applied his analytical skills to advance the fields of neuroscience, cognitive science, artificial intelligence and physics.

In 1987, he co-founded the International Joint Conference on Neural Networks (IJCNN), which celebrates its 32nd anniversary in 2019.

Hecht-Nielsen was an INNS Gabor Award Winner, an IEEE Neural Networks Pioneer Award Winner, INNS Fellow, and IEEE Fellow, among many other awards and recognitions.

Hecht-Nielsen was a member of the Â鶹´«Ã½ Institute for Neural Computation and was a founder of the Â鶹´«Ã½ graduate program in computational neurobiology.

Celebration of life

A celebration of Hecht-Nielsen’s life will be held at the 94th Aero Squadron Restaurant in San Diego on Aug. 1, 2019 at 3 p.m. In lieu of flowers, donations in his name can be made to the Young Eagles program (dedicated to giving young people their first free ride in an airplane) at EAA-Young Eagles, P.O Box 3086, Oshkosh, WI 54903.

Robert Hecht-Nielsen is survived by Judi, his wife of 41 years, his son Marcus (Carolina), and 4 grandchildren: Melissa (Pablo), Johann, Mia, and Carla.

Media Contacts

---

Daniel Kane
Jacobs School of Engineering
858-534-3262
dbkane@ucsd.edu