How a stubborn computer scientist accidentally launched the deep learning boom
During my first semester as a computer science graduate student at Princeton, I took COS 402: Artificial Intelligence. Toward the end of the semester, there was a lecture about neural networks. This was in the fall of 2008, and I got the distinct impressionβboth from that lecture and the textbookβthat neural networks had become a backwater.
Neural networks had delivered some impressive results in the late 1980s and early 1990s. But then progress stalled. By 2008, many researchers had moved on to mathematically elegant approaches such as support vector machines.
I didnβt know it at the time, but a team at Princetonβin the same computer science building where I was attending lecturesβwas working on a project that would upend the conventional wisdom and demonstrate the power of neural networks. That team, led by Prof. Fei-Fei Li, wasnβt working on a better version of neural networks. They were hardly thinking about neural networks at all.