Huang Renxun looks back on the key turning point: without GTX 580, there would be no NVIDIA today

In a recent episode of the Joe Rogan Podcast, NVIDIA CEO Jensen Huang recalled the key turning point regarding the origins of deep learning and the company’s fate:The breakthrough of deep learning began in 2012 and relied on the SLI dual-card interconnection configuration of two GTX 580 graphics cards that were not designed for AI.

Huang Renxun revealed that the core deep learning of today's AI and the hardware used for the first operation of its basic network are the high-end gaming graphics card GTX 580 based on the Fermi architecture and with 512 CUDA cores.

Although this graphics card was originally designed for top-level games, its powerful parallel computing capabilities have inadvertently become the cornerstone of rapid deep learning training.

In 2012, researchers Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton from the University of Toronto used a pair of 3GB GTX 580 graphics cards to train the famous AlexNet model.

This deep learning network with approximately 60 million parameters stood out in the ImageNet image recognition competition that year with an astonishing advantage of 70% over the manually designed algorithm at the time.

Huang Renxun pointed out that the developers of AlexNet optimized the algorithm to run in parallel on two GTX 580s, exchanging data only when necessary, which greatly shortened the training time. This also made the GTX 580 the world's first graphics card to run deep learning/machine learning AI networks.

Interestingly, when this milestone was reached, NVIDIA’s investment in the field of AI was minimal, and most of its research and development was still focused on 3D graphics and games.

It was the successful application of AlexNet on GTX 580 that made NVIDIA realize the huge potential of deep learning. Huang Renxun said that the company immediately shifted funding, development and research efforts to deep learning technology in 2012.

This transformation ultimately led to the original NVIDIA DGX supercomputer in 2016, the Volta architecture with the first-generation Tensor core, and subsequent DLSS technology.