NVIDIA's "Huang's Law" is the main catalyst that has driven chip performance and efficiency to increase by more than 1,000 times in less than ten years. For Nvidia, "Huang's Law" is a fundamental way to transcend traditional chip acceleration fundamentals such as Moore's Law, which has dominated the technology industry in the past.
Visit the purchase page:
JD.com NVIDIA series product summary
Nvidia CEO Jensen Huang has said many times that Moore's Law is "slowing down" and that the concepts it supports are beginning to become obsolete. Especially after Huang Renxun delivered the GTC2023 keynote speech, the debate became more intense. If we look at what Moore's Law is, it has to do with the number of transistors on a microchip and how it "should" double every year.
Although Nvidia has migrated GPUs from 28nm to 5nm semiconductor nodes over the past decade, the technology has only accounted for 2.5x total revenue.
Bill Dally, NVIDIA's chief scientist, clarified in a blog post that NVIDIA's attitude toward next-generation technology revolves around "Huang's Law." We’ll get into what that means in depth, but Nvidia itself claims that the term originated from an IEEE Spectrum report and has since become known to multiple media outlets. The concept that Nvidia has recently implemented into its products is really interesting and could be the key to unlocking the future of the industry.
Bill Dally said in his "Hot Chip 2023" speech that Nvidia has witnessed an astonishing 1,000-fold increase in computing chip performance in the past decade. According to the book, such an improvement is impossible to achieve if Moore's Law is adopted, and the reduction of process technology will not have any impact on this number. Now, you might ask me how this is accomplished, and my answer is to prioritize innovation within a single "stack" rather than chip development.
To support this claim, NVIDIA said in its blog post that the introduction of the "Hopper architecture" was the decisive factor in showing huge performance figures because they use "8-bit and 16-bit floating point and integer math." Furthermore, the introduction of "Ampere Architecture" improves the performance of statistical learning, increasing the performance of computing workloads by 2 times. In order to connect various technologies together, NVIDIA's "NVLINK" technology came in handy, and finally achieved the x1000 breakthrough.
NVIDIA mentioned in the blog post that during the entire 10-year period, the company switched from the 28-nanometer process to the 5-nanometer process, and the performance increased only 2.5 times. This violates Moore's Law, which states that every time a chip is "shrunk", its performance will increase by 2 times year-on-year. Daly said that Nvidia's future depends on "Huang's Law", and "Huang's Law" will bring some opportunities for industry progress.
"It's an interesting time to be a computer engineer." Daly said, "The industry situation has really verified this fact. It can be said that the computer industry is at a decisive moment, and it all depends on how companies treat "the development of chips and computing."