Abstract:
A new study driven by community developers shows that NVIDIA GeForce RTX 30 Series and RTX 40 Series graphics cards hide computing potential that far exceeds official specifications. By unlocking the underlying firmware and hardware configuration, developers have successfully released part of the originally restricted computing power, allowing some models to achieve up to nearly twice the performance improvement in artificial intelligence and matrix computing scenarios.
According to reports, the core used in the RTX 30 series "Ampere" architecture graphics cards is essentially the same silicon as NVIDIA's professional-grade GPUs, but some of the computing capabilities of consumer-grade products are limited by software and firmware policies. An unlocking solution developed by a researcher can increase the instruction issuance rate of FP/BF16 FMA to FP32 operations, and is suitable for GA102, GA103, GA104 and GA106 cores.
In actual tests, the BF16 matrix multiplication performance of RTX 3090 Ti was improved by up to about 80%. Since matrix multiplication is an important basic operation for running most neural network models, this improvement has obvious significance for local AI reasoning and machine learning tasks.
At the same time, another unlocking project for the RTX 40 series "Ada Lovelace" architecture also achieved a breakthrough. This solution achieves approximately twice the computing throughput of FP16 and BF16 Tensor while retaining the FP32 accumulation accuracy.
Taking the RTX 4090 as an example, in the FP16 Tensor throughput test, the graphics card could originally reach 165.6 TFLOPS, but after enabling unlocking, the performance increased to 331.6 TFLOPS, almost doubling. This level is close to or even reaches the computing power of some professional-grade products, such as RTX 6000 Ada.
The researchers believe that this indicates that the AD102 core has artificial computing limitations in consumer-grade products, especially in key AI computing areas such as matrix multiplication and accumulation. These limitations will directly affect the operating efficiency of AI applications such as large language models. Currently, the unlocking project has been confirmed to support the AD102, AD103 and AD106 core models.
It is worth noting that two unlocking solutions for the RTX 30 series and RTX 40 series appeared almost at the same time, covering the two generations of GPU architectures Ampere and Ada Lovelace respectively. The developer stated that these modifications can pass the complete verification mechanism of the NVIDIA driver and successfully bypass PCIe Function Level Reset (FLR) restrictions. Therefore, the official original driver can still be used and CUDA-related test software can be run normally.
Test data shows that both GA102 and AD102 graphics cards maintain the same power consumption limit before and after unlocking. For example, under the 450W total board power consumption setting, the performance improvement comes entirely from the released computing power rather than additional increases in power consumption or frequency.
For users who use local large language models, generative AI, and other high-intensity computing tasks, the additional released Tensor computing power can improve the efficiency of prompt word preprocessing and inference stages, thereby achieving higher Token throughput and faster response speed.
However, the developer also reminds that such modifications are unofficial and may invalidate the graphics card warranty. In addition, the relevant modules need to be loaded first during the system startup phase so that the unlocking operation can be completed before the NVIDIA driver takes over the GPU. Therefore, this type of solution is more suitable for advanced users who have certain technical capabilities and are willing to take risks.
Comments