Abstract:
Apple’s latest A20 Pro chip is demonstrating its true potential in the field of on-device artificial intelligence. A latest actual test shows that the iPhone 18 Pro equipped with A20 Pro can locally run a large AI model with 27 billion parameters, and its generation speed is twice that of the previous generation iPhone 17 Pro, triggering a new round of attention from the outside world about the local AI capabilities of mobile phones.
This test was conducted by developer Adrien Grondin. In the demonstration, the iPhone 18 Pro ran a 27 billion parameter model called Bonsai 27B. Without relying on cloud servers, the device completes inference tasks directly locally and exhibits a response speed that far exceeds that of the previous generation of products.
The core factor driving this performance leap comes from the A20 Pro’s new artificial intelligence hardware architecture. Unlike previous Apple chips that were only equipped with a single set of 16-core neural network engines, the A20 Pro adopts a dual 16-core neural network engine design for the first time, with a total of 32 neural network processing cores. This is also the first time in the history of Apple’s self-developed chips to adopt a dual neural network engine architecture.
In addition to the upgrade of the neural network unit, the memory system is also an important part of performance improvement. Both iPhone 18 Pro and iPhone 18 Pro Max are equipped with 12GB, 96-bit wide LPDDR5X memory, which has higher transmission speed and greater data throughput capacity than previous generation products.
Test results show that when running large-scale language models, the new neural network engine and the faster memory system form a synergy, giving A20 Pro a clear advantage in Token generation speed. For AI applications that rely on local reasoning, this means faster response times and a smoother interactive experience.
However, this breakthrough does not mean that smartphones can already run all large models without limitations.
Researchers pointed out that although the A20 Pro successfully pushed the iPhone into the 27 billion parameter model era, local AI is still constrained by memory capacity. For example, even after using 2-bit quantization compression, the newly released Bonsai 2 model is still too large and cannot be completely loaded into the local memory space of the iPhone 18 Pro, resulting in performance degradation.
This also reflects a practical problem in the current development of mobile phone AI: computing power is growing rapidly, but the contradiction between memory capacity and model size still exists. Even if the chip performance continues to improve, if the model itself cannot be completely loaded into the device memory, it will still affect the actual operating efficiency.
In contrast, the first-generation Bonsai is a 1-bit quantization model and has significantly lower resource requirements. Developers say that this type of model can even run on devices with only 4GB of memory, and can achieve higher performance on 8GB or 12GB memory platforms.
From the perspective of hardware specifications, the A20 Pro is not only equipped with dual neural network engines, but also has a unified memory bandwidth of up to 115.2GB/s. Some tests show that in workloads optimized for neural networks, the peak throughput capability of the neural network engine even exceeds that of the 7-core GPU inside the chip.
This result reflects that Apple is accelerating its "device-side AI" strategy. Unlike relying on cloud data centers, local AI can bring faster response times, better privacy protection and offline availability, which are the development directions that Apple has emphasized in recent years.
As generative artificial intelligence gradually becomes an important feature of smartphones, competition among mobile phone manufacturers is also shifting from imaging performance and processor benchmarks to local AI reasoning capabilities. The performance of the A20 Pro means that smartphones have begun to have the ability to run billions or even tens of billions of parameter models.
However, as model sizes continue to grow, memory capacity may still become a major limiting factor for future development. Analysts believe that if Apple further improves the memory configuration of iPhones in the future, the device's ability to run larger-scale AI models is expected to continue to improve, and the role of mobile phones as independent AI computing platforms will become increasingly important.
Comments