Abstract:
Microsoft officially announced the third-generation self-developed artificial intelligence chip Maia 300, and plans to deploy it into a production environment starting in 2027 to support AI reasoning tasks behind products and services such as Copilot. Microsoft said that compared with the current leading GPU system, Maia 300 is expected to be able to process 40% more tokens per megawatt of power; in a complete rack configuration consisting of 96 accelerators, its rack-level token throughput can be up to 7 times that of the previous generation Maia 200.
This release marks Microsoft’s further investment in self-developed AI chips. As the use of artificial intelligence services continues to grow, Microsoft hopes to reduce operating costs, improve energy efficiency, and reduce dependence on GPUs from suppliers such as Nvidia and AMD through self-designed accelerators.
Scott Guthrie, executive vice president of Microsoft's cloud computing business, said that the company plans to expand the deployment scale of the Azure Maia series chips to the multi-gigawatt level starting in 2027 and make it the main AI inference system for Microsoft products and services.
This goal means that Microsoft hopes to gradually upgrade self-developed chips from part of the existing infrastructure to a core computing platform that supports its own AI services. However, this does not mean that Microsoft will completely stop purchasing GPUs from Nvidia or AMD. Microsoft is likely to continue to use multiple hardware platforms for training and inference tasks that require different computing capabilities.
Maia 300 is the third generation product of Microsoft's Maia series, and it is also the next generation chip launched after the announcement of Maia 200 in January this year. Maia 200 is currently used for AI workloads in multiple Microsoft production environments, including Microsoft Copilot, GitHub Copilot, Microsoft Foundry, and Microsoft's self-developed artificial intelligence models.
Guthrie said that judging from the operating indicators of actual AI services, Maia 200 is already one of the best-performing AI infrastructures within Microsoft. The new generation Maia 300 is specifically designed for ever-changing AI workloads, especially those that need to handle long-context reasoning, real-time conversations that are highly sensitive to latency, and AI agent tasks that continuously execute multiple steps.
These applications place different requirements on hardware than traditional AI inference.
As large-scale language models are gradually used in programming assistants, enterprise automation, and complex task execution, the context that AI systems need to process is getting longer and longer, and a single task involves more and more model calls and tool operations. For cloud service providers, in addition to the performance of the model itself, response latency, number of concurrent requests, memory capacity, and the actual cost of each inference must also be considered.
Therefore, the design focus of Maia 300 is not just to increase peak computing power, but also to improve overall efficiency in a real-life business environment that continuously runs. Microsoft hopes to specifically optimize the chip architecture to make it more suitable for handling the computing load generated by new generation inference models and AI agents.
In terms of chip manufacturing, Maia 300 adopts TSMC's N2P advanced process and introduces a core-based design architecture. Compared with traditional designs that integrate different functions on a single large chip, the core chip architecture can divide functions such as computing, memory controller, and input and output into different modules, and then design them according to their respective characteristics.
Microsoft Azure Hardware and Infrastructure Team stated that this design allows more silicon area in the chip to be used for computing performance, while allowing different functional modules to adopt technologies that are more suitable for their own needs.
This idea has practical significance. For AI accelerators, the computing unit is important, but how data is transmitted between the computing unit, memory and external devices will also affect overall performance. If the computing unit needs to wait for data to arrive, even if it has high theoretical computing power, it will not be able to give full play to its performance.
By optimizing computing, memory control, and input and output functions separately, Microsoft hopes to improve the overall resource utilization efficiency of the chip and achieve a better balance between performance, power consumption, and manufacturing costs.
The use of advanced processes will also help improve transistor density and energy efficiency, but the final performance of the chip still depends on multiple factors such as microarchitecture, memory subsystem, interconnect design, and software optimization. The performance data released by Maia 300 currently mainly comes from Microsoft's own introduction, and the actual effect still needs to be evaluated based on different models, workloads, and deployment conditions.
In the field of AI reasoning, Token throughput is one of the important indicators to measure system efficiency. Token is the basic data unit used by large language models when processing text, usually corresponding to a word, word fragment or other text unit. How many tokens the system can process per unit of time directly affects how many users it serves at the same time and the computing resources required to complete the task.
Microsoft expects the Maia 300 to handle 40% more tokens per megawatt than leading GPU systems. This means that under the same power budget, Microsoft is expected to process more AI requests, or reduce power consumption while providing the same service capabilities.
For cloud service providers with large-scale data centers, this efficiency improvement may bring significant economic value. AI data centers not only need to purchase expensive accelerators, but also must bear the costs of power supply, cooling, network equipment, and infrastructure construction. As model usage continues to grow, power supply and data center capacity are becoming important factors restricting the expansion of AI services.
However, Token efficiency per megawatt is not the only indicator for measuring AI chips. The parameter scale, context length, accuracy settings and inference methods of different models will all affect the test results. To comprehensively compare products such as Maia 300 and NVIDIA GPUs, it is also necessary to consider latency, model compatibility, software ecosystem, deployment costs, and stability in actual business.
Microsoft also announced a more attractive indicator: In a complete rack configuration consisting of 96 accelerators, the rack-level token throughput of Maia 300 can reach up to 7 times that of Maia 200.
This figure shows that Microsoft not only hopes to improve the efficiency of a single chip, but also optimizes the computing power of the entire rack. But it describes rack-level throughput under a specific configuration. It does not mean that Maia 300 is 7 times faster than Maia 200 in all tasks, nor can it be directly deduced that the performance of each chip has improved by the same proportion.
For cloud computing platforms, rack-level performance is especially important. Real-world deployments typically consist of a large number of accelerators, with the chips exchanging data via high-speed interconnects and jointly processing model requests. Even if a single chip is powerful, the entire system can still be limited if the interconnect bandwidth, memory capacity, or scheduling efficiency are insufficient.
Therefore, Microsoft is trying to conduct overall optimization from chip design, rack configuration to software and model operating environment, rather than simply pursuing the theoretical computing power of the processor.
Microsoft AI chief Mustafa Suleyman said the company must start investing from the bottom of the entire technology stack, including chip design. He believes that when AI services reach such a huge scale, Microsoft needs to have its own platform and let chips and self-developed models be co-designed and optimized.
He particularly emphasized that allowing Microsoft's self-developed AI models to run on Maia chips is an important task for the Microsoft AI team.
This statement reveals another goal of Microsoft's development of self-developed chips: to combine hardware, models and cloud services to form a closer collaborative relationship.
Traditional cloud service providers can deploy AI models by purchasing general-purpose GPUs, but if they can design their own accelerators, they can optimize the computing characteristics, memory requirements and inference processes of their own models. In turn, the model team can adjust the model structure and operation method according to the chip architecture, thus improving overall efficiency.
Google and Amazon have invested in self-developed AI chips for many years. Google has a series of TPU accelerators, and Amazon has developed chips such as Trainium to support its own cloud computing and artificial intelligence services. Microsoft launched the Maia series to strengthen its infrastructure capabilities against this competitive background.
For Microsoft, the strategic value of self-developed chips is not only to reduce procurement costs, but also to improve supply chain autonomy, optimize data center energy utilization, and have greater hardware design control when AI models and services are rapidly iterated.
At present, Microsoft still maintains close cooperation with NVIDIA, and NVIDIA GPUs continue to play an important role in the field of AI training and inference. Microsoft's simultaneous development of the Maia series does not mean that the relationship between the two parties will inevitably change fundamentally, but it reflects that large cloud service providers are reducing their dependence on a single hardware platform through self-developed chips and multi-vendor strategies.
Microsoft also hopes to use self-developed chips to improve the competitiveness of its AI models. Suleiman said that the Maia platform is an important part of Microsoft's AI technology system and can help the company further develop its own models and improve its competitiveness against competitors such as Google DeepMind, OpenAI and Anthropic.
As AI models gradually evolve from simple text generation tools to agent systems that can call tools and perform complex operations, the importance of hardware infrastructure is also increasing. Long-context reasoning and multi-step task execution often involve longer computational processes and a large number of model calls. Only by achieving a better balance between performance, latency, and cost can enterprises scale their services in a sustainable manner.
Maia 300 is designed in response to this trend. Microsoft hopes to enable more AI tasks to run at lower costs through higher throughput and energy efficiency, while providing computing support for Copilot, GitHub Copilot, and future AI agent services.
However, for Maia 300 to truly become the main force of Microsoft's AI infrastructure, it still needs to go through the process from chip manufacturing, system integration to large-scale deployment. Microsoft plans to start putting it into production in 2027 and gradually expand the scale of deployment. By then, the chip's actual production capacity, system stability, software adaptation, and integration with existing data centers will all affect its final application scope.
Overall, the launch of Maia 300 shows that Microsoft is elevating AI chips from auxiliary infrastructure projects to a core component of its own artificial intelligence strategy. By using TSMC's advanced processes, core granular design, and system-level optimization for inference workloads, Microsoft hopes to improve the AI service capabilities it can provide per unit of electricity and reduce its dependence on external GPU suppliers.
If Microsoft can expand the production deployment of Maia 300 as planned in 2027 and achieve the expected efficiency improvements in real business, this series of chips is expected to become an important tool for it to control AI computing costs and expand service scale. But its long-term competitiveness ultimately depends on actual operating performance and whether Microsoft can establish a sustained and effective collaborative optimization system between hardware, models and software.
Comments