Microsoft has unveiled Maia 200, a next-generation AI accelerator purpose-built for inference workloads. Announced by Scott Guthrie, Executive Vice President of Cloud + AI, Maia 200 represents Microsoft’s most advanced first-party silicon to date—engineered to significantly improve the economics of AI token generation at cloud scale.

Designed for the AI era, Maia 200 combines cutting-edge silicon, advanced memory architecture, and cloud-native systems integration to deliver higher throughput, better efficiency, and lower total cost of ownership.

---

Built on 3nm, Designed for Low-Precision AI

Fabricated on TSMC’s 3nm process, each Maia 200 chip contains over 140 billion transistors and is optimized for low-precision AI workloads.

Key performance highlights:

• Over 10 petaFLOPS (FP4) performance

• Over 5 petaFLOPS (FP8) performance

• 750W SoC TDP envelope

• Native FP4 and FP8 tensor cores

Maia 200 delivers three times the FP4 performance of third-generation Amazon Trainium and exceeds the FP8 performance of Google’s seventh-generation TPU, positioning it as Microsoft’s most powerful inference accelerator yet.

The result: up to 30% better performance per dollar compared to the latest generation hardware currently deployed in Microsoft’s AI fleet.

---

Memory Architecture Built for Massive Models

High FLOPS alone don’t guarantee performance. Feeding data efficiently is equally critical.

Maia 200 addresses this challenge with:

• 216GB of HBM3e memory

• 7 TB/s memory bandwidth

• 272MB of on-chip SRAM

• Dedicated data movement engines

• Specialized DMA engines and high-bandwidth NoC fabric

This redesigned memory subsystem ensures high token throughput while minimizing bottlenecks—critical for running today’s largest language models and future-scale AI systems.

---

Part of Microsoft’s Heterogeneous AI Infrastructure

Maia 200 is integrated into Microsoft’s broader heterogeneous AI platform and will power multiple models, including the latest GPT-5.2 models from OpenAI.

It will bring performance-per-dollar improvements to:

• Microsoft Foundry

• Microsoft 365 Copilot

• Synthetic data generation pipelines

• Reinforcement learning workflows

Microsoft’s Superintelligence team will also leverage Maia 200 for synthetic data generation, helping accelerate domain-specific model improvements and next-generation AI research.

---

Scalable AI Networking with Standard Ethernet

At the systems level, Maia 200 introduces a two-tier scale-up network design built on standard Ethernet rather than proprietary interconnects.

Each accelerator provides:

• 2.8 TB/s bidirectional scale-up bandwidth

• Support for clusters of up to 6,144 accelerators

• Predictable collective operations at cloud scale

Within each tray, four Maia accelerators connect via direct, non-switched links for ultra-fast local communication. The same transport protocol extends across racks and clusters, simplifying scaling while reducing network hops and power consumption.

This architecture delivers both performance and cost efficiency—critical for hyperscale AI deployments.

---

Cloud-Native by Design

A defining feature of Maia 200 is Microsoft’s end-to-end co-development strategy. Long before physical silicon was available, Microsoft built a sophisticated pre-silicon simulation environment to model real-world large language model workloads.

This early co-design approach allowed:

• Optimization of silicon, networking, and system software as a unified stack

• Faster deployment timelines

• Reduced time from first silicon to datacenter rollout by more than half compared to similar AI infrastructure programs

Maia 200 also integrates directly into the Azure control plane, providing telemetry, diagnostics, security, and management at both chip and rack levels.

The accelerators are currently deployed in Microsoft’s US Central region (Des Moines, Iowa), with expansion to US West 3 (Phoenix, Arizona) and additional global regions to follow.

---

Developer Tools and SDK Preview

Microsoft is previewing the Maia SDK, offering developers tools to build and optimize models for Maia 200, including:

• Native PyTorch integration

• Triton compiler support

• Optimized kernel libraries

• Access to low-level programming capabilities

This approach enables both easy model portability and fine-grained hardware control across heterogeneous AI accelerators.

---

Final Thoughts

With Maia 200, Microsoft is not just building a faster AI chip—it’s delivering a vertically integrated AI infrastructure platform optimized for inference at cloud scale.

From silicon and networking to software and datacenter deployment, Maia 200 reflects Microsoft’s strategy to control the full AI stack, improve performance per dollar, and ensure scalable, efficient AI services for enterprise and hyperscale customers alike.