The Yield Imperative: Turning AI Infrastructure into Useful Intelligence

Redefining Yield Across AI Infrastructure Layers

Traditionally defined as the usable output produced from raw silicon wafers, yield must now be measured across an entire computing stack. According to Rani Borkar, who leads core hardware and infrastructure planning for Microsoft’s cloud computing platform, capacity at any single layer is only a starting point. The true measure of progress is how effectively layers work together to produce useful output from the system as a whole. Microsoft outlines two paths for future progress: an evolutionary path focused on incremental efficiency within current architectures, and a transformational path involving new materials, architectures, and system designs. Drawing on historical precedents such as the shift to multicore processors when CPU clock speeds hit a power wall, the company asserts that current scaling limits require holistic system co-design rather than mere additions of silicon and power.

Memory Constraints and System-Level Solutions

In artificial inference workloads, memory bottlenecks dictate overall system performance. Multi-turn agentic loops require holding larger models and preserving extended contexts over minutes or hours. According to Microsoft, treating memory strictly as a component supply problem fails to address these performance ceilings. Engineering useful yield from memory requires coordinated changes across multiple layers. Model architecture, data science techniques, and data compression reduce the KV cache, which stores working context during generation. Simultaneously, software manages memory hierarchies, silicon layout optimizes data movement, and compilers place data closer to compute hardware to extract more intelligence from every available byte.

Cluster-Level Networking and Fleet Efficiency

At the cluster level, computing power relies on thousands of chips operating in unison. Productivity depends on congestion management, workload placement, and failure recovery rather than raw link speed alone. When architecting the Azure Maia platform, Microsoft bypassed traditional separate scale-up and scale-out fabrics. The engineering team integrated network interface card functionality directly into the silicon, developed a custom transport layer, and implemented a two-tier scale-up network. This unified fabric delivers consistent performance across dense inference clusters while reducing the total network hardware required to run the system.

Power Co-Design from Grid to Chip

Rising power densities have transformed data centers, with individual racks scaling from tens of kilowatts to hundreds of kilowatts, and campuses operating on gigawatt scales. To manage distribution losses, the industry is adopting solid-state transformers and 800-volt direct current power delivery. At the silicon level, Microsoft’s Arm-based Azure Cobalt 200 server CPU features independent voltage and frequency controls for every core. Paired with software-based, per-virtual-machine power capping, this design allows targeted adjustments that maintain critical workload performance while fitting more servers into a fixed power envelope. According to Microsoft, achieving full yield requires the entire technology ecosystem—including hyperscalers, silicon providers, equipment makers, utilities, and software developers—to collaborate across traditional boundaries. The ultimate metric of progress remains the translation of infrastructure capacity into broad economic productivity, scientific discovery, and societal empowerment.