AI Chip Race Shifts to Advanced Packaging as 130M Accelerators Ship; TSMC’s CoWoS Dominates, Intel Pursues Alternatives

The Korea Economic Daily South Korea

Overview

The competitive landscape in the AI semiconductor market is shifting from process nodes to advanced packaging, with over 130 million AI accelerators projected to ship with advanced-packaged on-compute memory in the next five years. While TSMC’s CoWoS method leads, Intel is developing alternative technologies like EMIB, ZAM, and XBM to address packaging and memory integration challenges. The efficient consolidation of memory and compute is becoming paramount, intensifying competition in advanced packaging. This has emerged as a new bottleneck determining AI chip performance and cost-efficiency.

In Depth

Key Findings

The focal point of competition within the AI semiconductor market is rapidly transitioning from traditional process node development to high-performance ‘advanced packaging’ technologies. Over the next five years, more than 130 million AI accelerators are projected to be shipped with advanced-packaged on-compute memory, particularly High Bandwidth Memory (HBM). While TSMC’s CoWoS (Chip-on-Wafer-on-Substrate) technology currently leads this sector, Intel is actively developing its proprietary alternative technologies, including EMIB, ZAM (co-developed with SoftBank SAIMEMORY), and XBM, to address the challenges of efficient memory and compute integration. This shift is emerging as a critical bottleneck determining the performance and cost-efficiency of AI chips, thereby fueling intense competition in the advanced packaging market.

Technical / Clinical Details

To fully unleash the potential of AI chips, it is imperative to mitigate bottlenecks in data transfer rates between the processor (CPU/GPU) and memory. Advanced packaging technologies are developed precisely to overcome this ‘memory wall,’ by stacking HBM in close proximity to the processor, significantly reducing data transfer latency and energy consumption.

• TSMC’s CoWoS: This technology integrates HBM and logic chips via a silicon interposer, representing the current industry standard. It achieves exceptionally high bandwidth and low power consumption. However, CoWoS production capacity continues to lag behind demand, contributing to AI chip supply constraints.
• Intel’s EMIB (Embedded Multi-die Interconnect Bridge): A technology that connects multiple dies using small silicon bridges, offering greater cost-efficiency and manufacturing flexibility compared to CoWoS.
• Intel’s ZAM (Zero-Altitude Memory): Co-developed with SoftBank SAIMEMORY, this technology aims for even tighter memory integration.
• Intel’s XBM (eXtended Bandwidth Memory): Currently under development as a next-generation memory integration technology to provide even higher bandwidth.

These technologies also aim to address challenges such as the ‘performance wall’ (limits of computational efficiency) and the ‘copper wall’ (limits of traditional wiring technologies), thereby enhancing the overall efficiency of the chip.

Background & Context

The rapid advancement of AI, particularly generative AI, has led to an exponential surge in demand for AI chips. It has been revealed that the manufacturing cost of leading-edge AI accelerators like the NVIDIA H100 reaches $3,320, and the GB200 superchip costs $14,200, with over 60% of these expenses attributed to HBM memory and advanced packaging. This underscores that packaging technology plays a decisive role not only in enhancing AI chip performance but also in determining their manufacturing cost and availability. Major semiconductor manufacturers are investing heavily in securing HBM memory and expanding advanced packaging capabilities, striving to establish dominance in this new competitive arena.

Strategic Significance & Outlook

Advanced packaging will continue to be one of the most critical drivers for AI chip development in the coming years. Beyond TSMC and Intel, other major semiconductor manufacturers like Samsung are also expected to develop and strengthen their proprietary packaging technologies, vying for market share. This competition is poised to further elevate AI chip performance and facilitate the creation of smaller, more power-efficient AI systems. Moreover, efficient memory integration and packaging technologies will be key to accelerating the adoption of AI in new application areas such as autonomous driving, edge AI, and robotics. The semiconductor industry will continuously pursue innovations in materials science, design, and manufacturing processes to overcome this trillion-dollar bottleneck.

Get our weekly technology intelligence — free

Receive an infographic that lets you judge at a glance whether each field’s analysis report is worth reading.

Subscribe Free — Weekly Tech Intelligence

By subscribing, you’ll receive Troy-Technical’s weekly technology intelligence newsletter.

• Your email and selected fields are used only to deliver the newsletter.
• We never share your information with third parties.
• You can unsubscribe anytime via the link in each email.

See our Privacy Policy for details.

Agree & Continue ▶

Takes about a minute · Unsubscribe anytime