Key Findings
Thermal management issues have become acutely prominent in advanced-packaged AI chips, with thermal coupling between chiplets in 2.5D, 3D IC, CoWoS, and HBM architectures posing a significant hurdle. TSMC has unveiled plans to integrate around 10 large computing chips and 20 layers of HBM within a 14x reticle-sized CoWoS platform by 2028. This ambitious integration is expected to dramatically increase thermal density, leading microchannel cooling to be identified as the primary thermal management technology for future high-performance AI packages.
Technical / Clinical Details
As AI chip performance escalates, the power consumption and heat generation per unit area have risen sharply. In chiplet architectures, the tight integration of multiple logic dies and HBM stacks exacerbates thermal coupling, leading to hot spot formation and overall temperature increases. TSMC’s evolution of CoWoS (Chip-on-Wafer-on-Substrate) technology further intensifies this challenge. The planned 14x reticle-sized CoWoS for 2028 represents a substantial increase in area compared to current standard CoWoS, aiming to integrate up to 10 large computing chips and 20 layers of HBM into a single package. At such high integration densities, conventional air cooling or heat sinks are insufficient. This is where microchannel cooling technology steps in. This technique involves creating minuscule channels (microchannels) directly within or immediately beneath the chip to circulate a cooling fluid, achieving extremely high heat transfer efficiency. This enables efficient heat removal from the chip, maintaining stable operating temperatures crucial for sustained performance.
Background & Context
The increasing complexity and scale of AI models place immense pressure on the computational capabilities of semiconductor chips. With the deceleration of Moore’s Law, the semiconductor industry has shifted towards advanced packaging technologies like 2.5D/3D packaging, chiplets, and HBM to achieve performance gains. However, these technologies, by vertically and horizontally integrating chips at high densities, inherently lead to elevated thermal densities, making thermal management the system-level bottleneck. Without effective thermal management solutions, the full potential of AI chips cannot be realized. Roadmaps from leading foundries like TSMC, emphasizing cooling technologies, signal a clear direction for the entire industry.
Strategic Significance & Outlook
Microchannel cooling technology is set to become an indispensable component for future high-performance AI chips and HPC systems. Its commercial realization will necessitate further innovations in materials science, fluid dynamics, and precision manufacturing techniques. Key areas of focus will include selecting optimal cooling fluids, optimizing channel designs, and developing reliable pumping and piping systems. Integrating cooling systems within the package and their synergy with power delivery are also crucial considerations. Successful deployment of microchannel cooling is expected to break current thermal design limits for AI chips, open new frontiers in AI processing capabilities, and contribute significantly to energy efficiency improvements in data centers worldwide.
Source: https://eu.36kr.com/en/p/3978316580617224
Get our weekly technology intelligence — free
Receive an infographic that lets you judge at a glance whether each field’s analysis report is worth reading.
Subscribe Free — Weekly Tech Intelligence
By subscribing, you’ll receive Troy-Technical’s weekly technology intelligence newsletter.
- Your email and selected fields are used only to deliver the newsletter.
- We never share your information with third parties.
- You can unsubscribe anytime via the link in each email.
See our Privacy Policy for details.
Takes about a minute · Unsubscribe anytime
Comments