Abstract:
For the past fifty years, researchers of semiconductor technology have felt like walking inside a tunnel. There was a single path forward – two-dimensional down-scaling of device sizes, also referred to as 2D miniaturization. With device features approaching atomic scale, semiconductor technology has reached the exit of this tunnel. The future is bright at the exit of the tunnel as there are many possible paths that create new opportunities for architectures that are extremely difficult (or even impossible) to implement using existing technology approaches that rely mainly on 2D miniaturization. It is no longer possible to draw a clear boundary between design and fabrication, nor can we treat wafer fabrication and advanced packaging as separate technologies. Many innovations across the entire system stack – from architectures to circuits, devices, packaging, fabrication processes, and materials – will provide large multiplicative benefits at the system level. There will be a change to the supplier-integrator and fabless-foundry ecosystems that have been in place for over 30 years.
Specifically, AI computation benefits tremendously simply by having more on-chip memory capacity. To derive the highest energy efficiency, not only the chip architecture needs to be application domain specific, but the memory technologies must also be application domain-specific to capture the highest benefits while incurring the lowest impact on latency, bandwidth and cost. More importantly, these memory technologies can be integrated on Si logic chips to arrive at a computing system that integrates multiple heterogeneous device technologies in a 3D monolithic, stacked, assembled integrated circuit that smoothly transition from wafer level to package level technologies. Advanced packaging technologies have the potential to further increase the memory bandwidth by about 1,000×.
Acknowledgements: This work is supported in part by Department of Defense Microelectronics Commons California-Pacific-Northwest AI Hardware Hub, National Science Foundation (award number 2235329), SRC JUMP 2.0 CHIMES Center and PRISM Center, and member companies of the Stanford SystemX Alliance, Stanford Non-Volatile Memory Technology Research Initiative (NMTRI), and the Stanford Differentiated Access Memory (DAM) industry affiliate programs. We gratefully acknowledge the sustained support of our industry sponsors and collaborators. Abstract prepared with assistance from Shuhan Liu (Stanford, Yale).