Large Language Models (LLMs) have demonstrated exceptional performance across numerous generative AI applications, but require large model parameter sizes. These parameters range from several billion to trillions, leading to significant computational demands for both AI training and inference. The growth rate of these computational requirements significantly outpaces advancements in semiconductor process technology. Consequently, innovative IC and system design techniques are essential to address challenges related to computing power, memory, bandwidth, energy consumption, and thermal management to meet AI computing needs.
In this talk, we will explore the evolution of LLMs in the generative AI era and their influence on AI computing design trends. For AI computing in data centers, both scale-up and scale-out strategies are employed to deliver the huge computational power required by LLMs. Conversely, even smaller LLM models for edge devices demand more resources than previous generations without LLMs. Moreover, edge devices may also act as orchestrators in device-cloud collaboration. These emerging trends will significantly shape the design of future computing architectures and influence the advancement of circuit and system designs.
