Cerebras Systems
California-based Cerebras Systems has launched the Wafer Scale Engine 3 (WSE-3), an AI chip that delivers twice the performance of its 2021 predecessor the WSE-2.
The 5nm-based, 4 trillion transistor WSE-3 chip includes 900,000 AI optimised compute cores and is composed of a silicon wafer measuring 8.5 by 8.5 inches.
This third-generation chip will be used to power the Cerebras CS-3 AI supercomputer, delivering 125 petaflops of peak AI performance and training AI models up to 24 trillion parameters.
According to Cerebras, these 24 trillion parameter models can be stored in a single logical memory space without partitioning or refactoring, dramatically simplifying training workflow and accelerating developer productivity. It claims that training a one trillion parameter model on the CS-3 is as straightforward as training a one billion parameter model on GPUs.
“When we started on this journey eight years ago, everyone said wafer-scale processors were a pipe dream. We could not be more proud to be introducing the third generation of our ground-breaking wafer scale AI chip,” said Andrew Feldman, CEO and co-founder of Cerebras.
“WSE-3 is the fastest AI chip in the world, purpose-built for the latest cutting-edge AI work. We are thrilled to bring WSE-3 and CS-3 to market to help solve today’s biggest AI challenges.”
The company states that, compared to power-hungry GPUs, the CS-3 doubles performance but stays within the same power envelope. It also requires 97% less code than GPUs for large language models. For instance, a standard implementation of a GPT-3-sized model required just 565 lines of code on Cerebras.
Cerebras has formed partnerships with a number of interested parties, including a strategic partnership with G42, an AI development company. G42 is currently developing the Condor Galaxy 3 supercomputer, which will be made up of 64 Cerebras CS-3 AI system “building blocks” that are powered by the WSE-3 chip. Once developed, this 8-exaFLOP supercomputer will bring G42’s total system production of AI compute to 16 exaFLOPs.
“Our strategic partnership with Cerebras has been instrumental in propelling innovation at G42, and will contribute to the acceleration of the AI revolution on a global scale,” said Kiril Evtimov, group CTO of G42.
