Claimed to be the largest chip in the world, the Cerebras wafer scale engine (WSE) measures 216 x 216mm (8.5 x 8.5 inch). At 46,225mm2 the chip is 56x larger than the biggest graphics processing unit (GPU) ever made, claims Cerebras.
It has 400,000 cores and 18Gbyte on-chip SRAM. The large silicon area, more than the largest graphics processing unit, enables the WSE to provide more compute cores, tightly coupled memory for efficient data access, and an extensive high bandwidth communication fabric for groups of cores to work together, claims Cerebras.
The WSE contains 400,000 sparse linear algebra (SLA) cores. Each core is flexible, programmable, and optimised for the computations that underpin most neural networks. Programmability ensures the cores can run all algorithms for constantly changing machine learning operations.
The cores on the WSE are connected via the Swarm communication fabric in a 2D mesh with 100 petabytes (Pbytes) per second of bandwidth. The Swarm on-chip communication fabric delivers breakthrough bandwidth and low latency at a fraction of the power draw of traditional techniques used to cluster GPUs, says Cerebras. It is fully configurable. Software configures all the cores on the WSE to support the precise communication required for training the user-specified model. For each neural network, Swarm provides an optimised communication path.
The 18Gbyte of on-chip memory is accessible within a single clock cycle, and provides 9 Pbytes per second memory bandwidth. This is 3,000 times more capacity and 10,000 times greater bandwidth than the leading competitor, claims Cerebras. The WSE provides moree cores, more local memory and enables fast, flexible computation, at lower latency and with less energy than other GPUs, concludes Cerebras.
