AI IP and software tools to address the escalating demand for on-device and edge AI processing have been unveiled by Cadence. The scalable Cadence Neo neural processing units (NPUs) deliver a range of AI performance in a low-energy footprint, said the company and this is claimed to bring new levels of performance and efficiency to AI SoCs.
The Neo NPUs deliver up to 80TOPS performance in a single core, in order to support both classic and new generative AI models. They can also offload AI/ML execution from any host processor, including application processors, general-purpose microcontrollers and DSPs. This is achieved with a simple and scalable AMBA AXI interconnect.
Cadence has also introduced the NeuroWeave software development kit (SDK) which it said provides developers with a “one-tool” AI software solution across Cadence AI and Tensilica IP products for no-code AI development.
“While most of the recent attention on AI has been cloud-focused, there are an incredible range of new possibilities that both classic and generative AI can enable on the edge and within devices,” pointed out Bob O’Donnell, president and chief analyst at TECHnalysis Research. For these intuitive, intelligent devices to be realised will need a flexible, scalable combination of hardware and software solutions with a range of power requirements and compute performance, “all while leveraging familiar tools” he believed. “New chip architectures that are optimised to accelerate ML models and software tools with seamless links to popular AI development frameworks are going to be incredibly important parts of this process,” he added.
The Neo NPUs are suitable for power-sensitive devices as well as high-performance systems with a configurable architecture. SoC architects will be able to integrate an optimal AI inferencing solution in a range of products, including intelligent sensors, IoT and mobile devices, cameras, hearables/wearables, PCs, AR/VR headsets and advanced driver-assistance systems (ADAS). New hardware and performance enhancements and key features/capabilities include:
The single core NPUs are scalable from 8GOPS to 80TOPS, with further extension to hundreds of TOPS with multi-core devices, said Cadence. They support 256 to 32K MACs per cycle, allowing SoC architects to optimise embedded AI to meet power, performance and area (PPA) tradeoffs.
Offloading of inferencing tasks from any host processor (e.g., DSPs, general-purpose microcontrollers or application processors) significantly improves system performance and power, said Cadence.
Support for Int4, Int8, Int16 and FP16 data types across a wide set of operations that form the basis of CNN, RNN and transformer-based networks allows flexibility in neural network performance and accuracy tradeoffs while the NPUs offer up to 20 times higher performance than the first-generation Cadence AI IP, with two to five time the inferences per second per area (IPS/mm2) and five to 10 times the inferences per second per Watt (IPS/W)
Upgrades to the common software toolchain include the NeuroWeave software development kit (SDK). Providing customers with a uniform, scalable and configurable software stack across Tensilica DSPs, controllers and Neo NPUs to address all target applications, the NeuroWeave SDK streamlines product development and enables an easy migration as design requirements evolve. It supports many industry-standard domain-specific ML frameworks, including TensorFlow, ONNX, PyTorch, Caffe2, TensorFlow Lite, MXNet and JAX for automated end-to-end code generation, together with Android Neural Network Compiler; TF Lite Delegates for real-time execution and TensorFlow Lite Micro for microcontroller-class devices.
The Neo NPUs and the NeuroWeave SDK are expected to be in general availability beginning in December 2023.
