With the development of artificial intelligence technology and applications, as a dedicated chip in the field of deep learning and artificial intelligence, NPU is gradually moving to the center of the stage.

NPU, a neural network processing unit, is used to efficiently perform calculations of a neural network, and usually has an optimized hardware architecture, such as a vector processing unit, a matrix multiplication unit, a convolution unit, and an activation function unit, and can perform large-scale matrix operations and convolution operations at a hardware level to improve the computational efficiency of the neural network.

At present, various AI algorithms mainly use deep neural networks and other algorithms to simulate human neurons and synapses, and NPU can achieve higher efficiency and lower energy consumption to process artificial neural networks, random forests and other machine learning algorithms and deep learning models. Today, a number of mobile phone manufacturers have been equipped with NPU,AIPC will also through the "CPU (central processing unit) + NPU + GPU (graphics processing unit)" to create local hybrid computing. So, will NPU be another tuyere after GPU?

NPU: High performance, low power consumption, better reasoning

"Compared with CPU and GPU,NPU has the advantages of high performance, low power consumption, easy programming, lower development threshold, and support for multiple languages and frameworks to facilitate developer model development and deployment." IDC China analyst Du Yunlong told First Financial.

Traditional CPUs are usually used to perform general-purpose computing tasks, but for processing large-scale neural network calculations, the processing efficiency of the CPU is relatively low.

GPU is usually used as a coprocessor of CPU. Compared with CPU, GPU has fewer logic operation units, has obvious advantages in processing parallel computing, can share the calculation amount of CPU, and is also the most widely used accelerated computing chip on the data center side.

NPU adopts the architecture of "data-driven parallel computing", simulates human neurons and synapses at the circuit layer, and is especially good at processing massive multimedia data such as video and images. Different from the von Neumann architecture followed by CPU and GPU, NPU can realize the integration of storage and calculation through synaptic weights and improve the operation efficiency, so it is better at reasoning than GPU. And the NPU chip design logic is simpler, with significant energy savings when dealing with inference workloads.

However, since GPU already has a perfect ecology such as NVIDIA CUDA, Du Yunlong believes that the lack of a perfect ecological environment like GPU is the biggest bottleneck for NPU penetration.

according to IDC data, GPU will still be the mainstay of data center computing acceleration in China in 2022, with GPU accounting for 86% of shipments in the artificial intelligence chip market. NPU accounted for 12%, a significant increase over the past.

mostly used for end side and edge side

Unlike the cloud side, the end side is more sensitive to power consumption, and the demand for low-power chips is more obvious. Therefore, with the artificial intelligence application scenario landing, NPU easy to develop, high efficiency, low power consumption and other advantages gradually highlighted. It is generally believed in the industry that under the explosion of large computing power demand, the computing power demand on the cloud side will be transmitted to the end side. At present, the most common way to realize the computing power of intelligent terminal is to build NPU module in SoC chip.

"NPU is a chip specially designed for artificial intelligence applications. At present, NPU is usually used for more edge-side and end-side scenes, such as face recognition, face unlocking, image processing, etc." Du Yunlong said.

AIPC is expected to be listed in batch in 2024, and AIPC is generally equipped with NPU, which together with CPU and GPU constitute AIPC's core computing power.

Intel recently released the 14th generation Core Ultra mobile processor with built-in NPU. Intel said that in 2024, more than 230 models will be equipped with Core Ultra. Apple will also release a MacBook with M3 processor in 2024 and reveal that the NPU performance of its M3 processor has improved by 60% compared with M1.

the mobile phone started to carry NPU earlier. Huawei first adopted Cambrian NPU in Mate10 and later adopted the self-developed Da Vinci NPU in 990 series. Apple has joined the Neuralengine since A11SoC. In the newly released A14SoC, NPU computing power has been greatly improved. Neuralengine, the combination of machine learning accelerator on CPU can greatly improve AI application experience.

In addition to smartphones and AIPC, NPU is also used in automobiles, edge-side devices such as XR and various IoT smart terminals. As large models enter thousands of industries, end-side AI, edge-side AI penetration increases, will also bring more NPU demand.

Domestic chip manufacturers in-depth layout

At present, domestic chip manufacturers are struggling to develop NPU to grasp the AI wave. Chip companies, represented by Ali Flat Head, have launched artificial intelligence reasoning chips for data center AI applications, and currently include light 800 that have been successfully applied in data centers, edge servers and other scenarios.

Domestic SoC manufacturers are also in-depth layout of NPU to enrich and enhance SoC artificial intelligence processing capabilities.

A typical application of NPUs in SoCs is machine vision. Taking the new generation machine vision schemes RV1106 and RV1103 of Ruixinwei (603893.SH) as examples, the two chips have significantly upgraded the performance of NPU, ISP, video coding, audio processing, etc., with high integration and cost performance, and can provide excellent edge AI computing power while low standby power consumption. Ruixin Micro's high-performance RV1126 has quad-core ARMCortex-A7 and RISC-VMCU,2.0TopsNPU, and in-vivo detection rate can reach as high as 98.48. The latest flagship chip RK3588 supports NPU computing power of 6Tops.

jingchen shares (688099.SH) A311D is equipped with 5TOPS high-performance NPU while adopting high-performance A73 core, which can be widely used in various medium and high-end AIOT (artificial intelligence internet of things) devices.

in terms of NPU IP, core original shares (688521.SH) acquired graphics processor (GPU)IP through the acquisition of map core in the United States in 2016, and developed NPU IP independently on this basis. Xinyuan shares previously told First Finance and Economics that at present, in the AIoT field, the company's neural network processor IP for artificial intelligence has been adopted by more than 100 chips from more than 50 customers and is used in 10 application fields such as the Internet of Things, wearable devices, security monitoring, servers, automotive electronics, etc.