In the past two days, IBM made a low-key news and launched a brain-inspired chip "NorthPole". Compared with the NVIDIA H100 GPU implemented on the 4nm node, NorthPole's energy efficiency has increased five times, making it the most powerful AI chip in the world. Such incredible performance, but there is very little news about this chip in China. So, who is it?

Just put the brain into the chip, right?

First of all, IBM’s “Arctic” NorthPole is a brain-inspired chip. We need to first understand what it is.brain-like chip.

The so-called brain-like chip, as the name suggests, is a chip that highly simulates the computing principles of the human brain. Based on the understanding of modern neuroscience, we repeatedly think about how to imitate the computing of the human brain from transistors to architectural design, algorithms and software. If a brain-like chip is made more like a human brain, it will be given a new name——Neuromorphic Computing.

There are many differences between the way humans think and today's traditional chips. For example, humans have no separate memory, no dynamic random access memory, no hash hierarchy, no shared memory, etc.

"Storage" and "processor" are intricately intertwined in the human brain, and there are "neurons" in the structure of the human brain.In computers, digital cores communicate event-based information with each other, called impulses, similar to the way the human brain transmits information.

Human beings get inspiration from the human brain and create such chips to improve performance and become true "artificial intelligence", even infinitely close to humans and gain the ability to think.

Of course, the human brain is very complex, and the IC devices manufactured by people also have advantages, so people's ultimate goal is to integrate the advantages of the two.

All in all, to explain in human terms, a brain-like chip is a chip designed and manufactured by combining the advantages of biological brains and artificial devices, and it can think and learn by itself like humans.


If you want to build such a chip, you don't just need to change the device structure. Instead, you need to drive changes in algorithms and applications from materials, devices, circuits, and architecture. In a word, it is a combination of the most advanced technologies to create such a chip.

Although there are many implementation paths, this chip technology is too difficult. Not to mention whether it is easy to manufacture, it is difficult to design it. Therefore, it is still in the development stage and has not yet reached the commercialization level.

But the author understands that the reason why brain-like chips have not yet been commercialized on a large scale isFirst, because design problems depend on, taking the CMOS type that both Intel and IBM are optimistic about, it is difficult to solve problems such as chip interconnection of multiple all-digital asynchronous designs, the effectiveness and timeliness of chip connections, as well as software layer interconnection computing, distributed computing and flexible partitioning;Second, manufacturing, software, and ecology must be completely overturned. Although the silicon-based transistor route is partially reusable, the bottom layer cannot be completely copied, which further exacerbates the difficulty of large-scale commercialization.

So, why do we spend so much effort on brain-inspired chips?

Brain-inspired chips are so awesome. The author learned that in some cases, perfect neuromorphic chips can solve problems with 1,000 times lower energy consumption than traditional solutions. This means that we can pack more chips to solve larger-scale problems under a fixed power budget.

The ultra-high energy efficiency ratio of brain-like chips is enough for us to make full efforts to research and make breakthroughs. Take the most typical example. AlphaGo defeated humans in chess, but humans only used 20 watts of brain energy, while AlphaGo used 20,000 watts.


The fields and opportunities involved in neuromorphic chips, picture source | Nature Computational Science

Structurally, the current brain-inspired chips around the world are basically the same. They are composed of three parts: neuron calculation, synaptic weight storage, and routing communication. They also use the Spiking Neural Network (SNN) model.

However, based on materials, devices, and circuits, they are divided into three schools: neuromorphic systems dominated by analog circuits (digital-analog hybrid CMOS type), fully digital circuit neural systems (digital CMOS type), and digital-analog hybrid neuromorphic systems based on new devices (memristor is a candidate technology).

Globally, the institutions involved in the development of neuromorphic computing chips mainly include three categories: technology giants represented by Intel, IBM, Qualcomm, etc., universities/research institutions represented by Stanford and Tsinghua University, and start-ups.

According to the author’s previous conversations with Intel Research Institute,Digital CMOS type is currently the easiest form to industrialize. On the one hand, the technology and manufacturing are highly mature. On the other hand, there are no concerns and limitations of analog circuits..

Of course, it needs to be emphasized that the digital CMOS type is only the earliest brain-like chip. It is not a neuromorphic device that completely simulates the human brain. It can only be regarded as a chip that draws on the neuromorphic concept. But just by borrowing from the human brain, this chip can crush any chip in the world. IBM's NorthPole is such a digital CMOS brain-inspired chip.


Main types and research and development progress of brain-inspired chips, tabulation丨Electronic Engineering World

What is the level of IBM's chips?

Let me talk about the conclusion first, it should beSo far, it is the highest-level brain-inspired chip in human beings, and the research on brain-inspired chips has reached a new level..

IBM began researching brain-inspired computing in 2008, and achieved its first breakthrough in 2011: IBM's first-generation neurosynaptic computer chip. The researchers created two such chip models: one containing 262,144 programmable synapses and one containing 65,536 learnable synapses. Through tests, they demonstrated that they can perform simple functions such as navigation, machine vision, pattern recognition, associative memory and classification.

Until 2014, IBM published an article in Science magazine, showing the world its epoch-making technological progress: a brain-like processor with 1 million neurons that met the DARPASyNAPSE project specifications, namely TureNorth, which set off a craze in brain-like chip research at the time.

After eight years of silence, "NorthPole" came out. NorthPole was built on IBM's last brain-inspired chip, TrueNorth. At that time, TrueNorth had an energy efficiency ratio that was four orders of magnitude lower than traditional microprocessors. Comparing the two, you can intuitively feel the changes in IBM technology:

A single TrueNorth chip contains 4,096 computing cores, enabling dynamic mapping of synapses and neuronal arrangements. An attractive feature of the IBM TrueNorth system is that a single chip is composed of 5.4 billion transistors and consumes only 70mW of power density, accounting for only 1/10,000 of a traditional computing unit;

NorthPole is manufactured using a 12nm node process. It integrates 22 billion transistors within 800 square millimeters and has 256 cores. At 8-bit precision, each core can perform 2048 operations per cycle; at 4-bit and 2-bit precision, the number of operations may be doubled and quadrupled respectively. The speed of running image recognition algorithms driven by artificial intelligence is 22 times that of similar chips currently on the market, and the energy efficiency is 25 times that of similar chips. Without using the most advanced technology, the energy consumption of NorthPole chips is 1/5 of artificial intelligence chips using the most advanced technology. Overall, NorthPole is approximately 4,000 times faster than TrueNorth.


Picture/IBM

In terms of structure, NorthPole, like TrueNorth, consists of a large computing unit array (16×16), each unit containing local memory and code execution capabilities.

In terms of computing resources, each unit of NorthPole has been optimized to perform calculations with lower precision, ranging from 2bit to 8bit. To ensure the use of execution units, they cannot execute conditional branches based on variable values, that is, user code cannot contain if statements. This simple execution enables massively parallel execution per computing unit. At 2-bit precision, each unit can perform more than 8,000 calculations in parallel.

In terms of storage, all the memory is packaged in a single chip, which means that each core can easily access the memory on the chip. From the outside of the device, NorthPole looks like an active memory chip, which helps integrate NorthPole into the system.

Not only that, this NorthPole is currently manufactured using a 12nm nano-node process. The most advanced CPU technology currently is 3nm, and IBM is still developing 2nm nano-node technology. If 2nm is used, the performance of IBM's brain-inspired chips may be greatly improved.


Picture/IBM

Potential applications for NorthPole include image and video analysis, speech recognition, and Transformer neural networks, which are the large language models (LLMs) that power chatbots such as ChatGPT. These artificial intelligence tasks may be used in areas such as self-driving cars, robots, digital assistants and satellite observations.

Some applications require neural networks that are too large to fit on a single NorthPole chip. In this case, these networks can be broken into smaller pieces and distributed across multiple NorthPole chips.

NorthPole's ultra-high energy efficiency ratio means that it does not require a bulky liquid cooling system to operate, fans and radiators are enough, and it can also be deployed in smaller spaces.

Has China begun research on brain-inspired chips?

Such powerful chips have already been laid out in China.

Domestic research includes top universities and institutions such as Tsinghua University, Zhejiang University, Fudan University, and the Chinese Academy of Sciences. At the same time, start-up companies have emerged in the past two years, such as Lingxi Technology, Shishi Technology, Zhongke Neuromorphology, etc. Among them, Tsinghua University’s Tianji core and Zhejiang University’s Darwin chip are the most representative.


Specifically, the main domestic achievements include:

The first-generation Tianji core developed by Tsinghua University in 2015 used the 110nm process. In 2017, the second-generation Tianji core began to achieve advanced results. It was made based on the 28nm process and consisted of 156 functional cores FCore, containing about 40,000 neurons and 10 million synapses. Compared with the first generation, the density is increased by 20%, the speed is increased by at least 10 times, and the bandwidth is increased by at least 100 times. In addition, Tsinghua University has independently developed a software tool chain to support automatic mapping and compilation from the deep learning framework to Tianji core. According to Tsinghua University’s plan, the next generation of Tianji core will be a 14nm or more advanced process and will be more powerful;

The number of neurons in the brain-like computer jointly developed by Zhejiang University and Zhijiang Laboratory is equivalent to that of the mouse brain. The computer contains 792 Darwin 2nd generation chips, supports 120 million spiking neurons and 72 billion synapses, and its typical operating power consumption is only 350W~500W;

In October 2020, Zhang Youhui's team from the Department of Computer Science and Shi Luping's team from the Department of Precision Instruments at Tsinghua University and their collaborators published an article in Nature magazine for the first time proposing "brain-like computing completeness" and a hierarchical structure of brain-like computing systems with decoupling of software and hardware;

In September 2023, a research team led by researchers You Haihang and Tang Guangming of the Institute of Computing Technology of the Chinese Academy of Sciences developed the superconducting neuromorphic processor prototype chip "SUSHI". It is a superconducting computing chip based on superconducting single flux quantum (SFQ) circuits;

In October 2023, the brain-inspired supercomputer service independently developed by the Nanjing Intelligent Technology Research Institute of China Science and Technology was officially launched. It has achieved an intelligence scale of 500 million neurons and 250 billion synapses. It is more than 10 times more energy efficient than the existing computing system, and the core chip is independently controllable.

IBM's achievements represent that this technology that will lay out the future is one step closer to us, and currently some start-up companies are gradually forming plans and starting to apply it. It is not difficult to foresee that in recent years, the commercialization of this technology will gradually begin, and the research results at that time will also be reflected in reality.