An interdisciplinary team has discovered that artificial intelligence models, specifically Transformers, process memory in a similar way to the human brain's hippocampus. This breakthrough demonstrates that applying neuroscience principles, such as the NMDA receptor principle, to artificial intelligence can improve memory function, thereby advancing the field of artificial intelligence and providing insights into human brain function.

Researchers have found that the memory consolidation process of artificial intelligence is similar to that of the human brain, especially the hippocampus, which provides the possibility for the advancement of artificial intelligence and a deeper understanding of human memory mechanisms.

An interdisciplinary team of researchers from the Institute for Basic Science (IBS) Center for Cognition and Sociality and the Data Science Group has revealed striking similarities between the memory processing of artificial intelligence (AI) models and the human brain's hippocampus. This new discovery provides a new perspective on memory consolidation (the process of converting short-term memory into long-term memory) in artificial intelligence systems.

In the race to develop artificial general intelligence (AGI), led by influential entities such as OpenAI and Google DeepMind, understanding and replicating human-like intelligence has become an important research interest. At the heart of these technological advances is the Transformers model [Figure 1], the fundamentals of which are currently being explored in new depth.

Figure 1: (a) Schematic diagram of ion channel activity in postsynaptic neurons. AMPA receptors are involved in activating postsynaptic neurons, while NMDA receptors are blocked by magnesium ions (Mg²⁺), but when postsynaptic neurons are fully activated, NMDA receptors induce synaptic plasticity through the influx of calcium ions (Ca²⁺). (b) Flow chart showing the calculation process of the Transformer artificial intelligence model. Information is processed sequentially through stages such as feedforward layers, layer normalization, and self-attention layers. The graph describing the current-voltage relationship of NMDA receptors is very similar to the nonlinearity of the feedforward layer. Input-output plot based on magnesium concentration (α) showing non-linear changes in NMDA receptors. Source: Institute of Basic Science

Brain learning mechanism applied to artificial intelligence

The key to powerful AI systems is understanding how they learn and remember information. The research team applied the learning principles of the human brain, especially the principle of memory consolidation through NMDA receptors in the hippocampus, to the artificial intelligence model.

NMDA receptors are like a smart door in the brain, promoting learning and memory formation. When a brain chemical called glutamate is present, nerve cells become excited. Magnesium ions, on the other hand, act like a little gatekeeper, blocking the door. Only when this ion gatekeeper steps aside can substances flow into the cell. This is how the brain creates and retains memories, and the role of the gatekeeper (magnesium ions) in the entire process is quite special.

The team made a surprising discovery: The Transformers model appears to use a gatekeeping process similar to the brain's NMDA receptors [see Figure 1]. This discovery prompted the researchers to investigate whether memory consolidation in Transformers could be controlled through a mechanism similar to the NMDA receptor gating process.

Low magnesium levels are known to impair memory function in animal brains. Researchers found that Transformers' long-term memory could be improved by mimicking NMDA receptors. Just as changes in magnesium levels in the brain affect memory strength, adjusting Transformers' parameters to reflect NMDA receptor gating can enhance the memory of AI models. This breakthrough discovery shows that the way artificial intelligence models learn can be explained by existing knowledge from neuroscience.

Expert insights on artificial intelligence and neuroscience

C. Justin LEE, neuroscientist director of the institute, said: "This research is a critical step in advancing the development of artificial intelligence and neuroscience. It allows us to study more deeply how the brain works and develop more advanced artificial intelligence systems based on these insights."

CHAMeeyoung, a data scientist at the team and KAIST, noted: "Unlike large-scale AI models that require huge resources, the human brain operates with minimal energy, which is remarkable. Our work opens up new possibilities for low-cost, high-performance AI systems that learn and remember information like humans."

Integration of cognitive mechanism and artificial intelligence design

What makes this research unique is that it proactively incorporates brain-inspired nonlinearities into artificial intelligence structures, marking a significant advance in simulating human-like memory consolidation. The fusion of human cognitive mechanisms and artificial intelligence design not only promises to create low-cost, high-performance artificial intelligence systems, but also provides valuable insights into how the brain works through artificial intelligence models.

Compiled source: ScitechDaily