A research team at Yale University in the United States recently announced that they have successfully developed a new brain-computer interface (BCI) system that can control virtual characters in video games in real time by relying only on the user's brain activity, without any traditional handles or controllers. The researchers said that by "complying" with the inherent neural activity structure of the human brain, this system achieves rapid start-up in very little training time, and is expected to reshape many fields such as medical rehabilitation, mental health intervention, and human-computer interaction.
The study used functional magnetic resonance imaging (fMRI) to monitor subjects' brain activity in real time and convert these signals into game instructions. The research results were recently published in the journal Nature Neuroscience. The team found that when the design of the brain-computer interface fits the existing neural pathways and activity patterns of the brain, users can learn to control the system with "thoughts" significantly faster, and the brain's own activities will also be adaptively reorganized.
The first author of the paper, Erica Busch, who just completed her doctoral studies at Yale, pointed out that brain activity is not chaotic, but operates along an established "neural manifold". When the brain-computer interface conforms to this natural structure, the learning burden is greatly reduced, and the user can obtain stable control capabilities in a short period of time; on the contrary, if the system requires the brain to produce unnatural activity patterns, the learning effect will hardly be substantially improved.
Brain-computer interface is a technology that allows humans to interact directly with computers through brain activities. Human-related research has continued for many years, but the practicality and learning efficiency of many systems are still limited. In the past, brain-computer interfaces based on real-time fMRI usually required up to ten long training sessions, but the performance improvement was very limited. About one-third of the participants were unable to learn to effectively control the system no matter how hard they practiced. The Busch team believes that this is largely due to the way traditional systems are designed: they often ignore the inherent organizational structure of the brain and force users to learn "against the brain's natural habits."
In order to verify the idea of "complying with the geometric structure of the brain", the research team recruited a group of healthy young subjects and arranged for them to participate in four fMRI experiments. In the first round of experiments, participants used physical joysticks to control a virtual character in the scanner to move around the scene, while researchers recorded their brain activity. The team focused on brain areas related to navigation and spatial movement, and then introduced an algorithm "T-PHATE" developed in previous research to extract each participant's individualized "neural manifold", that is, the natural structural trajectory of their brain activity.
Based on this "brain activity map", the researchers constructed three different sets of "brain control-game mapping" systems for each subject. The first set is "intuitive mapping", which connects to the strongest and most natural activity patterns in the brain; the second set is "intra-manifold perturbation", which still relies on the inherent structure of the brain, but shifts to relatively minor activity patterns; and the third set is "extra-manifold perturbation", which requires the brain to generate activity patterns that it rarely naturally generates. In other words, these three systems respectively represent three different design ideas: "going with the trend," "reluctantly going with the trend," and "completely going against the trend."
In the following three experiments, the research team built a closed-loop system that collected new brain scan data every two seconds and immediately converted this information into movement instructions for the virtual character. Participants relied solely on "ideas" to try to control the game, with each experiment corresponding to one mapping method. The results show that when the brain-computer interface follows the natural manifold of the brain, subjects can usually learn to control the character relatively proficiently in less than an hour, and some people are even significantly faster; while under the "out-of-manifold perturbation" condition, almost no one can truly master control in the same amount of time.
In addition to behavioral performance, the brain itself also shows significant adaptive changes. As participants gradually master "mind control", the activity patterns of relevant brain areas will be reorganized to better match the needs of the system. Under some conditions, the degree of this reorganization is highly related to the participant's operational level; at the same time, this change is not limited to the initially targeted navigation brain area, but spreads to a broader neural network. Researchers believe that the "neural manifold" is both a constraint and an opportunity for learning - it determines what people can learn and how fast they can learn.
This discovery also provides a new perspective on understanding human skill learning. The research team pointed out that the reason why certain skills are relatively easy to master may not only depend on personal effort or talent, but also be closely related to whether the task itself "complies" with the existing structure of the brain. Humans tend to learn quickly for tasks that closely match the brain's natural patterns; however, if the task design deviates significantly from these patterns, no amount of training will achieve much.
At an applied level, the potential impact of this research extends far beyond the laboratory. In the field of mental health, researchers believe that interventions for diseases such as depression and anxiety may be more effective if they can be "step by step" adjusted along the existing activity patterns of the brain, rather than trying to completely reshape brain circuits. For patients with movement or communication disorders, this design concept that conforms to brain structure is also expected to lead to a more stable and reliable non-invasive brain-computer interface, allowing them to more naturally control external devices through brain signals.
More broadly, the approach could also be used to improve cognitive abilities in healthy people. By designing training programs around the brain's natural organization, people may be able to learn new skills more efficiently and optimize attention and memory performance. As Busch said, humans invest a lot of resources in education, training and treatment, hoping to become a "better version of themselves", and truly understanding the structure of their own brains may be the key to greatly improving the efficiency of this process.
The research was jointly completed by Erica L. Busch, E. Chandra Fincke, Guillaume Lajoie, Smita Krishnaswamy, Nicholas B. Turk-Browne and others. The paper is titled "Human learning of noninvasive brain–computer interfaces via manifold geometry." The research has received funding from the U.S. National Science Foundation, the Canadian Institutes for Advanced Study, the Sloan Foundation, and agencies affiliated with the U.S. Department of Health and Human Services.