Abstract:
AI’s ability to decode visual signals from the brain goes a step further. An Israeli scientific research team recently developed an AI model that can use brain scanning to restore the images seen by the human eye. The team also hopes to expand to video and even dream decoding in the future. At the same time, the risk of misuse of this technology has also raised concerns.
According to media reports on Saturday local time,
The team of Michal Irani, a professor at the Weizmann Institute of Science in Israel, recently developed an AI model called Brain-IT. The model can analyze functional magnetic resonance imaging (fMRI) data to reconstruct the visual picture of the subject viewing the image.
Source: Official press release of the Weizmann Institute of Science
According to the official press release, the model can restore the images seen by the subjects with extremely high accuracy.
Training this type of AI model requires a large amount of "picture + brain scan" paired data, which records a person's brain activity when viewing each picture.
But this type of data is extremely time-consuming and laborious to collect. The public "Natural Scene Dataset" used in the study is currently the largest data set in the field, but it only includes data from 8 subjects. Each person participated in 30 to 40 scans in the scanner, viewed thousands of images, and formed a total of approximately 73,000 pairs of data.
In order to solve the problem of insufficient data, the team trained a "back-translation" model: based on the picture, it predicts the brain activity of people when they see it, thus significantly expanding the training data.
Irani said that (previously) some models have been able to achieve high-level image reconstruction based on brain activity, but basic errors in composition and color often occur.
She noted: "The new model we developed is superior to these models in terms of restoring image content and details."
"Additionally, other models require dozens of hours of brain scan data to learn to 'read' a new person, whereas our model only takes one hour."
The left is the original image seen by the subject, the right is the Brain-IT reconstructed image, source: official press release of the Weizmann Institute of Science
Currently this study uses static images. Irani's lab is working to expand this type of "mind-reading" approach to decoding auditory information and eventually to video.
Particularly challenging is decoding video, such as during a dream, Irani said in a statement. Video images change dozens of times per second, and fMRI takes about two seconds to complete a scan. If we can overcome all these obstacles,
we may be able to read dreams in the future
. ”Media reports pointed out that scientists have been working on related technologies for many years. In 2017, a paper published in the international legal journal "Journal of Law and the Biosciences" explored the possibility of using neuroimaging technology to achieve "brain-based mind reading."
The paper points out that this type of technology can be used in courts to conduct polygraph assessments of defendants, prisoners and potential jurors, thereby improving the judicial system. The paper also cautions that this type of technology may also be used for unethical and coercive purposes.
Kenneth Norman, professor of psychology and neuroscience at Princeton University in the United States, believes that the quality of signal data from fMRI equipment has improved in recent years, and the methods for humans to analyze these data have also improved, both of which make this type of technology possible.
According to reports, in 2022, Norman said in a conversation with the American Psychological Association (APA) that such technology is expected to help patients with depression or anxiety.
He explained that therapists can use neurofeedback to make patients aware that their thoughts have slipped into negative loops and guide them in trying to control these thought patterns.
Currently, some neuroscientists speak highly of the capabilities of the Brain-IT model and believe that it has the potential to be used in the treatment of patients with neurological diseases. But there are also concerns that this technology may be abused and used to forcibly obtain personal ideological information.
Comments