The system converts the waves into a code that can be mapped to specific words
Scientists in Australia have developed a portable, non-invasive artificial intelligence system that decodes thoughts and turns them into text. The tool DeWaveas it was called, was tested on 29 volunteers.
Participants read silently while wearing a “cap” that recorded their brain waves via electroencephalogram (EEG) and decoded them into text, as the sciencealert.
With further refinement, DeWave could help stroke or paralysis patients communicate.
“This research is a pioneering effort to translate raw EEG waves directly into language, marking a major breakthrough in the field,” said Chin-Teng Lin, a computer scientist at the University of Technology Sydney (UTS).
Although DeWave achieved just over 40% accuracy based on one of two sets of measurements in experiments conducted by Lin and his colleagues, this is a 3% improvement over the previous standard for translating thoughts from EEG recordings . The researchers’ goal is to improve the system’s accuracy to around 90% – a rate comparable to conventional language translation methods or speech recognition software. Other methods of translating brain signals into language require invasive surgeries to implant electrodes or bulky, expensive MRI machines, making them impractical for everyday use.
After extensive training DeWave converted the EEG waves into a code that can be mapped to specific words. The researchers estimate that the integration of the model into large language models (Large Language Models, LLMs) also opens new avenues in neuroscience and artificial intelligence.
Lin and his team used trained language models and tested the tool on existing human datasets that included recordings of eye tracking and brain activity while reading text. So the system learned to match the brain wave patterns with the words. It was then trained on a large open source language model that essentially builds sentences out of words. DeWave performed best in predicting verbs. However, he tended to translate nouns as combinations of words that have the same meaning but are not an exact translation, such as “the man” instead of “the author”.
“We think this is because when the brain processes these words, conceptually similar words may produce similar brain wave patterns,” explained study lead author Yikun Duan, a computer scientist at UTS.
“Despite the challenges, our model gives meaningful results, matching keywords and forming similar sentence structures,” he noted.
The study was presented at the NeurIPS 2023 conference. A pre-publication is available online ArXiv.
