Online decoding of brain activity is the most promising approach to restore communication and interaction with the environment to people suffering from brain diseases that result from acute accidents or degenerative processes. Reading these subjects¿ intentions from the electrophysiological activity from their brains, i.e. devising a brain-machine interface (BMI), has indeed been proven to be a viable approach to extract the information needed to rebuild an otherwise interrupted communication channel. However, information transfer rates and patient usage of systems using non-invasive approaches, like those based on EEG signals, are still very low and recent results using on invasive intracortical recordings in humans have yielded only moderate advantages.
Aim of the present project is to contribute to a significant advance in the BMI state-of-the-art by devising novel approaches that rely on the multiscale nature of the signals to decode. Information will be obtained from neuronal activities in macaque monkeys learning rank order sets of symbols and then asked to manipulate the acquired relations to generate new knowledge. This cognitive ability will be taken as a model for studying how letters and syllables are combined to form words in the brain.
These findings will allow the development of algorithms, based on advanced recurrent neural networks for optimal decoding and classification of stimuli and intentions, for a next-generation speller able to efficiently support communication in people suffering of severe neurodegenerative diseases.
By combining a multiscale probe of neural signals from non-human primates with simultaneous EEG recordings it will be identified the neuronal signature of words encoding in both the intracortical and scalp signals. This information will support the identification of the proper spatiotemporal patterns at the EEG level and improve so the decoding of the signal for non-invasive approaches.
The project will have the scientific goal of better understanding the neural mechanisms of information encoding and manipulation in the non-human primates (NHPs) cerebral cortex, using data recording techniques currently used in human subjects during invasive neurosurgical approaches. Aim of the present project is the identification of the amount of information stored at different scale of the neural signal. The underlying hypothesis is that the brain uses different time scales to memorize events, to assemble these events in mental structures, to control effectors for the interaction with the environment. Many aspects of brain function can be understood in terms of a hierarchy of temporal scales. These different scales could be clustered in states when the neural signal is recorded at high-resolution and could facilitate decoders having (by generalization) the goal to extract, e.g., faster changes in the environment (sensory processing) from other events with slower dynamic (memory, errors-detection). Our approach on NHPs will provide information on the hierarchical relations between the different scales of the neural signal during the manipulation of serially ordered symbols in a task that models words construction. These knowledges will contribute to develop novel computational algorithms to support human patients suffering of severe neurological diseases and necessitate of efficient BCI system for environmental interactions, currently lacking in many BCI systems, including EEG-guided spellers. The current technology of EEG-guided spellers would make steps forward if associated to multiscale recording. The information carried by LFP or MUA signals, if efficiently decoded, will be used to provide correction feedback for decoding of EEG signals and improve the time and precision of words spelling (Figure 2).
A system able to extricate different time scales, and the underlying hierarchical structures, from the human cortex neural signal has the potential to interact with smart machines using the same organization as a speech synthesizer. A BMI controlling a synthetic speller could represent an important step on assistive technology also bypassing the need of separate communication devices (keyboard, eye trackers).
The project has a strong innovation potential because the novel approach based on an hybrid system for decoding thought and intention if extended beyond the context language interpretation it would improve the control of other BCI systems supporting environmental interaction (26).