Multimodal Neural Decoding: Data-Intensive Approaches to Understanding Long-Term, Unlabeled Human Brain Data

Fully automated decoding of human actions and intentions from neural signals is a major challenge in human-computer interactions. The current success of brain-computer interfaces (BCI) has hinged on labeled training data in laboratory conditions. To deploy BCIs in real-life, one must develop robust strategies to handle naturalistic disturbances. This research effort focused on decoding movements from large-scale human intracranial brain recordings, video, and audio, all continuously acquired over one week. Importantly, here the participants are simply behaving as they wish. This research demonstrated the ability to decode movements from neural recordings as well as that this decoder generalizes remarkably well on application to unseen participants.