Data Stethoscope for the Brain Connectome
Technical Report,15 Aug 2015,14 Nov 2016
University of Texas at Dallas Richardson United States
Pagination or Media Count:
The goal of this project was to develop a prototype dynamic tool that enables exploration of connectome network data in a 3D interactive video game environment using the UNITY gaming engine. We have drawn on and expanded existing taxonomies of data sonification to allow multiple ways of sonifying the same data to draw attention to different aspects of that data. The software embodies the concept of Data Stethoscopes, or developing data exploration tools that exploit human ability to compare and contrast two sounds. Collaboration researchers at the UT Dallas Cognitive Neuroimaging Laboratory and the ArtSciLab have tested a number of modules to examine sonic differences in brain network organization the data used allowed comparison of functional magnetic resonance imaging fMRI data of individuals of different ages, an area where the PI has published results that can be tested and compared with the sonification tool. The research methodology was based on a transdisciplinary collaboration between artists and scientists who jointly developed the tools for testing. To demonstrate extension to other types of data, the tools were successfully applied to a connectivity dataset obtained using intracranial electroencephalography recordings of human patients. While initial results are promising, interim results indicate that further work is needed in order to utilize the tools in a formal research setting. In addition, the tools and data were used in an artistic performance. The art performance was in Seattle October 2017 as part of the re-staging of the Nine Evenings performance originally carried out in 1967 between Bell labs engineers and NY artists. The second performance was held at the University of Caldas during the International Symposium of Electronic Arts.
- Medical Facilities, Equipment and Supplies