“You create music with your brain. Now, create music with someone else’s.”
These are the simple instructions for visitors to the new Neural Symphony project, an endeavor out of the laboratory of Assistant Professor of Speech, Language, and Hearing Sciences Erika Skoe.
Drawing on her laboratory’s expertise in the area of music and the brain, Skoe’s group has created a new approach to what they call neuromusic.
Through the project, listeners can not only hear recordings of popular songs re-imagined with brains sounds – such as the “Game of Thrones” theme song – but can also use these brain recordings as an instrument in their own recordings. The idea, says Skoe, is to “explore the creative space between neuroscience and the arts.”
“We want it to inspire people to use the timbre of the brain to create novel music,” Skoe says.
Listening to data
Scientists use a process called sonification to convert brain electric waves, recorded using electroencephalography (EEG) or other methods, into sounds. These sounds can reveal how different regions of the brain respond to input.
“The idea is that scientists can listen to their data,” says Skoe. “By listening to data, and capitalizing on the fact that our ears are very keen at detecting subtleties, we can gain insight into the mechanics of the brain in a way that wouldn’t have been possible if we put the same data on a graph.”
For example, brains with broader sound experiences – say, a person who grew up bilingual, or played an instrument from a very early age, will respond differently to the sounds they hear than a monolingual person or a non-musician. When those responses are recorded using EEG, then sonified into musical notes, people who speak more than one language, or play a musical instrument, will have a clearer signal with less background noise.
“It’s a useful way to demonstrate that there are individual differences in the clarity of the brain’s neural signal,” says Skoe. “And they’re not the kind of differences that you need to be a skilled musician or a scientist to hear.”
Skoe’s team recently published a mathematical model that allows scientists to predict how loud or soft the sonified neural output will be for a particular frequency. This is useful, says Skoe, to help understand what features of the brain’s auditory spectrum are common across different brains or distinctive to a specific brain.
For the Neural Symphony project, Skoe and her team recorded brain waves of one individual listening to each note of the piano hundreds of times. They, then, averaged the individual’s brain response for each note, creating a series of brain responses that map onto the piano keyboard.
Creative space
Using brain wave sonification to make music is not new, says Skoe. In 1965, composer and experimental musician Alvin Lucier used EEG recordings of his own brain to vibrate and play percussion instruments in his piece, “Music for Solo Performer.”
Skoe and graduate student Parker Tichko recently attended a music and cognition conference at Wesleyan University, where Lucier was the keynote speaker. Tichko, a graduate student in Psychological Sciences, studied music and psychology as an undergraduate, with a concentration in composition. He and Skoe shared an interest recording the sounds of the brain, and the Neural Symphony project was born.
“With my background in digital music and Erika’s neuroscience expertise, we knew we could make it happen,” says Tichko.
Tichko has developed what Skoe calls the “brain piano”: a keyboard programmed to play neural sonifications. It’s a big hit at academic conferences, where Tichko has also demonstrated the Neural Symphony library. Undergraduates in Skoe’s anatomy and physiology of the hearing class especially enjoy this demo, as they learn about the longstanding intersection between science and art.
Through the Neural Symphony website, anyone can download the library of “brain notes” and compose their own piece of neuromusic. The Neural Symphony Soundcloud page features various Neural Symphony tracks already produced by Tichko, ranging from an arrangement of J.S. Bach’s “Jesu, Joy of Man’s Desiring” to a percussive, driving drum loop.
Skoe and Tichko purposefully kept their brain recordings in a raw state, which includes the background noise of the brain’s inner workings: “In the ‘Game of Thrones’ theme song demo, you can hear that ‘static,’” says Skoe.
Tichko imagines creating an entire symphony using the sounds of biological processes, like the heartbeat and blood flow.
“Especially in science, which can sometimes be hard for the public to conceptualize, projects like the Neural Symphony can introduce people to difficult concepts, and might facilitate more cross-talk between the arts and science,” he says.
He notes that the current popularity of electronic music is partly because of the ability to create new sounds with electronic instruments.
“I like extending this thinking to the sounds that are inside of us – how can we harness the sounds of our brain and body for music?” he asks.
Skoe also hopes people are inspired to create new and different art.
“I want to be surprised by what people do with our neural sonifications,” says Skoe. “There are limitless possibilities.”
—
To learn about Skoe’s other work on the intersection of science and art, visit her Scientific Discovery Through Art page.
