Translating Brain Waves into Breakthrough Hearing Aids
Imagine a world where new devices could help us hear with perfect accuracy in any environment and help the non-verbal communicate with those around them
Mesgarani with students Laura Long and Vinay Raghavan in his lab.
Schematic illustrating how Mesgarani's smart device tackles the "cocktail party" problem.
The average human speaks thousands of words a day. Yet how the brain perceives speech—and the possibility of recreating this phenomenon—is a complex challenge for researchers.
Nima Mesgarani, associate professor of electrical engineering, develops novel speech-processing algorithms that enable a better class of hearing devices and speech brain-computer interfaces to transform the lives of the hearing impaired and the non-verbal. Also a principal investigator at the Zuckerman Institute, he has honed his research with neurosurgeons to gain high-resolution recordings of neural activity by placing electrodes directly on the brain.
One of Mesgarani’s key findings is that, during a conversation, the brain waves of a listener come to match the sound waves of any speaker on which they train their focus. He applied this knowledge to the classic “cocktail party challenge,” a multi-voice environment that presents a nightmare for those with hearing aids.
“In such environments, the sound waves that reach a person’s ear are completely mixed, yet our brain has a way of segregating and interpreting individual sound sources from this mixed signal,” Mesgarani says.
In the case of the cocktail party, an ideal hearing device would amplify just the voice of the target speaker. However, a standard hearing device has no way of distinguishing between a target speaker and background noise, muddling the full range of sound that reaches its user.
Mesgarani solved this problem by observing the brainwaves of the listener and comparing them with the voices in the acoustic scene. Using that data, he created a module that automatically separates multiple sources, identifies the target speaker, and amplifies that speaker above others.
Mesgarani worked with Columbia Technology Ventures to obtain a patent and funding to explore hearing aids and hearables for anyone who desires more control over their auditory environment. With the number of people with debilitating hearing loss estimated to rise to 1 billion within the next 30 years, his breakthrough will be music to many ears.