These eyeglasses can detect speech—even when you don’t make a sound

What is silent speech technology?

When demonstrated by an able-bodied user, the glasses’ multitude of applications is immediately evident. Here are just a few ideas:

• Mouthing a passcode for a nearby paired device or very quietly skip a song on Spotify. 
• Communicating with others in a silent setting, like a library—students, rejoice!
• Pairing with a stylus for mouse- and laptop-free design work

But these glasses are even more powerful in what they do for people with neurological injuries. 

“For people who cannot vocalize sound, this silent speech technology could be an excellent input for a voice synthesizer,” said study author Ruidong Zhang. “It could give patients their voices back.”

We’ve previously discussed technologies that help patients with various neurological conditions that limit their communication—namely, the brain-computer interface (BCI). Other less-invasive technologies, like an fMRI scanner, have proposed helping turn thoughts into words using AI. 

However, in the case that a user can mouth their words, no solution appears to match the simplicity and ease-of-use of these smart glasses.

A new kind of glasses-based wearable device

Another compelling aspect of this device is how low-maintenance it is. It operates on low power, requires just a few minutes of user training data to function, and all you need to run it is a smartphone.

The device incorporates a set of tiny microphones and speakers smaller than a pencil eraser, allowing them to process and transmit sound waves around the user’s face and detect mouth movements. The Cornell lab’s deep learning algorithm then analyzes these data inputs in real-time with an estimated 95% accuracy.

“We’re very excited about this system because it really pushes the field forward on performance and privacy,” said Cornell assistant professor Cheng Zhang. “It’s small, low-power, and privacy-sensitive, which are all important features for deploying new, wearable technologies in the real world.”

Previous silent speech-detecting devices relied on wearable video cameras, which the researchers point out are impractical for everyday use.

The lab is now exploring other applications of smart glass.

“We think glass will be an important personal computing platform to understand human activities in everyday settings,” Cheng Zhang said.

They certainly aren’t alone in that opinion. 

While some smart glass mass-market devices have failed to resonate with the general public (e.g., Google Glass), the tide on smart glass has been shifting. 

Here are a few reasons why we think this is so:

• For one, advancements in AI have made it possible to do more with data inputs gathered from these wearable devices. 
• At the same time, medical and entertainment applications of wearable glasses devices (i.e., VR glasses) have gathered steam, as we discussed in our lead Insight. 
• Finally, the design. Earlier products like Google Glass were notoriously clunky—much more visually prototype-adjacent than the sleek products hitting the market and even in development today.

Even Google appeared to be getting back into the glass market last year with its new Google Translate glasses, but we’re still waiting to see if the product pans out.

For now, we’re excited to see what more niche applications of this technology can bring to patients like those EchoSpeech aims to serve. All people deserve to be able to communicate with privacy, and we love that devices like these open up that possibility to people with a wider range of abilities.