This Young Startup Built a Futuristic Vest That Could Help Deaf People Hear

For the last month, Jonathan Leach has been hearing words through his back. Responding to sounds, a network of tiny vibrating motors, sewn inside a vest swing into action, function like live-action Braille to tell Leach what's going on around him. Now Leach is recruiting other people who, like him, have severe hearing loss to try the technology and hopefully pave the way for the hearing device of the future.

It's been a year since the VEST — the Versatile Extra-Sensory Transducer — hit its funding goal of $40,000 on Kickstarter to pay for data gathering, hiring people like Leach and developing the VEST's algorithm to map the sounds and make it more functional. In the time since the VEST secured that funding, the idea has gone from a Ph.D. project by Scott Novich to a full-fledged company, Neosensory Inc., co-founded with Novich's adviser, David Eagleman. Neosensory is working toward commercializing VEST technology for the deaf and hard-of-hearing community.

This is how it works: Think of the word "car." When that word runs through a smartphone carrying the software for the VEST, it's sent via Bluetooth to a sound-to-touch mapping device, which breaks down the word into sounds and turns those sounds into vibrations, which are then assigned to different spots on the vest's motor map.

When the vest senses a sound, it turns the sound into a sensation that represents each element of the word. So for "car," the "c-" sound triggers one sensation and "-ar" the other. Once you've learned the words "car," "bar" and "star," context would make it easy to understand new words like "smart," even if it's the first time you'd felt that word. 

Like an ear, the vest also picks up sounds that aren't words. You'd "hear" low-flying airplanes or fire trucks. Novich says you'd even feel a soft, persistent rumbling in a room where the air conditioning is turned up full blast.

Another way to think of it: When you're fully versed in using the vest, you aren't fluent in a language. You're fluent in patterns. But it takes a while to fully pick up.

Leach still can't use VEST to fully keep up with a conversation, especially if he doesn't have all the context for what people are saying. He hasn't spent months wearing the vest and learning to master it. But he's come a long way since a month ago, when he was only able to make out "each," a word he said has one of the clearest and most distinctive patterns — and also the first word he learned after a few days of hour-and-a-half testing sessions.

"At first, the rumblings felt unintelligible and scattered as I attempted to guess correctly which word I was feeling through the vest," Leach told Mic. "After a few minutes, the patterns became more definite and recognizable. I started sensing the patterns and began to associate them with words I was feeling. The sensation is similar to learning something for the first time, [like] riding a bicycle, doing a flip mid-air."

But there are some important problems to solve: The vest Eagleman shows off during his March TED Talk is still bulky. It looks more akin to outerwear or RoboCop's body armor than something you'd comfortably wear under your clothes. Figuring out ways to aestheticize it will be crucial to making it viable on a large scale — which could be done, given the advancements in liquid circuitry and wearable technologies.

Even solving the aesthetics issue leaves a lot of obstacles on the table: What's the most effective way to train people to use the vest? How can you make the vest run all day without having to stop and charge it? What's the best way to manufacture for scale? And of course, how can you make the algorithm better?

The vest can also be used to translate data it receives from the Internet to make predictions.

The VEST is meant for regular use. Its creators intend for it to be worn with the same frequency and inevitability as a hearing aid. The next step is to create enough prototype vests that users can take them home and see how they fare in their everyday lives.

"This is where we expect to see the truly exciting results roll in, as we think that it will take week so continuous use to really start to develop a new sense of things," Novich told Mic. "A lot of our focus over the past number of months has been turned towards the engineering, but we're pretty close to having the next iteration ready for gathering new data."

Here's where it gets really cool: The vest can also be used to translate data it receives from the Internet to make predictions, like what's going on in the stock markets, the general mood on social media after a crisis or what's happening in Syria — with literally no delay.

So not only would the vest allow wearers with hearing loss to follow the conversations happening around the office in real time, it would allow them to follow the conversations happening around the entire planet.