This Smart Glove Translates Sign Language to Text and Speech

Hadeel Ayoub, a Saudi designer and media artist, wants to bridge the communication gap between people with and without hearing disabilities. After a year of tinkering, she's come up with a "smart glove" that converts sign language into text and speech.

"I have an autistic niece who is four and who doesn't speak. When I saw her communicating with sign language, I wondered what would happen if she tried to communicate with someone who doesn't speak the same language," Ayoub told me over the phone. "Does that mean she wouldn't be able to get through to them?"

Ayoub told me that her motto is always "to design for a cause," and to produce products that will help a wider community. With the "smart glove," her aim is to "facilitate communication" between people, and break down language barriers in the process.

Her SignLanguageGlove is a wireless device that translates sign language gestures into visual letters or speech, which could be read by someone else's smartphone app or tablet device. Ayoub, who speaks French, English, and Arabic, also wants to incorporate multilingual features into the glove. The gloves have already gone through three prototyping stages.

A previous prototype as a work-in-progress. Image: Hadeel Ayoub

The glove works using five flex sensors on the fingers. These detect how the user bends their fingers as they sign, reporting the data to a serial monitor. An accelerometer keeps track of how the user orientates their hand as they sign.

Ayoub developed a computer programme that translates the motions coming from the glove into words displayed on a screen. In her latest prototype, she minimised and incorporated all the hardware into the lining of the glove, which now includes a text-to-speech chip that turns signs into spoken language. In terms of displaying the text on the screen, an upgraded software allows the text to scroll on the screen before deleting old sentences as new ones appear.

Over the years, many teams have invested time and effort into developing similar smart gloves. Back in 2012, Team QuadSquad from Ukraine developed a prototype that won a software design competition in the same year, and caused a lot of hype in the developer community. In July, researchers in Mexico also developed a prototype that transmitted the glove wearers signs via Bluetooth to a mobile device capable of translating the movements into text and speech. Ayoub maintained that her glove is "less bulky" than current designs, and as well as incorporating multilingual translation features she's also keen to make a smaller versions for kids.

"For me it's all about eliminating disability, language, and physical barriers," said Ayoub. "My next prototype will have wifi so that it will be able to send texts and emails as well to any smart device within a reasonable range."