I knew the AI on these smart glasses worked pretty well once it told me that someone else in the conversation was being the socially awkward one.
TranscribeGlass are smart eyeglasses that aim to do exactly what it says on the tin: transcribe spoken conversations and project subtitles onto the glass in front of your eyes. They’re meant for the Deaf and, primarily, the hard-of-hearing community who struggle to read lips or pick out a conversation in a loud room.
Most face computers are graceless and heavy, but these glasses are light, only 36 grams. TranscribeGlass is able to keep the weight off by relegating most of the main computing features to a companion app (iOS only for now). There are no cameras, microphones, or speakers in the frames, just a small waveguide projector in the rim of one eye that beams a 640 x 480p image onto the glass. That is just enough resolution for text to be legible when it is projected directly into your vision, subtitling the conversations picked up by the mic in your phone.
In the app, subtitles can be moved around in the wearer’s vision, anywhere within a 30-degree field of view. You can change the settings to adjust how many lines of text come in at a time, dialing up to a wall of text and down to one word at a time. The battery in the glasses should last around eight hours between charges. The frames cost around $377, and there’s an additional $20-per-month subscription fee to access the transcription service.
Subtitles are currently available in the glasses, but Madhav Lavakare, the 24-year-old founder of TranscribeGlass, has other features lined up. In the testing phase are a setting to translate languages in real time and one to analyze the tone of voice of the person talking.
Glass Dismissed
As Lavakare told me (and The New Yorker in April), he envisioned the idea for this product after wanting to help a hard-of-hearing friend engage in conversations that were not happening with his needs in mind. Lavakare, who is a senior at Yale University, figured glasses were the way to go. If he could just get them right. And, you know, make them look cooler than some other glasses out there.
“I was pretty obsessed with Google Glass when it came out,” Lavakare says.
“Oh,” I say. “So you were a Glasshole?”
“I was, I was!” he says with a laugh. “And then I was like, why are people calling me that?”
While we are talking, the words pop up onto the screen of the glasses I’m wearing. They show up in a Matrix-y green font that patters out across my vision. It does a pretty good job of transcribing the conversation, though it does split the word “Glasshole” into “Glass Hole,” which is honestly funnier.
Though Lavakare’s smart glasses are much more normal-glasses-adjacent than Google Glass ever was, they still can’t really help but look like smart glasses. The screen has a slight shimmer where the waveguides sit on the glass that is just visible enough to onlookers and is clearly noticeable to me when I am wearing them.
Aside from those minor gripes, the service itself works almost eerily well. At a bustling coworking space in San Francisco with many conversations happening around us, Lavakare and Nirbhay Narang, Transcribe’s CTO, talked to me while I wore the glasses. Most of the transcriptions were grammatically correct and were labeled with different speaker titles to make it clear who was talking. It all works so fast and so well, in fact, that the words popped up so quickly that I had trouble reading them as the conversation went on and new lines of text appeared almost simultaneously. The transcriptions are also sometimes a little grainy and hard to focus on at the moment. Still, with a little practice, it’s hard not to see how this would be extremely useful for people who are hard of hearing.
TranscribeGlass has a few competitors. Companies like Even realities and XRAI make glasses that look flashier and offer more features, like turn-by-turn directions and chatbot interaction. But Lavakare says the limited functionality is what makes his spectacles special.
“All these smart glasses exist, but no one’s found a great use case for them,” Lavakare says. “We think we’ve really found a use case that’s just insanely valuable to the end user.”
While he says these glasses can’t play music or use AI to answer questions, they only really need to do one thing well to get people to wear them: help them understand what is being said around them. Lavakare likens that feeling of missing out on a conversation happening around you to a kind of social isolation.
That said, he does hope to pack other conversational features into the glasses, with the goal of enhancing what you can glean from the subtext of a chat. One upcoming feature is language translation.
Narang and I have a short conversation to test the translation abilities. He speaks to me in Hindi while I speak to him in English. On my glasses, I see whatever he’s speaking to me translated into English on my screen. When I respond in English, the Hindi text pops up on his phone app. It’s a service that also seems to work well enough, though some words are mistranslated. That’s why the feature hasn’t yet come to the few hundred customers TranscribeGlass has now.
IMAGE COURTESY OF TRANSCRIBE
More Features to Come
There are other features in the works. Lavakare wants to let users have the option of translating a spoken language into something more like the syntax used in a visual language such as American Sign Language, which tends to have a different order of nouns, verbs, and tense than spoken English. Trusting that translation to AI, when most Deaf people can and do already read in English just fine, could cause some inaccuracies or misinterpretations. Lavakare acknowledges that potential for error, noting that he has talked with Deaf educators at the American School for the Deaf to try to get it right.
“Sign language grammar is actually very different than English grammar,” Lavakare says. “That’s why this is still experimental.”
He’s also testing an even more dubious capability—recognizing the emotion of a speaker based on tone of voice alone. Emotion tracking is a fraught topic in the AI space, albeit one people just can’t seem to help putting into smart glasses. While TranscribeGlass hasn’t released the ability to catalog emotions during a conversation, the team is testing it with the goal of releasing soon. It makes sense for helping with conversational comprehension, given that detecting how a person says something is often as important as knowing what they say.
Lavakare lets me test it out, switching on the feature while I’m wearing the glasses.
“Watch this,” he says. Then, “Hey Boone, how’s it going?”
His words pop up on the screen. I start to answer, and then a dialog tag appears with the emotion attached to his words: [Awkwardness].
I laugh and say, “Oh no, are we that awkward?” Then the tag pops up on my words: [Amused]. Now my words have my name next to it, which the platform had picked up when Lavakare said it earlier. As soon as I finish talking, it changes my dialog tag to [Awkwardness].
Well. Maybe this thing does work.