App that helps deaf communicate without sign language

Sagar Savla, a product manager in Google's Artificial intelligence Research group makes a presentation at the Google offices in Nairobi. [Peter Oloo, Standard]

Sagar Savla, 26, is a proud man. He is part of a team at Google that has developed an app that will change the way people communicate with the deaf.

The app - Live Transcribe - seeks to fundamentally reduce the use of sign language.

It is available for free on Google Play Store.

According to Sagar, the app is designed to transcribe speech in near-real-time, allowing people to communicate in situations where they might ordinarily be unable to.

“When the app is open, it immediately starts writing out what it hears in large, easy-to-read text. It adds punctuation, understands some contexts and is able to correctly capitalise phrases,” he explains as he demonstrates with his Android phone.

That app can be used in different languages including Kiswahili.

Live Transcribe has so far recorded over one million downloads.

It is an Android app available as a pre-installation in Pixel three phones in accessibility settings and via the Google Play Store for other phones.

This means the app is accessible to millions of Android phone users worldwide.

“It is noteworthy to mention that today 89 per cent of all mobile phone platforms in Africa are Android phones,” said Google Kenya Country Manager Charles Murito.

Mr Murito said this was in tandem with Google objective of accessibility and reach to many.

During the demonstration of how the app works at Google Kenya Offices in Nairobi, Sagar was showed how it easily recognises sounds including laughter, a knock, water, wind among other sounds.

“This is significant because one day in India a deaf mum who was using the app was able to tell that her child was crying and she was able to locate her and act accordingly,” he explained.

Sagar who was born and raised in Bombay, India lived with his grandmother, now 75, who slowly but steadily was losing her hearing.

“She would switch the volume of the TV to the highest level just to hear something from it," Sagar said.

I met a deaf colleague from Russia who together we developed the app from real-life inspiration,” he added.

They then took the app to Gallaudet University in Washington DC that schools deaf students.

Here the original app was refined and finally produced as the product it is today.

As a senior product manager in Google’s artificial intelligence (AI) research team, Sagar has been involved in building machine learning technology into products like pixel phone's camera, Youtube and Google Home.

Before this, he worked at Facebook on data science and virtual reality, on the best car dispatch system at Lyft and fraud detection at PayPal.