An innovative application has been on the market for a few months now that has extended the use of FaceTime to more and more users. Thinking about people with hearing disabilities and other people with disabilities, developer Jordi Bruin launched 'Navi' in November 2021, an application capable of creating subtitles and live translations, available on this video calling application.
Bruin, who is also the creator of Cibo, an iOS application that can scan and translate restaurant menus while traveling abroad, decided to create Navi while visiting WWDC .
After participating in a SharePlay session, he began to imagine ways to realize the idea of making a transcript among people with disabilities. Navi helps users on FaceTime in the following:
- Instant captions for FaceTime calls.
- Translate incoming subtitles in real time into more than 20 languages.
- Experience built specifically around Picture in Picture (PiP).
- On macOS, subtitles are directly superimposed on the video window.
- Captions are encrypted end-to-end via Apple's servers.
How was Navi created
Bruin, who as mentioned above also owns Cibo, explained to 9to5Mac how he created Navi:
“In April 2021, I participated in a hackathon and played with the MixToTelephonyUPLink API to allow mute people to talk on phone calls with Text to Speech. During WWDC I watched the SharePlay session and started creating image shapes to make the idea of transcription real again.”
Although the main objective was to help people with hearing impairments and other disabilities that prevent them from easily joining a video call, the app is also useful for people who don't speak the same language but still want to participate in real time.
“SharePlay adds a very simple API for developers to pass information between people in a FaceTime call, all encrypted and with very low latency. I had more than 500 people on TestFlight, but since testing required me to call them all, I had to find better ways, so in the end, I had to FaceTime many times from different devices. And it still amazes me when I use the live translation feature and can have a conversation with someone whose language I don't speak. It really feels like a science fiction movie sometimes,” Bruin added.
With these two audiences in mind, the developer still wants to continue improving Navi for people with hearing impairments and other disabilities. “I heard from some of the hearing impaired users on TestFlight that it's already super useful, so most of all, I want to hear from them if there's anything I can add to make the app work better for their specific needs.”
Navi has translation, so it can be used in FaceTime video calls
Currently, 'Navi' allows free translation into 20 languages: Polish, Turkish, Thai, Ukrainian, German (Austria), French (Switzerland, France and Canada), Finnish, Vietnamese, Italian, Swedish, German, Japanese, Portuguese (Brazil and Portugal), Spanish (Mexico and Spain), Norwegian, Danish, Croatian, Chinese (traditional and simplified).
However, starting in 2022, on an unknown date, users will need to make a one-time payment to enjoy the unlimited service. To check how it works, just follow these simple steps:
1. Start a FaceTime call.
2. Open Navi.
3, Press on the button that says 'Enable Captions' to invite the other people in the FaceTime call.
3. Accept the SharePlay invitation, and you're done.
KEEP READING