Published on September 9th, 2020 | by Emergent Enterprise0
Facebook Wants its AR Glasses to Give Wearers Superhearing
The convergence of several emergent technologies such as augmented reality, artificial intelligence and spatial audio is resulting in interesting potential for XR experiences. Adi Robertson reports at The Verge that Facebook is developing enhanced audio features for its AR prototypes that gives users some compelling abilities. The most interesting conversations surrounding these new advancements have to do with security. In order for this tech to function it needs to collect what’s being seen and heard and that’s personal data. Security assurance is a difficult balancing act for XR developers like Facebook and Apple. What will they do with all this information? Be alert!
Photo source: Facebook
A new Facebook experiment shows how sound could play a major role in augmented reality. Chief scientist Michael Abrash and his team at FRL Research (formerly Facebook Reality Labs) released details today about what the team calls “perceptual superpowers” — AR systems that figure out what you’re trying to hear, then amplify it and dampen background noise. Combined with spatial audio features, the device creates the aural equivalent of a hologram overlay in a pair of glasses.
Facebook has pursued high-quality virtual sound for years, largely through its Oculus virtual reality headsets. FRL Research’s latest work focuses on AR applications. To give one example, “imagine being able to hold a conversation in a crowded restaurant or bar without having to raise your voice to be heard or straining to understand what others are saying,” the company explains.
AR glasses could do this by picking up audio with microphones, using contextual clues to gauge which sounds are important, and feeding those sounds through a noise-canceling earpiece. Conversely, if you’re on a phone or video call, improved spatial sound could project participants’ voices or other audio to specific parts of the room, increasing the sense that you’re really with somebody else — or “audio presence,” in FRL Research’s terms.
As Facebook acknowledges, the lab’s “perceptual superpowers” pitch is very similar to the function of existing hearing aids, which also amplify sound and reduce background noise. (One experimental system even uses brain implants to focus on specific voices.)
AR glasses, however, present unique opportunities for people with and without impaired hearing. Directional sensors and outward-facing or eye-tracking cameras can collect detailed contextual information, including your bodily orientation and where your gaze is focused. That helps the earpieces pinpoint an “acoustic spotlight” to amplify. And, of course, an AR system can mix its own soundscape with the amplified audio.