Published on March 30th, 2021 | by Emergent Enterprise0
Facebook Reveals Latest Wrist-worn Prototypes for XR Input & Haptics
A desired future for XR/VR/AR is one without controllers. That still appears to be some ways off as seen in this post at RoadtoVR by Ben Lang. Facebook has revealed a “wrist-worn haptic device” that would be the interactive component for hand motions and sensory experience. Yes, Oculus Quest 2 has hands free interaction but it is still clunky to work with. No doubt the big tech companies and ambitious startups will continue to work on technologies that provide controller free experiences which then would become another tempting feature for XR to go mainstream. Until then we will need to deal with clunky.
Facebook today revealed some of its latest research and vision for a wrist-worn input device that the company expects to form the basis of AR and VR interactions and haptics in the future. The device, still in a research prototype phase, senses electrical signals in the user’s arm to detect intentional inputs. In addition to functioning as a simple ‘button’ input, the company says the device can even enable accurate, keyboard-less typing, and more.
In a media briefing this week, researchers from Facebook Reality Labs Research shared some of their latest work in developing new input technology which the team believes will form the foundation of interactions for XR devices of the future. The group shared a concept video of what it believes will be possible with the technology.
Beyond the concept video, the researchers also discussed the work happening to bring it to fruition.
Input on the Wrist
The Facebook researchers seem increasingly convinced that a wrist-worn controller is their best bet as an ‘always on’ wearable input device that can enable “ultra low friction” interactions for XR experiences.
Facebook has continued to build atop the wrist-wearable input technology it picked up in an acquisition of CTRL-Labs in 2019.
The heart of the wrist controller is electromyography (EMG) sensors which can detect the electrical signals which control the muscles in your hands. Rather than just course movements, the researchers say that EMG can be used to sense individual finger movements with precision down to one millimeter. In a video shared by the company, Facebook says the movements of the hand shown below are detected entirely with EMG:
While the near-term use-cases of this kind of technology could be to enable an ‘always available button’ which you can use to confirm choices presented to them through contextually relevant AR systems, the researchers say, further out it will be used for manipulating virtual interfaces and objects, and even to type without a keyboard.
“It’s highly likely that ultimately you’ll be able to type at high speed with EMG on a table or your lap — maybe even at higher speed than is possible with a keyboard today. Initial research is promising,” the company writes. “In fact, since joining FRL in 2019, the CTRL-labs team has made important progress on personalized models, reducing the time it takes to train custom keyboard models that adapt to an individual’s typing speed and technique.”
The researchers shared what is purported to be a live demo of this personalized keyboard model in action, using the wearable prototype to enable reasonably fast typing without a keyboard: