Published on July 21st, 2020 | by Emergent Enterprise0
Researchers Develop 3D Hand-sensing Wristband
The technology behind hand tracking in virtual reality seems to be improving rapidly. This announcement from Louis DiPietro at Cornell University reveals an innovative wrist wearable that can create a VR image of the hand and its movements just by monitoring “wrist contours.” Why is hand tracking important? It can analyze if a person is completing tasks in the correct order or procedure. It can add gestures like grasping and pushing or pulling to VR experiences. Enterprise uses for technology such as this innovative wearable from Cornell have lots of potential.
In a potential breakthrough in wearable sensing technology, researchers from Cornell and the University of Wisconsin, Madison, have designed a wrist-mounted device that continuously tracks the entire human hand in 3D.
The bracelet, called FingerTrak, can sense and translate into 3D the many positions of the human hand, including 20 finger joint positions, using three or four miniature, low-resolution thermal cameras that read contours on the wrist. The device could be used in sign language translation, virtual reality, mobile health, human-robot interaction and other areas, the researchers said.
“This was a major discovery by our team – that by looking at your wrist contours, the technology could reconstruct in 3D, with keen accuracy, where your fingers are,” said Cheng Zhang, assistant professor of information science and director of Cornell’s new SciFi Lab, where FingerTrak was developed. “It’s the first system to reconstruct your full hand posture based on the contours of the wrist.”
Yin Li, assistant professor of biostatistics and medical informatics at the University of Wisconsin, Madison School of Medicine and Public Health, contributed to the software behind FingerTrak.
Their paper, “FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist,” was published in June in the Proceedings of the Association for Computing Machinery on Interactive, Mobile, Wearable and Ubiquitous Technologies. It also will be presented at the 2020 ACM International Joint Conference on Pervasive and Ubiquitous Computing, taking place virtually Sept. 12-16.
Past wrist-mounted cameras have been considered too bulky and obtrusive for everyday use, and most could reconstruct only a few discrete hand gestures.
FingerTrak’s breakthrough is a lightweight bracelet, allowing for free movement. Instead of using cameras to directly capture the position of the fingers, the focus of most prior research, FingerTrak uses a combination of thermal imaging and machine learning to virtually reconstruct the hand. The bracelet’s four miniature, thermal cameras – each about the size of a pea – snap multiple “silhouette” images to form an outline of the hand.