Imagine the ability to control machines with your mind, instead of having to type on a keyboard or click on a mouse. Now Facebook’s parent company Meta is aiming for the next best thing—a new wristband that can, with the help of AI, infer electrical commands sent from the brain to muscles and convert them into computer signals, all in a noninvasive way. Although experts doubt it will replace keyboards and mice for traditional computing, it might have new uses for a wide range of applications, such as wearable interfaces for mobile devices, or thought-controlled assistive technologies for people with disabilities.
The bracelet from Reality Labs at Meta uses metal contacts placed against the skin to detect electrical signals from muscles—a technique known as surface electromyography (sEMG)—which are generated in response to commands from the brain. The highly sensitive new system transmits this data to a computer using Bluetooth to help it recognize gestures such as pointing and pinching in real time, findings detailed in Nature on 23 July.
The bracelet is not a direct interface with the brain. “It is not a mind-reading system. It cannot make you act in a different way as imposed by your will, it does not connect you to other people neurally, it does not predict your intentions,” says Dario Farina, chair in neurorehabilitation engineering at Imperial College, London, who did not take part in Meta’s research but has tested the technology. (Meta was unable to make anyone available for comment as of press time.)
How AI Enables Meta’s Wristband
Previous “neuromotor” devices, such as the discontinued Myo armband, also sought to use sEMG for computer interfaces. A key challenge these earlier devices faced was how they each needed time-consuming personalized calibration for each user to account for differences in human anatomy and behavior.
See how the device detects thumb swipes, finger taps, and handwriting gestures.Reality Labs at Meta
In contrast, Meta says its bracelet can essentially work off-the-shelf. The key was training deep learning artificial intelligence systems on data from more than 6,000 paid volunteers who wore the device. This generated models that could accurately interpret user input across different people without requiring individual calibration, says Joe Paradiso, head of the Responsive Environments Research Group at the MIT Media Lab, who did not participate in this study.
“The amount of information decoded with this device is very large, far larger than any previous attempt,” Farina says. “The device can recognize handwriting, for example, which would have not been conceivable before.”
The wristband uses surface electromyography to non-invasively measure electrical activity of the muscles that control hand gestures.Reality Labs at Meta
A possible concern with using this wristband is how users might not want every hand motion interpreted as input—say, if they had to scratch an itch, or pick up a glass of water. However, the device is trained to recognize only certain commands, while ignoring other gestures. “A potential user of the device can perform any activity of daily living without the risk to activate the device, and yet control the device with the specific gestures for which the device has been trained,” Farina says.
In tests, volunteers who had never previously tried the bracelet could use handwriting gestures to input text at 20.9 words per minute. (In comparison, mobile phone keyboard typing speeds average roughly 36 words per minute.)
Given this interface speed, “I doubt most computer users would rush out to buy this wristband if it becomes commercially available,” says Eben Kirksey, a professor of anthropology at the University of Oxford who studies the interplay of science, technology, and society. “Most people type at around 40 words a minute and highly skilled typists can bang out upwards of 100 words a minute. Since this new wearable device only enables users to write 20 words a minute, I doubt many people will want to take the time to learn how to use this new way to interface with computers.”
Instead, the new study argues the bracelet might prove useful in scenarios where keyboards or mice would be limiting, such as mobile and wearable applications. “It’s not a keyboard replacement. It’s something else, and I think things like this are needed for the computational paradigms that are coming,” Paradiso says.
For example, when it comes to the virtual reality and augmented reality glasses that Meta, Google, and others have pursued, interfaces previously envisioned for these devices include vision-based trackers that track the motions of hands held up in front of users, Paradiso says. However, the fatigue that can set in with this “gorilla arm” approach limits long-term use, and “the need to use space in front of you has its drawbacks,” Paradiso notes.
Because this new bracelet interprets electrical signals instead of motion, it could instead let users keep their hands to their sides and interact with devices using subtle finger motions. “Interfaces like these are needed for the everywhere-computing eyewear that’s emerging,” Paradiso says. “My favorite scenario is on a crowded train with my head-worn display catching up with the news, or communicating with friends. You just nudge it around with your hands by your side. Going for a walk, and so on—same deal.”
Applications for Accessibility
The researchers also suggest the wristband could help people with disabilities better interact with computers, particularly individuals with reduced mobility, muscle weakness, finger amputations, paralysis and more. “Some members of the disabled community have difficulty typing on conventional keyboards, or using computer mice,” says Kirksey, who did not take part in this research. “This device offers a new option that might help some members of this community, who have very specific bodily challenges, interface with computers.”
For such applications, a virtue of the new bracelet is that it is relatively easy to don and doff. In contrast, brain-computer interfaces (BCIs), which also rely on electrical signals from the brain, often require invasive brain surgery. Non-invasive BCIs do exist, such as ones that apply electrodes onto the scalp, “but a wristband is more ergonomic than a skullcap,” Paradiso says.
However, many questions remain before this new technology might help people with disabilities. “This type of wearable assumes typical limb shape and fine motor control,” says Solomon Romney, formerly the head of Microsoft’s Inclusive Design Lab. “As a limb-different person, I am always looking for ways to move activities to my limblet rather than continue to overload my typical hand. How easily can it be adjusted to fit non-typical limbs and-or musculature? How does it filter tremors? How effectively would it work for someone with no hands to wear on their ankle?”
Ultimately, “the main obstacle is large-scale deployment,” Farina says. “To be distributed to millions of individuals, the system needs to be robust across different anatomies and be used with minimal error rate. To match the error rate of a mouse or keyboard in a vast human population is certainly a huge challenge.”
