How AI Is Helping Restore Speech and Gesture After Paralysis

Communication is more than the words we choose. A nod can signal agreement. A wave can create connection. A raised hand or change in posture can add meaning that speech alone does not carry.

Researchers at the University of California, San Francisco, have now shown that a brain-computer interface can use machine learning to translate brain activity associated with intended speech and gestures into the movements of a digital avatar. The early research offers a glimpse of how people with severe paralysis might one day regain more of the expression that makes conversation feel fully human.

What the Researchers Created

The research team worked with three people who had paralysis affecting both speech and body movement. A thin array of sensors implanted over the brain’s sensorimotor cortex recorded patterns of neural activity while each participant attempted to speak selected phrases, make familiar gestures, or do both at the same time.

Machine-learning systems called decoders were trained to recognize the activity associated with those attempts. In two participants, the decoded signals controlled personalized full-body avatars that could produce speech while also nodding, waving, shrugging, or making other upper-body gestures.

The system did not create the participants’ meaning for them. It translated signals connected to what they were intentionally trying to express.

Why combining speech and movement matters

Speech and gesture are not separate layers pasted together after a thought is formed. In natural conversation, they often emerge at the same time. A gesture can reinforce a word, soften its meaning, communicate emotion, or sometimes replace speech altogether.

The researchers found that brain activity during simultaneous speech and movement was not simply the sum of the signals recorded when each action was attempted separately. The decoders performed better when they had been trained using combined speech-and-gesture activity. That finding matters because restoring communication is not only about producing accurate words. It is also about helping a person convey emphasis, context, and presence.

What the AI contributes

The implanted sensors record patterns of brain activity, but those signals do not arrive as recognizable words or movements. Machine-learning decoders learn the patterns associated with each participant’s attempts to speak and gesture, then translate them into commands that animate the digital figure.

In this partnership, the AI acts as an interpreter—not an author. It does not supply the thought, choose the message, or decide what the person wants to communicate. The intention remains entirely with the participant; the technology helps make that intention visible and audible.

What this does—and does not—prove

This was an early proof-of-concept study involving three participants, with two using the decoders to control full-body avatars. The system recognized a limited set of phrases and gestures in a controlled research setting. It also relied on surgically implanted sensors connected by wires to external equipment.

The results do not mean that this technology is ready for everyday use. More research is needed with more participants, a broader vocabulary, and a wider range of natural movements. Researchers are also working toward a wireless, fully implanted system.

What the study does demonstrate is that intended speech and gesture can be decoded together. That is a meaningful step toward restoring communication that conveys more than words alone.

Why this matters

For someone living with paralysis, restoring communication is not simply a matter of producing words. Expression includes timing, emphasis, movement, and the subtle signals that help one person feel present with another.

This research reflects the kind of partnership Bright AI Horizons was created to explore. The person provides the intention. Researchers and clinicians provide their expertise. Artificial intelligence helps translate signals that might otherwise remain inaccessible.

The technology is still at an early stage, but the possibility is profound: not replacing human expression, but helping more of it reach the world.

Sources

1. Highlight “Neuroprosthesis for paralysis enables simultaneous speech and body language” and link it to:
   https://www.nih.gov/news-events/news-releases/neuroprosthesis-paralysis-enables-simultaneous-speech-body-language
2. Highlight “Simultaneous speech and gesture decoding for multimodal communication in paralysis” and link it to:
   https://doi.org/10.1038/s41593-026-02446-2