Frank Willett | High-performance brain-to-text communication via handwriting

Virtual London Machine Learning Meetup - 30.06.2021 @ 18:30

We would like to invite you to our next Virtual Machine Learning Meetup.

Agenda:

• 18:25: Virtual doors open
• 18:30: Talk
• 19:10: Q&A session
• 19:30: Close

Sponsors
https://evolution.ai/ : Machines that Read - Intelligent data extraction from corporate and financial documents.

• Title: High-performance brain-to-text communication via handwriting (Frank Willett is a Research Scientist working in the Neural Prosthetics Translational Laboratory at Stanford University)

*Paper:
High-performance brain-to-text communication via handwriting | Nature
https://www.nature.com/articles/s41586-021-03506-2

Abstract: Brain–computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. So far, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping, or point-and-click typing with a computer cursor. However, rapid sequences of highly dexterous behaviours, such as handwriting or touch typing, might enable faster rates of communication.

Here we developed an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time, using a recurrent neural network decoding approach. With this BCI, our study participant, whose hand was paralysed from spinal cord injury, achieved typing speeds of 90 characters per minute with 94.1% raw accuracy online, and greater than 99% accuracy offline with a general-purpose autocorrect.

To our knowledge, these typing speeds exceed those reported for any other BCI, and are comparable to typical smartphone typing speeds of individuals in the age group of our participant (115 characters per minute).

In an effort to engage the machine learning community, we have publicly released all data, consisting of the neural activity recorded during the attempted handwriting of 1,000 sentences (43,501 characters) over 10.7 hours. The data can be used to explore Important next steps, which include reducing the amount of data needed to train the neural network decoder, and eliminating the need to recalibrate the system when neural activity changes over time.

Bio: Frank Willett is a Research Scientist working in the Neural Prosthetics Translational Laboratory at Stanford University. His work is aimed broadly at brain-computer interfaces and understanding how the brain represents and controls movement. Recently, Frank has developed a brain-computer interface that can decode attempted handwriting movements from neural activity in motor cortex. Frank has also worked on understanding how different body parts are represented in motor cortex at single neuron resolution. This work led to a surprising finding: what was previously thought to be “arm/hand” area of motor cortex actually contains an interlinked representation of the entire body. Prior to working at Stanford University, Frank earned his PhD in the Department of Biomedical Engineering at Case Western Reserve University.