Harnessing Brain Signals for Communication

 

by Yael Arbel

 

"You survive, but you survive with what is so aptly known as 'locked-in syndrome.' Paralyzed from head to toe, the patient is imprisoned inside his own body, his mind intact, but unable to speak or move. In my case, blinking my left eyelid is my only means of communication..." (from Bauby, "The Diving Bell and the Butterfly," Fourth Estate, London, 1997, p. 12).

 

These are the words of Jean-Dominique Bauby, a former journalist and editor of the French magazine, Elle, who at the age of 43 suffered a massive brain stem stroke that left him completely paralyzed, unable even to speak. All he retained was his ability to blink the left eye. After seeing he could communicate "yes" and "no" by blinking, Bauby's speech-language pathologist, Henriette Durand, set up a special alphabet he could use to blink the letters of words. Thus, he was able to communicate with Durand, his other health care providers, family, and friends.

Ultimately, Bauby achieved the near-miraculous feat of dictating a 139-page book about his locked-in experience to his assistant, Claude Mendibil. The book, "The Diving Bell and the Butterfly," was published in France shortly before his death in 1997, and 10 years later was made into an Academy Award-nominated movie of the same title. The butterfly symbolized the words and thoughts trapped within the diving bell-Bauby's steel-trap of a body. Expressing those thoughts via eye blinks was by no means easy or fast. Mendibil had to sound out the French alphabet in frequency order, and Bauby blinked as soon as she uttered the right letter. The book took about 200,000 blinks to write in four-hour sessions over 10 months. It required the constant presence of Mendibil to produce.

If Bauby suffered his stroke today, he could have been able to write his book on his own, by using brain-computer interface technology being developed, in part, by communication sciences and disorders professionals. About nine years before Bauby's book was on the shelves, a paper published in 1988 in the journal Psychophysiology signaled the starting point for an alternative, less onerous communication technique for people with similar severe motor impairments. This paper, by cognitive neuroscientist Emanuel Donchin and his former student Lawrence Farwell, first described a brain-computer interface system, called that P300 Speller, that allows people to communicate without moving a muscle by using their brain waves in lieu of fingers. About 10 years later, the BCI-2000 package, which implemented the P300 Speller described by Farwell and Donchin, was developed and distributed to researchers to test with patients with amyotrophic lateral sclerosis (ALS or Lou Gehrig's disease).

The goal of our research at the University of South Florida BCI laboratory and at Northeastern University is to ultimately develop an interface that allows locked-in patients to communicate more quickly and independently, and to control their environment efficiently-for example, to reach for and manipulate objects. Another goal is to adapt the interface to control commercially available augmentative and alternative communication devices.

The brain interface at work

Here is a look at the BCI-2000 in action: Scott Mackler, a professor at the University of Pennsylvania's School of Medicine who has advanced ALS, uses the P300 BCI to communicate with students and others via e-mail and stay actively involved in his research on addiction. During the two and a half years he has used it, the system has maintained high accuracy level of 83 percent. Mackler is one of several people with ALS who interact with others daily via the P300 BCI as part of the New York Wadsworth Center's effort to develop a home version of the BCI-2000 system. Researchers in the Wadsworth laboratory facilitate these clients' use of the technology via the Internet.
But how does the system actually elicit words from somebody who can't move? By recording electrical brain activity and isolating a particular type of brain wave-the P300. This type of brainwave is recorded in what's known as the "oddball" paradigm, in which a person is presented with a sequence of events (for example, names). Each event should belong to one of two categories (for example, female and male names), one of which should be presented more frequently than the other (for example, an 80 to 20 ratio). It is critical for the person to perform a task that requires categorizing each of the events into one of the two categories. The events in the rare category elicit a P300.

The P300 BCI system records people's brain activity via an electro-cap worn on the head, an amplifier to process the incoming signal and computer programs that allow:

• The presentation of the oddball paradigm to the user through a visual, auditory, or tactile interface. For example, the visual interface is a 6 x 6 matrix with letters and numbers. The size of the matrix can be changed to accommodate the user's needs.
• The detection of the P300 in real time.

In the original design of the P300-Speller, the "oddball paradigm" was implemented by flashing, successively, and randomly, the six rows and the six columns of the matrix. The participant focuses on a specific character. In the sequence of 12 flashes, there are thus two categories: Two items contain the target character, while the other 10 flashes do not. Thus, as was predicted on the basis of decades of P300 research, the row and the column containing the target would elicit a P300, and the other 10 flashes would not. The system would then "type" the character at the intersection of the row and column that elicited a P300.

A major advantage of the P300-Speller, compared with other BCI systems, is that it required virtually no prior training of the user. Other BCIs require extensive training, and users' ability to control brain activity fluctuates due to lack of clear instructions on how to control this activity. The systems differ in the brain activity they rely on. For example, one that uses the "Mu" rhythms requires a relatively long training period of weeks before users are able to control the system with 80 percent accuracy. Users deploy the mind as a mouse to "control" a cursor to select items on a screen. In these systems, motor imagery (for example, imagining moving a hand) is typically used during training.
Returning to the P300 BCI, data from one recent study, led by Christoph Guger and published in Neuroscience Letters in 2009, suggest that after only five minutes of training on the system, close to three quarters of the 81 participants could spell a five-character word with 100 percent accuracy. Most of the remaining participants achieved an accuracy level above 80 percent.

It should be noted, however, that not everyone is the right candidate for the P300 BCI. To be eligible, a user must be able to:

• Perform a simple oddball task. To determine this, the tester presents the user with a simple oddball task, typically the letters X and O presented in different probabilities, and sees whether a P300 is elicited by the rare category-for example, the X.
• Understand and follow the categorization instructions. To produce the P300, cognitive abilities must be intact.

What's next?

Bauby's cognitive abilities were obviously sharp as ever, suggesting he would have benefited from the P300 BCI. It would have freed him from relying on an assistant. And although the differences in spelling rate between the P300 and Bauby's method are only marginal, the benefit of the BCI system lies in its provision of increased independence and an increased range of functions: The system can be much more than just a speller. It allows people to use their brain activity to surf the Internet, write e-mails, and-with some adjustments-control their environment.

That said, increasing the speed and accuracy of the P300-BCI is a major goal of our work at the University of South Florida BCI laboratory. Working with electrical and mechanical engineers, we're fine-tuning an interface to help users reach and manipulate objects. We're also developing a "switch" that would allow users to control augmentative and alternative communication devices with the P300 BCI. Switches to control AAC devices are typically offered when the ability to speak intelligibly has deteriorated and only minimal, residual movement is left, which is typical in later ALS. In many cases, ALS patients use an AAC tied to eye gaze, although even these devices become inefficient or unstable as the disease progresses.

Although the P300 BCI was developed as a speller, it is a communication system that allows people to communicate with the environment at different levels. Not only can they spell words, but they can select common phrases or use the interface as a keyboard to control their computer or surf the Internet. They do this by selecting commands that are communicated to an external device. Given that the P300 can be elicited by visual, auditory or tactile events, is not modality specific, and does not require a motor response, we are hoping to develop a system that presents events in the user's strong modality-for example, a visual display for people with hearing loss and auditory input for people with poor vision-and that is suitable for users who are unable to execute a motor response.
As the P300 BCI develops into a home-based AAC device, its use among "locked-in" patients will likely increase, and SLPs will play an important part in its optimal operation. Some users will benefit from a device that allows them to indicate a simple "yes," "no," "stop" answer, while others will be able to control an 8 x 9 matrix that emulates a computer keyboard. It is the role of the clinician to determine the level of complexity that best suits the user.

What if the P300 BCI had been available to Bauby? Although his spelling time would have been only moderately reduced, he would have been able to work independently, at his own pace. He would have been able not only to write his book, but also to edit it. He could have used the P300 BCI not only for spelling, but also for communicating with his family members, for example by sending and receiving e-mails, and-if he lived today-by using video-phone applications such as Skype. With some adjustments, the P300 BCI may have allowed Bauby to use a television remote and otherwise control his environment, freeing him, at least somewhat, from the diving bell that so constrained him.

Yael Arbel, PhD, CCC-SLP, is a visiting clinical assistant professor in the Department of Speech-Language Pathology and Audiology at Northeastern University in Boston. She is also affiliated with the Departments of Psychology and Communication Sciences and Disorders at the University of South Florida in Tampa.

cite as: Arbel, Y. (2013, January 01). Harnessing Brain Signals for Communication. The ASHA Leader.

Web source: http://www.asha.org/Publications/leader/2013/130101/Harnessing-Brain-Signals-for-Communication.htm

Sources

Donchin, E., & Arbel, Y. (2009). P300 Based Brain Computer Interfaces: A Progress Report. Foundations of Augmented Cognition. Neuroergonomics and Operational Neuroscience Lecture Notes in Computer Science, 724–731.
Farwell, L. A., & Donchin, E. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology, 70(6), 510–523.
Guger, C., Daban, S., Sellers, E., Holzner, C., Krausz, G., Carabalona, R. ...Edlinger, G. (2009). How many people are able to control a P300-based brain-computer interface (BCI)? Neuroscience Letters, 462(1), 94–98.
Mak, J. N., Arbel, Y., Minett, J. W., McCane, L. M., Yuksel, B., Ryan, D. ...Erdogmus, D. (2011). Optimizing the P300-based BCI: current status, limitations and future directions. Journal of Neural Engineering, 8(2), 1–7.
Sellers, E. W., Arbel, Y., & Donchin, E. (2012). P300 Event-Related Potentials and Related Activity in the EEG. In Wolpaw, J. R. & Wolpaw, E. W. Brain-Computer Interfaces: Principles and Practice. Oxford, N.Y.: Oxford University Press.
Sellers, E. W., Vaughan, T. M., & Wolpaw, J. R. (2010). A brain-computer interface for long-term independent home use. Amyotrophic Lateral Sclerosis, 11(5), 449–455.