Showing posts with label voice banking. Show all posts
Showing posts with label voice banking. Show all posts

Monday, March 13, 2017

'Voice banks' step in to keep chronically ill patients from falling silent


For thousands of Minnesotans with neurodegenerative diseases, losing one's ability to speak is one of the most devastating consequences.

Every so often, Robert White breaks into song and serenades his 13-year-old son Kieran with a tender Irish lullaby, “That Little Boy of Mine,” taught to him by his father.
But the deep-throated voice that used to fill his family’s living room in West St. Paul now quivers and shakes. There are times when White, who has amyotrophic lateral sclerosis, or ALS, a terminal disease that impairs his motor skills, can barely finish the chorus before his voice dips to a hoarsened whisper.
For White, and thousands of other Minnesotans afflicted with neurodegenerative diseases, losing one’s ability to speak is one of the most devastating consequences.
Now, however, researchers have found a way to preserve the unique essence of a human voice — in all its idiosyncratic nuance and power — for people with incurable and often debilitating illnesses.
Using new voice database technology, University of Minnesota speech pathologists can record people saying hundreds of sentences and phrases, break them down into phonetic units, and then reconstruct a personalized voice that can be used on a speech-generating device. The end result of this process, known as “voice banking,” is a voice that is nearly identical to the person’s original, healthy voice.
While the technology is still new, many people feel they are in a race against time to store their original voices before they become unrecognizable to their families and friends. Even the simple act of saying “I love you” can be too much of a strain for people in the later stages of ALS or other neurodegenerative illnesses.
“This is about preserving a person’s essential dignity, rather than having to depend on a canned synthetic voice that many find dehumanizing,” said Dr. H. Timothy Bunnell, director of the speech-language laboratory in Wilmington, Del., that pioneered the voice-banking technology about a decade ago.
But the process of preserving a voice is an emotional one for families. It comes with the recognition that a loved one is dying, and that even a carefully reconstructed voice — built over a period of weeks or months — will never be an exact match to a person’s original speech. Even the most sophisticated audio technology cannot recapture the spontaneous emotion of someone who bursts into laughter, or the gentle timbre of a parent’s voice as they sing to a child, researchers say.
‘It’s bittersweet’
On a recent morning, Wendy Eickhoff, 49, struggled to contain her emotions as she began the arduous process of recording more than 1,600 preselected sentences at a speech laboratory tucked in a brick building on the University of Minnesota’s East Bank campus.
Last October, Eickhoff was diagnosed with a rare form of ALS known as bulbar onset, which first attacks the muscles that move the tongue, mouth and vocal cords. It was a particularly brutal diagnosis for Eickhoff, a self-described “talker” who works as a technology relationship manager at Wells Fargo. Slurred speech and weakened tongue muscles were among her first symptoms.
“It was devastating because I have always relished talking,” she said.
As Eickhoff rattled off sentences in a soundproof room, including lines from the “The Velveteen Rabbit” and “The Wizard of Oz,” her daughter Logan, 21, watched intensely through a window in an adjoining room. With a tinge of sadness, Logan expressed hope that voice banking would preserve the unique character of her mother’s speech, from her high-pitched laugh to the gentle way she called out “Good morning sunshine!” to her daughter each morning.
At one point, Logan broke down in tears after a speech pathologist calmly informed her that her mother’s recorded voice would survive her death and would always be available to the family.
“It’s difficult, really difficult,” said Logan, who is pursuing a degree in linguistics. “I don’t want to have to acknowledge that in the potentially near future that my mom won’t be able to speak. But I also feel lucky, because this means that even after she’s gone, I’ll be able to hear her voice. It’s bittersweet.”
Those who have been through the process speak of it in glowing terms.
When Robert White finished recording the last of 1,610 sentences last month, after five grueling sessions, he threw up his hands in victory and handed out assorted chocolates to the speech-language pathologists and young students who helped him through the process.
“There is a real sense of accomplishment when you get to the end,” said his wife, Argerie White. “I mean, how wonderful would it be, if Bob ever loses his voice, that we can still hear him sing?”
Sophisticated technology
Until recently, people like the Whites and the Eickhoffs relied on interactive whiteboards or robotic synthesizers, similar to that used by renowned physicist Stephen Hawking, to communicate with the world after illnesses had taken their speech. But voice-banking technology has become so sophisticated that it can now cull through thousands of phonetic syllables and sort them in a way that replicates the pattern of a person’s speech. The recording process emphasizes the repetition of vowels, because the medley of “ohs” and “ahs” are largely what make a person’s voice distinctive, researchers say.
While the technology is still being tested and enhanced, that has not dissuaded the regional chapter of the ALS Association from encouraging people with the disease to take advantage of the technology while their voices are still strong. The voice-banking service is now available at university speech clinics throughout the Upper Midwest, from Fargo to Duluth. The cost of creating a voice can run from a few hundred dollars to more than $1,000, depending on the provider. In Minnesota and the Dakotas, the cost for those with ALS is covered by the regional ALS Association.
“It’s crazy-amazing how well this works,” said Rebecca Lulai, a clinical supervisor and speech-language pathologist at the University of Minnesota.
Pete Klinkhammer, who remembers the exact minute he was diagnosed with ALS (11:47 a.m. on June 14, 2013), was among the first to test voice-banking in Minnesota and is now an outspoken proponent. Klinkhammer, 54, a gregarious former social worker from Albertville and bulky ex-college football player, has a bellowing laugh and a penchant for off-color jokes and puns. The technology enables him to keep speaking through an iPad, often late in the evening, when his throat muscles cramp and his voice starts to slur.
During a recent visit with his 28-year-old daughter Chelsea, Klinkhammer pulled out his iPad and began tapping out a few of his favorite puns as his daughter watched with amusement.
“A guy just threw milk at me. How dairy!” he wrote, throwing his head back in laughter. The words “I love you, Chelsea,” came out of a small speaker attached to his iPad with virtually the same inflection as his regular voice.
“That is definitely your voice, Dad,” gushed Chelsea, hugging him.
“This illness can come to define you because it takes away so many outward physical aspects of your being,” Klinkhammer said. “But voice banking epitomizes the idea that this illness can’t take everything away, because it can’t take away your voice.”
 Via http://www.startribune.com/new-technology-helps-minnesotans-with-neurological-disorders-preserve-their-voices/415954614/

Thursday, May 22, 2014

Phrase archive restores lost voices

Phrase archive restores lost voices

Staff Writer 



“I bake sweet-chestnut bread,” a volunteer says into a microphone.


“I no longer understand what’s going on,” she carefully reads out next.

The volunteer, Kotobuki Hayashi, 56, is reading short lines of text popping up on a computer screen in front of her. The phrases have been taken randomly from newspapers and books. Studio staff check for any misreads.


In an hourlong session, Hayashi gets through about 150 phrases that will be used to create synthesized voices for people with amyotrophic lateral sclerosis, also known as ALS or Lou Gehrig’s disease, who can no longer speak.


“I thought it would be great if I could help those people simply by recording my voice,” Hayashi said. “Also, it’s exciting to imagine that fragments of my speech will be used to reconstruct voices.”
Hayashi is one of around 200 volunteers who have participated in a so-called voice bank project that kicked off in November last year. The goal is to reconstruct the voices of ALS sufferers by creating synthetic ones using an archive of other people’s voices.


The technology was developed by a team of researchers led by Junichi Yamagishi, an associate professor at the National Institute of Informatics who specializes in speech synthesis.


ALS is a progressive neurological disease that attacks the nervous system and paralyzes muscles. As it develops, patients lose the ability to speak, and in some cases can lose their voices within six months of diagnosis, experts say. One ALS sufferer widely known worldwide is physicist Stephen Hawking.


According to the Japan Intractable Diseases Information Center, there are some 9,000 people with ALS in Japan. Many communicate by typing into a personal computer or tablet PC by using whatever muscles they still have, and having a synthesized voice read it out loud. But the voice sounds impersonal and robotic.


“Patients very much needed to communicate in their own natural voice. But no such system existed that could provide a personalized synthesized voice for them,” Yamagishi said in a recent interview in Tokyo with The Japan Times.


Yamagishi and his team set out to re-create their own voices, initially in trials at Britain’s Edinburgh University in 2011.


The project has been running for three years and has seen about 600 volunteers take part, with 10 patients using the software. It is considered to be in its evaluation phase.


In Japan, the project is still in its initial phase and Yamagishi needs to collect as many voices as possible. The recording is done at rented studios in Tokyo, Osaka and Nagoya.
He anticipates it will take one or two years to develop a synthesizer but thinks it could help people with ALS and possibly other disorders.


Yamagishi’s system analyzes the recordings, processing them by using statistical models of the components of speech, and produces a basic voice model for each age group, sex and dialect. This model then serves as the framework for synthesizing the patient’s voice.


“It’s like transplanting part of the volunteers’ voices,” he said. “We find donors whose background matches the patients’ voices, such as in terms of age and home town. We then transplant elements of the donors’ voices, such as the speed with which they move their tongues,” to reconstruct the patients’ voices.


Some companies in Japan already conduct personalized voice synthesis by cutting and pasting recordings of the patients’ own voices. However, this requires hours of recording and is physically impossible for some ALS patients or for those who are already mute.


Yamagishi’s technology requires a 5-minute recording of a patient’s voice. Even if some of the words cannot be pronounced, the system can draw on examples from volunteers to guess at the patient’s original pronunciation.


It also helps to hear the voices of the patients’ siblings, Yamagishi said, since close relatives often have a similar accent or tone.


“There is no particular cure for ALS and it’s really hard for their families to see (their loved ones) develop the illness,” Yamagishi said. It’s important for families to have something to help improve the quality of the patients’ lives even a little, and the voice bank may be one of those things, he said.
“Now we are conducting a large-scale demonstration experiment . . . I want many volunteers from all the regions across Japan,” he said.

Wednesday, August 7, 2013

Banking on your voice: Machine stores speech for patient's future use



— The Miami Herald
                
— Carole Shearn isn't quite sure when she will lose the ability to speak, but she is sure of one thing: Her voice will still be heard, even when the words can no longer be spoken from her lips.
The 70-year-old West Palm Beach, Fla., resident was diagnosed with ALS, also known as Lou Gehrig's disease, last October. Her form of ALS weakens the throat muscles first.

The progressive disease has no cure, but at the University of Miami Miller School of Medicine's Department of Otolaryngology, Shearn is the first patient to take part in voice banking - a program where patients who will eventually lose their voice due to diseases such as Parkinson's, ALS and cancer - can record key words, phrases and personalized messages to communicate when speaking is no longer an option.

Shearn uses Tobii Assistive Technology, a speech-generating device, which stores her recordings into categories. She can then use a mouse, touch the screen or even use her eyes to retrieve the sound files.

In case of an emergency, Shearn has even programmed a message saying "get help immediately" and "call 911." Tobii, which is compatible with Bluetooth technology, will call for her.
"Truthfully, I was so amazed when I found out about the Tobii," she said. "It makes me feel good that I can personalize my messages to whomever. It is my lifeline."

ALS is a neurodegenerative disease that attacks the motor neurons. As the disease progresses, these neurons begin to degenerate and stop sending messages to muscles. Eventually, individuals diagnosed with the disease lose the ability to move their legs, arms and body.

According to the National Institute of Neurological Disorders and Stroke, 20,000 to 30,000 people are living with ALS in the United States. The average lifespan for a person with ALS is three to five years.

Because ALS patients lose strength and movement in their limbs, Tobii has a built-in eye tracker. Shearn can sit in front of the device - without moving any part of her body but her eyes - and still communicate effectively.

Jocelyn Odlum, a speech pathologist at the University of Miami, met Shearn at an ALS support group in West Palm Beach and then began seeing her at the clinic. After evaluating Shearn, she got her started on voice banking. Shearn has been recording her voice every day for the past two months.
"Carole is an inspiration," Odlum said. "She took this disease and is doing everything she possibly can to be prepared."

Odlum says that once individuals are diagnosed with ALS, they should see a speech pathologist immediately so they can be introduced to these devices and helped. "Unfortunately by the time people come to see me they have no voice," she said.

Shearn has recorded basic phrases such as "Hello. My name is Carole," and "How was work?" But, she also has recorded some of her other favorite phrases: "What's up buttercup?," "What's new, super glue?," and "You snooze. You lose."

When Shearn was initially diagnosed, she had no idea what ALS was. She also didn't know that the disease had no cure. "That was very hard to hear," she said.

"I had tears in my eyes of course and I asked how long I would have to live and my doctor said he didn't know, three to five years, so we left the office on that," she said.
Shearn's daughter, Jennifer Wagner, had been suspicious of early symptoms she had been noticing: slurred speech and choking spells, even when she wasn't eating. She researched her mother's symptoms and learned about ALS.

"It was very daunting and difficult to read," she said. "I had a pretty good idea of what the disease was and what it was going to entail, and I didn't want my mom to go through that."

Although Shearn can still drive and walk without any assistance, Wagner drives her mother from West Palm Beach to the university. She also started an open Facebook page called Carole's Crusade, where people can become more aware about the disease and follow her mother's progression.

"I know that I cannot cure the disease so my main goal is to bring attention to it," she said.

For Shearn, keeping a positive mindset has helped her cope with the disease, but she acknowledges she is not as hopeful and optimistic as she was eight months ago because she notices herself "getting a little worse."

"I was certain I was going to live past 90," she said. "I would still like to." But if she doesn't, Shearn says she feels she has done everything she has wanted to do - traveled to Rome, Alaska, Paris, England, Italy, Spain and the tip of Africa, been surrounded by good people, and had a successful 36-year teaching career.

"My life has always been about people so I don't crave a lot of outside influence," she said. "Our family is small, but we are tight. That has always been everything to me. I feel my life has been blessed."

Read more here: http://www.sunherald.com/2013/08/07/4854978/banking-on-your-voice-machine.html#storylink=cpy