Monday, October 16, 2017

The eyes have it: how technology allows you to speak when all you can do is blink

Closeup of an eye

Developments in eye-gaze technology – which converts minute movements of the eye into spoken words – are opening up undreamed of opportunities for people with motor neurone disease

Steve Thomas and I are talking about brain implants. Bonnie Tyler’s Holding Out For a Hero is playing in the background and for a moment I almost forget that a disease has robbed Steve of his speech. The conversation breaks briefly; now I see his wheelchair, his ventilator, his hospital bed.
Steve, a software engineer, was diagnosed with ALS (amyotrophic lateral sclerosis, a type of motor neurone disease) aged 50. He knew it was progressive and incurable; that he would soon become unable to move and, in his case, speak. He is using eye-gaze technology to tell me this (and later to turn off the sound of Bonnie Tyler); cameras pick up light reflection from his eye as he scans a screen. Movements of his pupils are translated into movements of a cursor through infrared technology and the cursor chooses letters or symbols. A speech-generating device transforms these written words into spoken ones – and, in turn, sentences and stories form.
Eye-gaze devices allow some people with limited speech or hand movements to communicate, use environmental controls, compose music, and paint. That includes patients with ALS – up to 80% have communication difficulties. Other conditions that can benefit from these devices include cerebral palsy, strokes, multiple sclerosis and spinal cord injuries. It’s a far cry from Elle editor-in-chief Jean-Dominique Bauby, locked-in by a stroke in 1995, painstakingly blinking through letters on an alphabet board. His memoir, written at one word every two minutes, later became a film, The Diving Bell and the Butterfly. Although some still use low-tech options (not everyone can meet the physical or cognitive requirements for eye-gaze systems; occasionally, locked-in patients can blink but cannot move their eyes), speech-to-text and text-to-speech functionality on smartphones and tablets has revolutionised communication.

 A person using an eye-gaze device. Photograph:
Eye-gaze technology has also evolved rapidly with open source and lower-cost options, solid NHS funding, and improved accessibility – some users are as young as 13 months. Recent prototypes have implanted similar technology into smartphones and glasses. But in practice, even some newer systems allow for only eight words per minute. Technical hitches during my visit mean that Steve reaches 3 wpm at times – our average speech rate in free-flowing conversation is 190 wpm. “Eye gaze allows me to do anything from Skype my wife to playing games,” he says, “but the downside is that sometimes I can’t really join in with conversa
Yet Steve still sounds like Steve. He preserved his speech before it disappeared through voice banking, recording around 1,500 phrases. Now his communication device generates an infinite number of sentences from those sounds, his identity in some way preserved. Prices start at around £100 for voice-banking services; some charities offer equipment loans and financial support.
In the Netherlands, Peter Desain, chair of artificial intelligence at Donders Institute for Brain, Cognition and Behaviour, is researching brain computer interfaces (BCI), a technique where brain signals are analysed and used to control external devices. BCI recently enabled three paralysed patientselsewhere to type with their thoughts through brain implants. That system, still in early development, is slow (2-8 wpm), expensive, and can take months to learn.
But Desain’s NoiseTag method, developed at Radboud University Nijmegen, does not require surgery. Instead, users wear a headset fitted with electrodes that record brain activity. They look at a screen with a matrix of flashing characters, each tagged with a specific code. Focusing on a given character causes its signature to appear in brainwaves recorded by the headset; the corresponding character is typed, and then a device “speaks” a completed sentence. The system exploits machine learning and predicts a user’s response for each pattern; this means that limited, if any, calibration is needed – one letter can be typed every one or two seconds with 95% accuracy. Twenty patients with ALS are trialling the technology, with a spin-off aiming to scale up numbers over the next year and extend the method to the internet of things (IoT).
Recently, Facebook has announced it is stepping into the BCI ring, ready to decode neural activity within two years using a yet-to-be-developed non-invasive technique at the mysterious Building 8 in Menlo Park, California. Mark Zuckerberg believes we will soon be typing using only our thoughts at five times the speed currently managed on smartphones. In addition, Elon Musk has just founded the $100m company NeuraLink and says we will all be benefiting from its BCI technology within a decade. Others express healthy scepticism; bandwidth and biocompatibility remain significant challenges, let alone surgical risks and our still-evolving understanding of neuroscientific princip

In the US, Darpa (the Defense Advanced Research Projects Agency) has also invested $65m in neural implants to restore speech, hearing, vision and memory in those with brain trauma and neurodegenerative disease. One group is working to wirelessly record signals from up to 1m neurons at once (currently, BCI reads around 100 neurons) by implanting tens of thousands of microdevices called neurograins across the brain. This “cortical intranet” will read out and write in information – brain enhancement might follow. Neuroprostheses have already shown promise in improving memory in rats and decision-making in monkeys; humans are another matter, of course. Perhaps that is why Kernel, a $100m start-up that aims to build “the world’s first neuroprosthesis to enhance human intelligence”, has quickly shifted focus. Founder Bryan Johnson says invasive BCI “is really interesting, but not an entry point” into a commercially viable business.
Meanwhile, Facebook has sought to dampen concerns around privacy and brain hacking: “This isn’t about decoding random thoughts. This is about decoding the words you’ve already decided to share by sending them to the speech centre of your brain. Think of it like this: you take many photos and choose to share only some of them. Similarly, you have many thoughts and choose to share only some of them.” Of course, photos not meant for sharing don’t always stay unshared …
“I already have a [prototype] BCI,” Steve tells. “My death wish is when I don’t even respond with BCI.” His words bring to mind something written by Bauby, mourning the loss of true conversation: “Sweet Florence refuses to speak to me unless I first breathe noisily into the receiver that Sandrine holds glued to my ear. ‘Are you there, Jean-Do?’ she asks anxiously over the air. And I have to admit that at times I do not know any more.”
For many like Steve, communication technology is not about sophisticated brain enhancement; it is simply about existence.

Wednesday, September 27, 2017

Brainternet: one step closer to mind controlling the internet

A team of South African scientists have managed to hook up a human brain to the internet in real time for the first time. Dubbed Brainternet, it is a significant first step toward a plethora of brain-controlled IoT applications.
The researchers used a tiny Raspberry Pi computer paired with an EEG, a device that receives electrical information from electrodes placed on a person’s head. The device sent information instantly to a website, where any person or device was able to access it in real time.
The Brainternet team is led by Adam Pantanowitz, electrical engineer, researcher and lecturer at Wits University in Johannesburg.


Adam Pantanowitz
In a press release, Pantanowitz explained:
“Brainternet is a new frontier in brain-computer interface systems […] Brainternet seeks to simplify a person’s understanding of their own brain and the brains of others; it does this through continuous monitoring of brain activity as well as enabling some interactivity.”
In other words, while the Brainternet discovery is still limited to a mere display of brain activity, other researchers around the globe are working on turning EEG electrical signals into actionable commands that computers and other internet-enabled devices can understand and follow, called brain-computer interfaces (BCI).
With BCIs, typing would become way faster, medical conditions would not be as disabling, and Elon Musk’s NeuraLink and Bryan Johnson’s Kernelare among the most ambitious projects in this front.
Pairing a BCI with a Brainternet-like device is the step forward that will begin erasing the boundaries between the human brain and machines, allowing us to envision a very near future where our knowledge and physical reach can expand as we have instant access to the internet. 
As mobile internet becomes more widespread and with the proliferation of companies working in interfaces for human interaction like Google Glass, Oculus Rift and other virtual and augmented technologies, it is not a stretch to envision that a tight integration of thoughts and machines will be common within our lifetimes.
Wikipedia, news, social media and all kinds of online content are getting closer to becoming seamlessly integrated with the human brain’s electrical activity. Met someone you like? Find them on Instagram or Facebook right away. Met a Silicon Valley entrepreneur or talent? Connecting to Lifographwould give you all the relevant info you need to decide whether they are good fit for your company or not—in a matter of seconds. 
Furthermore, as we move on from the IoT to the Tactile Internet, the flow of real-time information from your brain to internet-connected devices could get your coffee brewing with a thought from bed, or enable a surgeon to control a robotic arm and save a child in Africa. 
The possibilities are as endless, as are the profound changes BCI and Brainternet-like technologies imply for our tech-dependent existence.

LipSync Helps Paralyzed People Use Computers and Mobile Devices

The Neil Squire Society and the Google Foundation developed the mouth-controlled joystick

 14 September 2017

For people who are paralyzed or have mobility issues due to injury or illnesses, using a mouse, keyboard, or touchscreen can be difficult, if not impossible. And computer interfaces designed for them can be prohibitively expensive.
Giving people with disabilities better access to the world around them is the aim of the Neil Squire Society, a nonprofit in Burnaby, B.C., Canada. The organization is named after a man who became paralyzed in the early 1980s from injuries he suffered in a car accident. To help him communicate, Squire’s cousin, an engineer, connected an Apple 2E computer to a Morse code transmitter that Squire could control using “sip and puff” technology, which sends electrical signals by sipping and puffing on a tube. The device became known as the Joust.
Last year the society teamed up with the Google Foundation to build a more advanced version of Joust: the LipSync, a joystick that allows a person to control a computer cursor with minimal head and neck movements. A hollow mouthpiece is attached to a sensor on the joystick that requires only slight pressure to move a cursor on a computer screen. Users can “click” the left and right mouse buttons by inhaling or exhaling into the mouthpiece.
“While similar solutions exist for desktop computers, they can cost up to US $1,500 and do not work well on mobile devices,” says Chad Leaman, the society’s director of innovation. He’s also the  founder of its Makers Making Change initiative, which connects tinkerers with people who have disabilities and need assistive technologies.
“What makes the LipSync unique is that we are releasing it open-source, so it can be built for about $300 worth of parts by anyone who wants to volunteer their time to put the unit together.”
Leaman gave a presentation about Makers Making Change at this year’s CSUN (California State University, Northridge) Assistive Technology Conference, which The Institute attended. Recently the Makers Making Change group released a number of other low-tech assistive technologies that can be made with a 3D printer, including tools to help people with limited use of their hands hold pens, turn keys in locks, and open bottles.


Makers Making Change has held hackathons throughout the year in which students and other makers help build the LipSyncs. The organization sends engineers to the events to guide the participants through the building process. They provide instructions as well as 3D printers and soldering irons as needed. The LipSyncs are then donated to rehabilitation centers, disability organizations, or directly to those in need.
At the society’s first hackathon, held in January in Vancouver, volunteers built a system that could help a paralyzed man alert his wife, who is hard of hearing, if he’s in distress. The LipSync connected with a vibrating alarm outfitted with flashing lights that could alert her even if she’s sleeping.
The society recently held a make-a-thon in partnership with TOM:Alberta and the University of Calgary. The participants, including a team of IEEE student members, built more than 20 LipSyncs during the weekend. Events also were held in June in Philadelphia and Seattle as part of the National Week of Making.
“This year alone, with the help of the maker community, we’ve built 150 LipSync devices,” Leaman says. “Our goal is to build 1,500 more over the next year and a half. We need engineers and students across Canada and the United States to help bring the device into their communities.”
For more information on how to make LipSyncs or other open-source assistive technologies, visit the Makers Making Change website.

Friday, September 22, 2017

Here’s How The Trend Of Assistive Tech Is Rapidly Changing

09/21/2017 07:22 am ET Updated 1 day ago
The Kenguru Wheelchair Car is just one of many advancements in Assistive Technology
While Obamacare hangs in the balance and late night talk show host Jimmy Kimmel stirs the pot in a big way, technology is stepping in to potentially fill the gap in healthcare via inspiring and innovative ways. The rising trend of what is called Assistive Technology is fascinating to track. Everything from exoskeletons synced with apps that help with rehabiltation after stroke to Verily’s (formerly Google’s life sciences arm) tremor spoon already on the market for use by people with Parkinson’s disease, we are witnessing devices that were completely unheard of even just a few decades ago. An umbrella term that includes adaptive, and rehabilitative devices for people with disabilities, Assistive Technology advances are those that will change the future of health and well-being. 
Indeed, the World Health Organization (WHO) estimates that 285 million people are visually impaired worldwide. Another 360 million people globally suffer from hearing loss. Over 70 million people use a wheelchair. Thus, there are a staggering number of people who need or could benefit from assistive products to maintain or improve certain physical functions. From hearing aids to memory aids, assistive devices are in high-demand.
However, the news is that certain strides in this arena are actually being developed by companies outside of the U.S. For example, Laevo BV, an exoskeleton company based in The Netherlands, is the manufacturer of the like-named Laevo and actually introduced the first product in 2015. The Laevo is designed to prevent, reduce or relieve work-related back pain. The made-to-measure device, an exoskeleton-of-sorts once worn, is designed for various activities including repetitive heavy lifting and bending. The Laevo is customized but has interchangeable structure sets allowing various individuals on, for example, a construction team, to utilize the same device. 
While some construction workers, for example, may never suffer a crippling injury, their bodies can and typically do deteriorate after years of work. With tenuous healthcare situation, many are starting to research such devices to maximize the care of one’s being. “The design of the Laevo respects the human body and lets the user stay in control of the movements, “ explains Laevo founder Boudewijn Wisse. Already sold in 15 countries around the world and is now making its debut in America.
Subscribe to The Morning Email
Wake up to the day's most important news.
Laevo In Action
Another offering in assistive devices is being developed out of Israel. Open Sesame is a touch-free smartphone app that is the first of its kind and is designed for quadriplegics and people with disabilities. The app uses the standard front-facing camera of any Android device, to track users’ head-movements and provide them with a means of using mobile devices completely touch-free. Complemented by voice commands, and compatibility with any Android application, the companies Open Sesame technology provides private and independent use of a smartphone or tablet. This offering is poised to hit the U.S. market soon as the product becomes more customizable no matter what the state is of one’s physical condition.
However, some critics have cautioned that such assistive devices prevent greater work toward independence thereby hindering an individual to reach one’s own true potential. Other experts have said that the supportive products might be matched inappropriately to the needs of the user or be too complex, thereby exacerbating an already challenging physical or mental state. 
While such concerns will continue to be researched, this particular area of technology has no signs of slowing down soon. Indeed, this particular area of technology may face many challenges but also, potentially, many triumphs as it continues to develop.

Monday, September 11, 2017

How Microsoft's technological and social impacts have changed the world

Jason L Ward is a columnist at Windows Central. He provides unique big picture analysis of the complex world of Microsoft. Jason takes the small clues and gives you an insightful big picture perspective through storytelling that you won't find anywhere else. Seriously, this dude thinks outside the box. Follow him on Twitter at @JLTechWord. He's doing the "write" thing!

Two main goals stand out as primary objectives for many companies: a desire for profitability, and the goal to have an impact on the world. Microsoft is no exception.
Microsoft's story is founded on the goal of having a world impact. Under the leadership of its first CEO Bill Gates, and on to his successor, Steve Ballmer, to its current leader Satya Nadella, Microsoft has been an integral part of social change and world affairs.
Nadella defines Microsoft as the "do more" company. Its mission as a platform provider is to equip individuals and businesses with the tools to "do more." Microsoft equipped these entities to achieve their goals and has been a global force that has profoundly affected communities, other companies, and individuals.

A computer on every desk and in every home

Four decades ago, Microsoft Founders Bill Gates and Paul Allen set the ambitious goal of putting a PC on every desk and in every home. Forty years later that goal has been largely realized. Over a billion PCs are now in use in businesses and homes around the world.
Ironically, in those early days of computing naysayers mocked the idea of personal home computers. The large enterprise-focused devices that defined what computers were, and what they were useful for, were not something doubters could reconcile as useful tools in homes. This negative perception didn't hinder Microsoft's objective to define personal computing, however.
In a pivotal deal with IBM in 1980, Microsoft created the PC-DOS operating system for a fee of $80,000. Recognizing that other PC makers might attempt to copy it, Microsoft retained the OS's copyrights. Eventually, Microsoft's MS-DOS became the foundation of the Windows OS that propelled the company's personal computing vision around the world. Through partnerships with PC makers, Windows became the dominant PC OS in businesses and homes around the globe. Microsoft grew from a company that employed just 30 people to a multibillion global force that employs 124,000 individuals and affects business, social and global change.

PC power to the people

With user-friendly tools like Windows, Office and MS Paint, Microsoft was successful with making computing, something that was remote and foreign to most people, a personal and easy experience. Consumers and businesses also benefitted from the fact that Microsoft's platform became the dev box and target of a massive community of developers who ultimately supplied Windows with 16 million programs.
Microsoft not only put a PC in every home and on every desk but was crucial to defining the relationships between OEMs, platform providers, developers, businesses and consumers that are the foundations of the present age of personal computing. Microsoft mainstreamed personal computing to the extent that PCs are now available in a range of formsand price points. Third-world countries, where there is no or limited electricity, are even provided with durable wind-up PCs.
PCs and Windows have become the fundamental tools that have helped authors write best sellers, artists create masterpieces, musicians craft music, social service and volunteer organizations monitor resources, manufacturers optimize production, banks power ATMs and much more. Their integration in almost every part of our lives has changed our world.
Though the PC market declined after the smartphone's advent, the existence of these "modern mobile PCs", is a testimony of Microsoft's impact on the world and personal computing. Especially since Microsoft's $150 million contribution to Apple in 1997 saved the Cupertino company, enabling it to exist long enough to introduce the world-changing iPhone in 2007.

Business and government the Redmond way

Many of the world's businesses, which drive local and global economies and have intricate relationships with and impact on local and global communities, run on a Microsoft-based IT infrastructure. From Windows Server to Intune device management, to Azure cloud services, to Surfacedeployment, Office 365 and more, Microsoft is the platform that many large and small businesses around the world run on.
The global scope and the impact of these relationships cannot be overstated. Multibillion-dollar companies rely on the integrity and reliability of Microsoft's tools daily. We have witnessed the impact and costs to businesses when these systems are compromised.
Businesses are not the only high-profile consumers of Microsoft tools. Governments, from local municipalities to agencies like the U.S. Department of Defense to various governments around the world, such as China, rely on Windows and other Microsoft services for smooth operations and security.
It is a testimony to the powerful role Microsoft plays in global affairs that its tools are relied upon by governments around the world.

Head of the class

Despite Google's success in the U.S., Microsoft in Education is still the dominant force in school systems around the world.
To bolster its efforts, Microsoft purchased and integrated the popular game Minecraft into its education strategy. It also created Windows 10 S, which runs only Store apps and is bringing affordable Windows 10 S PCs to elementary and secondary school systems, and the Windows 10 S Surface Laptop to college students. With various management tools like Intune for Education Microsoft has improved administrators abilities to manage and deploy hardware.
Microsoft's goal with its education efforts is to familiarize the next generation of decision-makers and employees with the Microsoft toolsthat it has established as the drivers of businesses and governments around the world.
A seat at the table
Microsoft's position of global influence gives its leadership a voice on matters of moral consequence and humanitarian concern. The company notably exercised that voice in its stance against the Trump administrationin relation to its handling of the events in Charlottesville, Va., where an anti-protester was murdered and others were injured during a white supremacist rally.
More recently, Microsoft President Brad Smith, who recently left the Digital Advisory Committee due to the aforementioned concerns, expressed Microsoft's opposition to the Trump administrations ending of the Deferred Action for Childhood Arrivals (DACA) program.
This program currently protects 800,000 U.S. immigrants that were brought to the country as children from deportation. Smith expressed the following of the 39 "Dreamers," (as they are called), that Microsoft employs:
In short, if Dreamers who are our employees are in court, we will be by their side.
Microsoft's resources have also been employed to provide relief to victims of Hurricane Harvey. The massive wealth Bill Gates reaped from the company he cofounded has been used to provide education support, health care efforts and other philanthropic global work via the Bill and Melinda Gates Foundation.

Looking forward

Microsoft is a company built on a dream that has been largely accomplished. Though it has had many misses, including smartphones, Zune, and Microsoft Bob, the company is looking forward. Surfaceintroduced a new PC category, Windows 10 on ARM opens the door to potential telephony-enabled PCs, and Windows Mixed Reality and HoloLens position the company to democratize augmented and virtual reality for a future of holographic computing.
Microsoft's influence raises some concerns as well. It's AI-driven camera technology that can recognize, people, places, things, and activities and can act proactively has a profound capacity for abuse by the same governments and entities that currently employ Microsoft services for less nefarious purposes.
Additionally, visionaries like Gates, Tesla's Elon Musk and physicist Stephen Hawking have concerns about rouge AI and quantum computing.

What the future holds is uncertain, but one thing is clear: Microsoft has changed the world, and its impact will leave enduring effects long into the future.