Voice-Activated Vocal Joystick Makes Using the Internet Easy for the Handicapped

New software that has been developed at the University of Washington has the possibility to make surfing the internet a whole lot easier for people with various disabilities who find it had to hold and manipulate a mouse. It is called the Vocal Joystick and it uses the human voice to move the curser.

It can detect sounds 100 times a second and it turns that sound into movement on the monitor screen. It uses four vowel sounds, namely "ah," "ee," "aw" and "oo" to indicate the direction then other sounds make the curser move in the indicated direction. There are 8 different directions it can move in. It is easy to go from one vowel to another and the louder the sound, the faster the cursor moves. The sounds "k" and "ch" imitate the actions of clicking and releasing the mouse buttons.

There are different versions of the Joystick for browsing the web, drawing in a screen, controlling a curser and one for playing a video game. They also have one that can operate a robotic arm, and they are hoping to be able to create one that can control an electric wheelchair.

There have been aids on the market for years to help the handicapped do similar tasks such as eye trackers, the sip and puff, and head tracking systems, but each one of them has a drawback, for instance the eye trackers are really expensive and it requires the eyes to do two things at the same time - take in information and control the cursor. This can be the cause of a lot of confusion. As for the sip and puff devices, it must be help in the mouth and if the person wants to speak, they have to spit out. And the head tracking software requires a lot of neck movement and expensive hardware.

As long as the person can make a sound, they can use the Vocal Joystick. All it requires is a microphone and a computer that has a sound card.

One of the doctoral students, Jonathan Malkin, who also helped to develop the joysticks, used it to play a game named Fish Tale. It took him two minutes to train the program to recognize his voice and he moves the little fish around the screen without any trouble and raises his voice to make the fish escape from a bigger one.

To test the device, they used a group of about 8 spinal cord injury patients at the UW Medical Center, and the early results show that someone using the joystick and have just as much control as someone who is not handicapped would using a mouse.

The lead researcher is Jeffrey Bilmes, a UW associate professor of electrical engineering.

Source: University of Washington http://uwnews.washington.edu/