Encouraging inclusivity with an interactive tool for learning the ASL alphabet
Fall 2017
Inspiration
A longtime friend of mine who is hearing, is the daughter of profoundly deaf parents and is family to many deaf relatives. Through her, I was introduced to the deaf community and I was very intrigued by sign language, although for many years I experienced a sense of helplessness and shyness when trying to communicate with her family. Reflecting on the dynamic of this language barrier, I realized that the slight discomfort I felt was minute in comparison to the adversity they face each day. Upon learning the ASL alphabet, I felt significantly more confident communicating with deaf people and this small effort was well received. This personal experience inspired me to create a fun and interactive way to learn the basics of ASL.
Challenge
The Canadian Association of the Deaf estimates that there are approximately 357,000 profoundly deaf Canadians and 3.21 million hard of hearing Canadians. Hearing loss has been associated with lower quality of life by way of social isolation, depression, safety issues, mobility limitations and reduced income and employment.
Outcome
An interactive tool for learning the American Sign Language (ASL) alphabet that also encourages empathy, understanding and inclusivity. To be used by children in educational settings and implemented on a large scale to ensure everyone has a standard knowledge of deaf culture and communication.
If everyone had this basic knowledge of ASL it could lead to more successful interactions between the hearing and the deaf. This small shift could potentially result in increased inclusivity and accessibility for the deaf, which would ultimately improve their quality of life.
In the following video, team member Yael Hubert, demonstrates the working prototype in action.
Research & Exploration
Primary research, including flow diagrams, empathy maps, and SWOT analyses
Desk research of scholarly articles to understand the social science of deafness
In-depth reviews of personal stories and case studies of the deaf and their parents and children
Key Insights
American sign language consists of thousands of signs that are conveyed through hand gestures, facial expressions and body language
Existing sign language translation gloves have failed to capture the nature of ASL and although they are presented as devices to improve accessibility, they ultimately serve hearing people
Improving quality of life of the deaf should not focus on how the deaf can make it easier for the hearing to understand them, but instead, how the hearing can have a basic understanding of deaf culture and communication
Making the glove
The glove was crafted with care, making it easy and comfortable to wear, robust and colourful. The game-like program makes the learning process more exciting, enjoyable and almost imperceptible, especially for children. In the following image, I am wearing the glove in the midst of fabrication.
Circuitry
The ASL Glove uses one flex sensor per finger, connected to an Arduino Uno. These flex sensors output a resistance value to the Arduino's serial port. As the flex sensors bend, their outputted resistance values change. The values of each of the five flex sensors indicate which fingers are bent and to what degree. To make this technology wearable, the wiring is confined to a mini breadboard to keep the size down.
Fabrication
In order to allow the flex sensors to slide up and down the finger as it bends, they are attached to the tip of each finger with thread and sewn down the length of the sensor, creating loops. These loops were sewn loose enough that the sensor can freely slide within them, but tight enough that the bend of the sensor remains true to the bend of the finger. Coloured thread was used not only to improve the glove's aesthetics, but also to increase usability, in that we could match the colours to those in the illustrations that would later be used in the program interface.
Data Collection
For the program to know when a letter was being signed correctly by the user, each letter of the alphabet had to be tested several times. This testing was conducted with several users, as the sensors bend differently based on the size of the user's hand. The resistance output values recorded during testing were used to determine the range for every finger for each letter. This allowed the program to recognize when each finger was in the correct position for any given letter, regardless of variations among users.
The following code checks if the letter A is being signed correctly.
Coding the program
The program created for the glove was built in Processing, because it has the ability to communicate with the Arduino's serial port, allowing it to receive values from the flex sensors. To make the program game-like, it chooses a random letter and displays both the English letter and the ASL hand position. The program then gives the user 10 seconds to properly demonstrate the letter. If the user succeeds, the program provides the user with positive feedback in the form of a green screen with a check mark. If the user cannot correctly demonstrate the letter within 10 seconds, the program provides the user with negative feedback in the form of a red screen with an X. The program then chooses a new random letter and displays it on the screen.
The following code runs every frame to read the information from the serial port, check if the letter is being signed correctly, handle the 10-second timer, display the green or red screen, and tell the program to start again.
The Interface
The user interface was designed to be very simple, bright, and easy to follow, which makes the program fun and usable for children. The ASL illustrations have colour-coded fingers that afford to the user that they are supposed to match their hand position to that which is shown on the screen. The colour-coding also helps the user to correctly make the hand position by visually connecting what finger goes where.
User Testing
Although most of the user testing was with college students in their 20s, the program was also tested on octogenarians, who may have different physiology (e.g. arthritis), cognitive skills, and reaction times. The tests were successful, but the 10-second time limit that the program imposes on users proved challenging for the older users, and can be modified if necessary.
Next Steps
Now that this prototype has been successfully created and works well, further iterations would focus on improving the user experience and exploring methods of gamification.
Roles
Interaction Design, Physical Computing, Usability Testing
Tools
Arduino, Fabrication, Processing
