Fashionable Wearable American Sign Language Recognition System

WearASL explored how computer vision and wearable computing could support two-way communication between people who use American Sign Language (ASL) and people who communicate through speech. Rather than treating the technology as separate equipment, the project incorporated its cameras, computer, controls, displays, speakers, and power supply into clothing and accessories.

Communication concept

The fashion-show exhibit presented Van and WearASL as coordinated wearable systems. A camera embedded in the hat observed the wearer's hand movements and sent the images to a computer carried in the backpack. The system was designed to recognize signed gestures and convert them into synthesized speech played through speakers in the backpack straps.

For communication in the other direction, a microphone captured spoken input and the wearable system presented the translated output through a Braille display. An LCD necklace provided visual feedback so that the wearer could check the system's interpretation.

Technical approach

The project poster describes a computer-vision pipeline for segmenting and tracking the hands, followed by gesture recognition using coupled or single hidden Markov models. The design distributed the hardware across the outfit: the camera was integrated into the hat, the computer was carried in the backpack, and the batteries were stored in the shoes.

The surviving project material documents the system concept and recognition pipeline, but reports neither recognition accuracy nor a user study. WearASL should therefore be read as a fashion-integrated research prototype, not as a validated general-purpose translator between ASL and spoken language.

Collaboration and presentation

The project brought together Nuria Oliver, Barbara Rosario, Joshua Weaver, and Alex Pentland at the MIT Media Lab with Maria Ella Carrera, Ricardo Prado, and Josefina Batres at Domus Academy in Milan. It was presented in October 1997 as part of the MIT Media Lab's Beauty and the Bits Smart Clothes Fashion Show.

The MIT Media Lab's archived exhibit page provides the original description of Van and WearASL.

WearASL system components and recognition pipeline.