American Sign Language Media Development Laboratory
Mission of the Laboratory
The ASL Media Develop Laboratory at Stephen F. Austin State University is dedicated to developing interactive media for enhancing American Sign Language instruction and for educating students with disabilities.
Research - The ASL Media Develop Laboratory generates media and software needed for researching and improving teacher preparation programs, American Sign Language instruction, and materials available to Deaf and hard of hearing students. Media formats include video, images, and 3D animations.
Service - The ASL Media Develop Laboratory has produced materials currently used to enhance American Sign Language courses. Additional materials will be released for use in Deaf Education classrooms with deaf and hard of hearing children.
Goals and Objectives
Short term goals:
· Continue media and software development for all ASL courses in the form of study enhancement software.
· Develop a useable and accessible set of instructor tools to use the software for assessment and data collection
· Develop promotional materials for recruiting Deaf and Hard of Hearing majors.
· Duplicate and distribute study resources to ASL and Deaf Education classes.
Long term goals:
· Develop and test software for use with Deaf and Hard of Hearing children in public school settings
· Market software and media developed in the laboratory
· Develop and test media and software for children with other disabilities
Software for acquiring American Sign Language skills
The ASL StudyTool supports classroom instruction by providing students with abundant vocabulary and phrase practice. ASL students and Deaf Education majors may practice both receptive skills and expressive skills, although expressive practice generally works better using a highly skilled language model to judge the accuracy of signs and phrases. The software rapidly focuses study on words or phrases that present the biggest challenge by eliminating videos as a student masters the content.
Figure 1: ASL StudyTool
The ASL StudyTool is distributed on a flash drive to all of our ASL students so that they may review on their own outside of class.
Figure 2: ASL StudyTool Flash Drive
Software for testing the usefulness of 3D animations of American Sign Language
The Science Signs StudyTool allows comparison of recorded performances from live actors with 3D animations generated through motion capture of Deaf performers. The 3D animations can be viewed from any angle, allowing students to clarify handshapes, positions and motions that may be occluded due to camera angles in a two-dimensional QuicTime video.
Figure 3: Science Signs StudyTool
Improving the Efficiency, Affordability and Quality of the Motion Capture process
QuickTime videos have the following advantages:
1. The equipment is much more affordable than traditional motion capture equipment
2. The process of recording, editing, and publishing requires a fraction of the time and effort needed to record, process and publish motion-captured animations of equivalent quality
3. Facial expressions and finger animations have been much more difficult to record using motion capture than it is to record with an off-the-shelf video camera.
Our First Efforts
The first attempt used in the ASL Media Development laboratory involved using mechanical motion capture which was prone to slippage and required constant calibration. Converting the data to useable animations required a great deal of effort by skilled 3D animators. The process also involved wearing equipment which often impedes the natural production of signs.
One major drawback from the perspective of a researcher is that the 3D animations and the QuickTime videos had to be recorded separately. A direct comparison of identical signs differing only in 3D vs. 2D was impossible.
Figure 4: Mechanical Motion Capture
Our Second Efforts
The second approach to motion capture data took advantage of recent software development by IPIsoft. IPIsoft use optical motion capture. The software used 4 Sony Playstation Eye cameras to record video from 4 angles. The video quality was somewhat grainy, but it was used only for IPIsoft applications to process the video converting the video from four angles into motion data. Converting a 30 second clip into motion data required approximately 4 hours using our available computer. One major advantage of this approach is that the Deaf performers do not have to wear any bulky equipment. This translates into the ability to directly compare identical signs in 2D QuickTime videos and 3D animations - they do not need to be recorded separately.
Figure 5: IPI Recorder
Our Current Efforts
The current approach is investigating the use of Microsoft Kinect with iClone 5 Pro software to get more complete motion capture data and reduce the processing time required by IPIsoft. The data will no longer need 4 hours to process 30 seconds of data - it is now recorded in real time using the Kinect motion capture capabilities.