1 / 33

Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk

Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk. By Kyle Rector. My experience with ASL. English to ASL Dictionaries. Can ASL be put in a database?. Written forms of ASL. ASL to English Dictionaries. Can I use ASL novices?.

fruma
Download Presentation

Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk By Kyle Rector

  2. My experience with ASL

  3. English to ASL Dictionaries

  4. Can ASL be put in a database?

  5. Written forms of ASL

  6. ASL to English Dictionaries

  7. Can I use ASL novices? Grosjean and Lane (1977) non signer can distinguish a pause for phrase boundaries or sentence ends Gibet, Courty, Duarte (2011) ASL signer observed animations of French signed language to see if they could replicate LSF

  8. Socially Computed Scripts to Support Social Problem Solving Skills

  9. Real-Time Captioning by Groups of Non-Experts

  10. AudioWiz: Nearly Real-time Audio Transcriptions

  11. Better Vocabularies for Assistive Communication Aids: Connecting Terms using Semantic Networks and Untrained Annotators

  12. How are those last projects similar?

  13. Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk By Kyle Rector

  14. What are ASL Attributes?

  15. Identifying American Sign Language Attributes Using ASL Novices on Mechanical Turk By Kyle Rector

  16. Research Questions • Is it possible to have ASL novices identify attributes of a sign? • How accurate are ASL novices when identifying parameters? • What attributes of a visual language are easier or harder to solve with lay people? • What types questions are needed to get this data?

  17. How can a novice do this? • 33 Handshapes • 10 Locations • 9 Orientations • 30 Movements • 6 Relative Positions

  18. Mechanical Turk • 50 words • 72 questions • 3 people • 2 cents per person • $216 total • Answered overnight!

  19. ASL Expert http://asl2eng.cs.washington.edu Same 50 words Finished over an average of 4.5 hours

  20. Evaluation - differences Handshape: #:54 %:3.27 Location: #:34 %:6.8 Orientation: #:213 %:47.33 Movement: #:143 %:20.43 Relation: #:44 %:14.67 Total: #:488 %:13.56

  21. Understanding the differences • ASL Expert: no Mturk: yes

  22. ASL Novice errors • Completely missed it • Crowd undecided • Close resemblance • Confused relative position and movement

  23. ASL Expert errors • Completely missed it • Close resemblance • ASL Experience

  24. Is this sustainable? • “I liked that they were different and I'm interested in sign language so I found them interesting. I did these awhile back and just came back looking for more to do and found this! :)” • “They were interesting and quick. I suppose if I was in contact with any one who used sign language I might pick up a few words.”

  25. Future Work • Use technology or ASL Experts to compensate • Orientation • Movement • Only use possible combinations of attributes • How do we determine what these are?

  26. Future Work • Put this data into ASL to English dictionary • Close resemblance in hand shapes or locations • search metric http://asl2eng.cs.washington.edu

  27. Conclusion • Getting data from novices, good or bad is useful for future applications • Orientation is very hard for novices to comprehend • ASL Experts entering a term in the dictionary seems easy, but can be costly

  28. Thank you! • Advisor: Richard Ladner • Research Assistant: Travis Smith • Grad students: Shiri Azenkot, Lydia Chilton, Supasorn Suwajanakorn

More Related