A User-Powered American Sign Language Dictionary - Bragg, Danielle; Rector, K.; Ladner, R. (Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, 2015)
Sounds provide informative signals about the world around us. In situations where non-auditory cues are inaccessible, it can be useful for deaf and hard-of-hearing people to be notiﬁed about sounds. Through a survey, we explored which sounds are of interest to deaf and hard-of-hearing people, and which means of notiﬁcation are appropriate. Motivated by these ﬁndings, we designed a mobile phone app that alerts deaf and hard-of-hearing people to sounds they care about. The app uses training examples of personally relevant sounds recorded by the user to learn a model of those sounds. It then screens the incoming audio stream from the phone’s microphone for those sounds. When it detects a sound, it alerts the user by vibrating and providing a pop-up notiﬁcation. To evaluate the interface design independent of sound detection errors, we ran a Wizard-of-Oz user study, and found that the app design successfully facilitated deaf and hard-of-hearing users recording training examples. We also explored the viability of a basic machine learning algorithm for sound detection.