Handsfree Decision Support: Toward a Non-invasive Human-Computer Interface

We describe our efforts to develop and test a prototype handsfree human–computer interface that allows healthcare personnel to access computer-based records and decision support without interfering with their daily clinical interactions and procedures. The methods are based on the integration of a computer-based decision-theoretic diagnostic system, speaker-independent speech-understanding, and a head-mounted, transmissive personal display that has the ability to embed translucent computer-based data, images, and inference results in a user’s field of view. The handsfree techniques highlight opportunities for user interfaces that can provide real-time and interactive decision support with minimal changes to existing patterns of clinical activity. We review the components of the system and describe the system’s performance and limitations. We describe methods for enhancing speech understanding for noisy environments by coupling the output of the Bayesian inference procedures with user models to provide methods that dynamically influence the language models used in speech recognition. Finally, we present several areas of ongoing and future research and development.