Title:  Design and Evaluation of a Multimodal Assistive Technology using Tongue Commands, Head Movements, and Speech Recognition for People with Tetraplegia

Committee: 

Dr. Ghovanloo, Advisor

Dr. Anderson, Co-Advisor   

Dr. Inan, Chair

Dr. Jackson

Abstract:

The objective of the proposed research is to design a wireless and wearable multimodal assistive technology that will utilize multiple remaining abilities of the people with physical disabilities, such as tetraplegia, to do their daily life activities more efficiently and independently, thus improving their quality of life. The focus of this work has been designing a system, called multimodal Tongue Drive System (mTDS), that can capture speech, tongue commands, and head movements and convert them to text and commands to interact with devices such as computers, smartphones, and wheelchairs. High accuracy signal processing algorithms will be developed to process user inputs which are efficient to compute in a limited resource embedded hardware. The algorithms will also be optimized to reduce power consumption and increase battery lifetime. This work will also focus on designing experiments to evaluate the pros and cons of the multimodal interactions of the proposed assistive technology (AT) in effectively utilizing the remaining abilities of the user, and apply them to accessing computers, using smartphones, and driving wheelchairs intuitively and efficiently. The evaluation study will include both the healthy able-bodied and tetraplegic individuals. Able-bodied individuals will allow us to compare the mTDS performance against default and gold standard interfaces, i.e. keyboard-mouse combination, touchscreen, and joystick, for accessing computers, smartphones, and wheelchairs, respectively. Tetraplegic individuals, on the other hand, will allow us to evaluate the mTDS performance by the intended population and collect valuable feedback from the end users, caregivers, clinicians, and care providers. As the ultimate outcome of the proposed research we intend to test the hypothesis that by providing multiple simultaneous control abilities which are still accessible to the end users, one can significantly reduce the performance gap between healthy and tetraplegic individuals in controlling and accessing three key target devices that are necessary in today’s daily living tasks: computers (for work, education, entertainment, etc.), smartphones (for communication, environment control, seeking assistnace, etc.), and wheelchairs (for mobility).