Title: Combining Natural Language and Direct Manipulation for Human-Data Interaction through Visualizations

---------------------------------

Arjun Srinivasan

Computer Science Ph.D. Student

School of Interactive Computing

College of Computing

Georgia Institute of Technology

https://arjun010.github.io/

Date: Monday, April 29th, 2019

Time: 2:00pm to 4:00pm (EDT)

Location: GVU Cafe (TSRB 223)

Committee:

---------------------------------

Dr. John Stasko (Advisor, School of Interactive Computing, Georgia Institute of Technology)

Dr. Alex Endert (School of Interactive Computing, Georgia Institute of Technology)

Dr. Keith Edwards (School of Interactive Computing, Georgia Institute of Technology)

Dr. Bongshin Lee (Microsoft Research)

Dr. Vidya Setlur (Tableau Research)

Abstract:

---------------------------------

Visualization is a powerful tool for human-data interaction, enabling people to better understand their data, identify patterns, and discover insights. Correspondingly, there exist numerous systems that let people create and interact with visualizations to explore their data. Traditionally, interaction in these systems is facilitated through direct manipulation and control panels composing of common user interface widgets such as dropdown menus and sliders. However, recent advances in hardware and interpretation technologies present the opportunity to explore new visualization tools grounded in naturalistic human ways of interacting with other people and objects in the real world. Augmenting interaction techniques in current systems with natural language, this thesis explores the design of multimodal interfaces for data analysis with visualizations.

Specifically, I describe the design and implementation of a series of multimodal visualization systems that combine natural language and direct manipulation interaction. Furthermore, based on evaluations of these systems, I highlight the usage and utility of multimodal input for different visualization types (e.g., networks, unit visualizations) and devices (e.g., laptops, large touchscreens). By examining multimodal visualization systems in diverse contexts, this work demonstrates how multimodal interaction can promote the design of new visualization tools and support enhanced human-data interaction experiences during both individual and collaborative analysis.