PhD Defense by Richard Savery

Primary tabs

Title: Machine Learning Driven Emotional Musical Prosody for Human-Robot Interaction


Richard Savery

PhD Candidate in Music Technology

School of Music

Georgia Institute of Technology


Date: Thursday, October 28th

Time: 2pm

Location: Hybrid - In person at Couch 104, or Online at https://bluejeans.com/328992087/3044 



Gil Weinberg (Chair, Professor, School of Music, Georgia Institute of Technology)

Claire Arthur (Assistant Professor, School of Music, Georgia Institute of Technology)

Jason Freeman (Professor, School of Music, Georgia Institute of Technology)

Ayanna Howard (Dean of the College of Engineering at Ohio State University)



This dissertation presents a method for non-anthropomorphic human-robot interaction using a newly developed concept entitled Emotional Musical Prosody (EMP). EMP consists of short expressive musical phrases capable of conveying emotions, which can be embedded in robots to accompany mechanical gestures.  The main objective of EMP is to improve human engagement with, and trust in robots while avoiding the uncanny valley.  We contend that music - one of the most emotionally meaningful human experiences - can serve as an effective medium to support human-robot engagement and trust. EMP allows for the development of personable, emotion-driven agents, capable of giving subtle cues to collaborators while presenting a sense of autonomy. 


We present four research areas aimed at developing and understanding the potential role of EMP in human-robot interaction. The first research area focuses on collecting and labeling a new EMP dataset from vocalists, and using this dataset to generate prosodic emotional phrases through deep learning methods. Through extensive listening tests, the collected dataset and generated phrases were validated with a high level of accuracy by a large subject pool. The  second research effort focuses on understanding the effect of EMP in human-robot interaction with industrial and humanoid robots. Here, significant results were found for improved trust, perceived intelligence, and likeability of EMP enabled robotic arms, but not for humanoid robots. We also found significant results for improved trust in a social robot, as well as perceived intelligence, creativity and likeability in a robotic musician.  

The third and fourth research areas shift to broader use cases and potential methods to use EMP in HRI. The third research area explores the effect of robotic EMP on different personality types focusing on extraversion and neuroticism.  For robots, personality traits offer a unique way to implement custom responses, individualized to human collaborators. We discovered that humans prefer robots with emotional responses based on high extraversion and low neuroticism, with some correlation between the humans collaborator’s own personality traits. The fourth and final research question focused on scaling up EMP to support interaction between groups of robots and humans. Here, we found that improvements in trust and likeability carried across from single robots to groups of industrial arms.  Overall, the thesis suggests EMP is useful for improving trust and likeability for industrial, social and robot musicians but not in humanoid robots.  The thesis bears future implications for HRI designers, showing the extensive potential of careful audio design, and the wide range of outcomes audio can have on HRI.


  • Workflow Status: Published
  • Created By: Tatianna Richardson
  • Created: 10/04/2021
  • Modified By: Tatianna Richardson
  • Modified: 10/04/2021


Target Audience

No target audience selected.