SIGCHI sponsors numerous conferences, publications, web sites, and other services that advance HCI through workshops and outreach. In a blog post for SIGCHI, Kumar said that she and the other incoming executive committee members aim to continue the long history of advancing the group’s key missions.

“We hope to continue to expand the excellent work that our many colleagues in this (executive committee) have done, with their commitment (among other things) to accessibility, equity and inclusion, to the safety of our community, global community building, and a #SIGCHI4ALL,” she wrote. “Together the six of us represent a wide range of perspectives; our hope is that this representation with ensure that we remain answerable to our entire global membership as we work towards supporting and fostering participation and growth locally and globally.”

Kumar’s research at Georgia Tech lies at the intersection of human-centered computing and global development. She has produced research that improves technology design for historically underserved communities. Her TanDEm Lab – short for Technology and Design towards ‘Empowerment’ – has focused on health and wellbeing on the margins, centering topics such as gender, stigma, and knowledge production.

Kumar has received other honors, such as the National Science Foundation’s CAREER Award, and also chairs the Association of Computing Machinery’s Future of Computing Academy.

Georgia Tech Ph.D. graduate Tamara Clegg is also on the SIGCHI executive committee, serving as the vice president of membership and communication.

Yang was recognized for her work leading the Social and Language Technologies Lab, concentrating on research across fields of natural language processing, machine learning, and computational social science. Yang’s research works to understand social aspects of language and build responsible NLP systems with social intelligence.

“We live in an era where many aspects of our daily activities are recorded as textual data,” Yang said in her proposal to Microsoft Research. “Over the last few decades, NLP has dramatically improved performance and produced industrial applications like personal assistants. Despite being sufficient to enable these applications, current NLP systems largely ignore the social part of language.”

This ignorance limits the functionality of the programs, Yang said. This research examines what is said, who says it, in what context and for what goals in hopes of developing systems to facilitate human-human and human-machine communication. So far, her team has produced projects on mitigating bias in text, detecting mental health issues, improving support in online support groups, and more.

According to Microsoft Research’s website, Yang is the first Georgia Tech faculty member to be named a Microsoft Research Faculty Fellow since 2011 and only the third overall. Yang has earned a number of other awards and recognitions, such as Forbes 30 Under 30 in Science and IEEE AI 10 to Watch.

The ALOE Institute will develop new AI theories and techniques for enhancing the quality of online education for lifelong learning and workforce development. According to some projections, about 100 million American workers will need to be reskilled or upskilled over the next decade. With the increase of AI and automation, said Co-Principal Investigator and Georgia Tech lead Professor Ashok Goel, many jobs will be redefined.

“There will be some loss of jobs, but mostly we will see individuals needing to learn a new skill to get a new job or to advance their career,” said Goel, a professor of computer science and human-centered computing in Georgia Tech’s School of Interactive Computing (IC) and the chief scientist with the Center for 21^st Century Universities (C21U). “So, how do you help 100 million workers reskill or upskill in 10 years? Because AI is in part responsible for this need, it is our belief it should also be responsible for finding a solution.”

That is the goal of this project, which will be led by principal investigator Myk Garn, assistant vice chancellor for New Models of Learning at the University System of Georgia and senior advisor to the Georgia Research Alliance (GRA).

“Online education for adults has enormous implications for tomorrow’s workforce,” Garn said. “Yet, serious questions remain about the quality of online learning and how best to teach adults online. Artificial intelligence offers a powerful technology for dramatically improving the quality of online learning and adult education.”

To do that successfully, the education must be personalized and scaled to unprecedented levels. Educating 100 million people in online environments will, of course, require far more time and energy than in-person educators can offer their students. That is where AI comes into play.

Researchers will build new AI techniques that can adequately and efficiently train other AI agents to interact with humans in a classroom setting, similar to the virtual teaching assistant Jill Watson that Goel has used in his online computer science classes for the past five years. This will help satisfy the scalability requirement.

“That’s the fundamental advancement in AI,” Goel said. “A human can train an AI agent in just a few hours how to teach other AI agents on how to interact with humans on various subjects.”

To satisfy the need for personalized AI, researchers will train machines to have a mutual theory of mind with their human counterparts. In other words, there will be a greater understanding by both machine and human of the others’ needs, knowledge, and expectations.

“Our vision is to develop AI agents that achieve a mutual understanding of learning expectations, outcomes, and methods between students and teachers,” said Alex Endert, an assistant professor in Georgia Tech’s College of Computing who will help the team analyze and understand data from the project. “Along with my students, I look forward to developing visual analytic interfaces that serve that purpose to foster trust and interpretability of AI for this domain.”

Ultimately, the hope is that education becomes more available, affordable, achievable, and, thereby, equitable. Such an expansive project, understandably, requires the expertise of many kinds from many people. In addition to Endert and Goel, who will be executive director of the ALOE Institute, there will be a host of faculty at Georgia Tech will participate.

Senior Georgia Tech members of the ALOE team include Stephen Harmon (Industrial Design and C21U), Michael Hoffmann (Public Policy), David Joyner (Online Master of Science in Computer Science), Ruth Kanfer (Psychology), Brian Magerko (Language, Media, and Culture), Keith McGreggor (IC and VentureLab), Chaohua Ou (Center for Teaching and Learning), and Spencer Rugaber (Computer Science).

Other partners in the ALOE Institute include Arizona State University, Drexel University, Georgia State University, Harvard University, the Technical College System of Georgia, the University of North Carolina at Greensboro, IMS Global, Boeing, IBM, and Wiley.

Georgia Tech is a key partner in two additional institutes in partnership with the U.S. Department of Agriculture, the National Institute of Food and Agricultures, the U.S. Department of Homeland Security Science & Technology Directorate, and the U.S. Department of Transportation Federal Highway Administration. Georgia Tech will lead the AI Institute for Advances in Optimization (AI4Opt) and the AI Institute for Collaborative Assistance and Responsiveness Interaction for Networked Groups (AI-CARING), the latter of which is led by College of Computing Associate Professor Sonia Chernova to support aging-related issues.

Released by the Georgia Institute of Technology and open-source web application Highcharts, Highcharts Sonification Studio (HSS) enables users to enter data into a spreadsheet to create traditional visual charts such as line graphs, scatterplots, and bar charts. At the same time, the tool creates non-speech audio tracks based on the data, a process known as sonification.

“The goal of this tool is to provide a simple, intuitive, and accessible way for users to import, edit, visualize, and sonify their data, and then export the results to a useful format,” said Professor Bruce Walker, director of Georgia Tech’s Sonification Lab. “We want users to be able to use the tool without having to download software or write code, and without prior sonification expertise.”

The data visualization+sonification approach lets users explore data with visual, auditory, or both modalities. This can lead to novel discoveries in its own right, and can also support users who may have limited ability to see or hear a given display. 

“Visually impaired readers find sonification and auditory graphs to be very useful for getting an overview of the data, as well as identifying patterns, outliers, and points of interest,” said Walker.

Brandon Biggs, a researcher and entrepreneur who is blind, highlighted the software’s ability to allow users such as himself to create a graph that he can trust will be visually appealing.

“I love how accessible all the components are with a screen-reader and how easy it is to create a sonification,” Biggs said.

And for all users—even those who can see—sound can communicate information without requiring visual attention. For instance, instead of looking at a weather forecast or a chart of a stock price on a screen, imagine being able to hear the ups and downs played like a melody, with additional sounds highlighting points of interest in the data.

HSS is the culmination of a multi-year collaboration between Highsoft—the makers of Highcharts—and the Georgia Tech Sonification Lab. The goal of the collaboration is to develop an extensible, accessible, online spreadsheet and multimodal graphing platform for the auditory display, assistive technology, and STEM education community.

Walker said that HSS is a systematic re-implementation of his lab’s Sonification Sandbox to integrate Highsoft’s industry-leading web-based Highcharts technology with Georgia Tech’s expertise in sonification and interactive auditory displays.

The tool is open-sourced under the MIT License to allow for extensions and forks in development from the community and to ensure the tool is available to all. A Highcharts license is required for commercial use of the tool, but otherwise, usage is completely free.

“This system will complement other tools and libraries actively used by the auditory display research community and help bring sonification to an even wider audience, especially in the visualization community and in situations of limited resources,” said Øystein Moseng, the Highcharts developer leading the implementation of the HSS.

A paper describing the research and development of the open-source tool is part of the 26^th annual International Conference on Auditory Displays (ICAD.org), which took place June 25-28, 2021. The paper Highcharts Sonification Studio: An Online, Open-Source, Extensible, And Accessible Data Sonification Tool is co-authored by Stanley Cantrell, Walker, and Moseng.

The Highcharts Sonification Studio web app, source code, and developer community are available at https://sonification.highcharts.com.

ICML is the leading international academic conference in machine learning. Along with NeurIPS and ICLR, it is one of the three primary conferences of high impact in machine learning and artificial intelligence research. It is supported by the International Machine Learning Society (IMLS).

Explore Georgia Tech people, research abstracts, and when authors will present (Tues-Thurs) in an interactive data graphic of Georgia Tech at IMCL 2021. Also explore the whole program in a second data graphic: Who’s Who at ICML 2021.

Georgia Tech’s work is represented in 2% of the program with 22 papers in a range of topics including (asterisk denotes a single paper):

• Applications (CV and NLP)*
• Applications (NLP)*
• Deep Learning Algorithms*
• Deep Learning Theory *
• Deep Reinforcement Learning*
• Learning Theory – 2 papers
• Optimal Transport – 2 papers
• Optimization (Convex)*
• Optimization and Algorithms – 2 papers
• Privacy *
• Reinforcement Learning – 2 papers
• Reinforcement Learning and Optimization*
• Reinforcement Learning and Planning*
• Reinforcement Learning Theory*
• Time Series – 4 papers

To help students begin thinking about ethics with regards to research, faculty at Georgia Tech – in conjunction with Mozilla – held the first workshop on integrating ethics and responsible computing into courses this summer.

The workshop was a collaboration between faculty researchers at Georgia Tech in both the Ethics, Technology, and Human Interaction Center (ETHICx) and Computing and Society, as well as Mozilla. The workshop received a strong response, which organizers say indicates a growing desire for ethics at the center of computer science courses.

Members of the College of Computing’s Division of Computing Instruction, the Schools of Interactive Computing, Computational Science and Engineering, Computer Science, and Electrical and Computer Engineering, along with attendees from Georgia State all participated in the online workshop.

“It’s really gratifying to have broad representation because it demonstrates the desire for people from so many different areas to think more deeply about the role of ethics in our education,” said Ellen Zegura, professor in the School of Computer Science and Fleming Chair in Telecommunications.

The goal of the workshop was to help instructors consider ways in which to implement ethics as a central piece in courses not just later in a student’s study, but from the very beginning. There’s an issue of urgency, Zegura said, that needed to be considered.

“Computing has reached a point where it is being used for critical decision making that really affects people’s lives,” she said. “The need to use computing responsibly has moved up incredibly. And if we don’t talk about ethics early in the curriculum, we’re sending a message that it’s not important. If you only hear about it in one course and it’s later in your career, then what does that say about the importance? Students see that.”

While official plans aren’t currently in place to continue the program, Zegura said the idea is to continue this as a series of activities that are responsive to what people’s needs are, specifically those who want to do a better job of embedding ethics into their computer science curriculum.

Georgia Tech graduate Kathy Pham (CS ’07, MS CS ’09), now at Mozilla, has been instrumental in engaging the computer science community from 15-20 universities on focusing on ethics, Zegura said.

Portions of the workshop can be viewed on YouTube here.

Yang’s research, which is being led by her Ph.D. student Jiaao Chen, aims to alleviate dependence of supervised models on labeled data via data augmentation approaches. Supervised learning is the machine learning task of learning a function that maps an input to an output based on example pairs, inferring the function from training data that has been tagged with identifying properties or characteristics (labeled data).

The hope is that they may improve upon the ability to transfer models from one setting to another despite the relative lack of intensive training examples.

“In the era of deep learning, natural language processing (NLP) has achieved extremely good performances in most data-intensive settings,” Yang said. “However, when there are only one or a few training examples, supervised deep learning models often fail. This strong dependence on labeled data largely prevents neural network models from being applied to new settings or real-world situations.”

Yang’s group has published a couple of papers in this field already, and she said the Salesforce grant will further support efforts to extend it to broader contexts, especially when NLP tasks involve complicated outputs.

“These examples might include performing named entity recognition that finds the important information in a text, or semantic parsing that converts a natural language sentence into a structured command,” she said.

You can read previous papers on the subject at the links below:

• MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification (Jiaao Chen, Zichao Yang, Diyi Yang)
• Local Additivity Based Data Augmentation for Semi-supervised NER (Jiaao Chen, Zhenghui Wang, Ran Tian, Zichao Yang, Diyi Yang)

Yang was chosen from a group of over 180 quality proposals from more than 30 countries.

The award recognizes participants in the Focus Program who have demonstrated academic excellence, community leadership, and been granted admittance to a graduate program. The Focus Program aims to introduce minority students to graduate school in hopes of increasing the number who pursue higher degrees.

Rogers attended the Focus Program five years ago as an undergraduate student at Georgia Tech.

“It helped me learn about grad school and set me up for success,” Rogers said of the program.

The award, which carries a prize of up to $2,500 per student based on funds available and number of awardees, is not based on specific research but recognizes overall accomplishments. In an application essay, Rogers shared how OMED was pivotal to is success at Georgia Tech.

As an undergraduate, he participated in the Challenge Program, a five-week academic residential program for incoming first-year students. Later, he became a counselor in the same program, an OMED tutor, a Focus participant, a Focus panelist, and last summer a computer science (CS) instructor in the Challenge program.

“It was really spooky because I was teaching the new Challenge students in the exact same room that I sat in when I was learning CS for the first time in Challenge a decade ago,” Rogers said. “Truly full circle. OMED has truly been a foundation for me here at Georgia Tech, and I am eternally grateful.”

Rogers’ research focuses on human-robot interaction, investigating the effects that intelligent agent verbal deception has on human interaction.

“Animals deceive. Humans deceive. Should robots and AI deceive?” Rogers poses in his research tagline.

Additionally, the work aims to provide AI systems the ability to autonomously produce contextually meaningful and successfully deceptive utterances while determining when it is appropriate to verbally deceive humans.

He is advised by IC Chair Ayanna Howard.

Twenty-three members of the Georgia Tech community contributed to 11 papers that will be presented at the conference, while two longtime contributors will join the ranks of the prestigious AAAI Fellows program.

School of Interactive Computing Chair Ayanna Howard and Professor Ashok Goel join College of Computing Dean Charles Isbell (elected in 2019) and Regents’ Professor Emerita Janet Kolodner (elected in 1992) are 2021 inductees to the fellowship, giving the Institute four members. The program recognizes individuals who have made significant, sustained contributions to the field of artificial intelligence.

[Related news: IC Professors Howard, Goel Named 2021 AAAI Fellows]

Notable research among the eight papers accepted to AAAI 2021 includes work from a multi-institution team working to understand and improve forecasting models of influenza-like illnesses like Covid-19. Effective forecasting is even more challenging amidst the current pandemic, when counts are affected by various factors such as symptomatic similarities.

The approach in this paper steers historical forecasting models to new scenarios where the flu and Covid-19 co-exist, demonstrating success in adaptation without sacrificing overall performance.

Georgia Tech’s Alexander Rodríguez and B. Aditya Prakash are co-authors on the paper, along with Nikhil Muralidhar, Anika Tabassum, and Naren Ramakrishnan of Virginia Tech, and Bijaya Adhikari of the University of Iowa.

[Related news: Research Team Wins Two Covid-19 Challenges in One Week]

Explore Georgia Tech’s presence in this visualization and view a list of papers below.

INTERACTIVE VISUALIZATION: Georgia Tech @ AAAI 20201

• DeepCOVID: An Operational Deep Learning-driven Framework for Explainable Real-time COVID-19 Forecasting (Alexander Rodríguez, Anika Tabassum, Jiaming Cui, Jiajia Xie, Javen Ho, Pulak Agarwal, Bijaya Adhikari, B. Aditya Prakash)
   
• Semantic MapNet: Building Alocentric SemanticMaps and Representations from Egocentric Views (Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, Dhruv Batra)
   
• Steering a Historical Disease Forecasting Model Under a Pandemic: Case of Flu and COVID-19 (Alexander Rodríguez, Nikhil Muralidhar, Bijaya Adhikari, Anika Tabassum, Naren Ramakrishnan, B. Aditya Prakash)
   
• Bias and Variance of Post-processing in Differential Privacy (Keyu Zhu, Pascal Van Hentenryck, Ferdinando Fioretto)
   
• Branch and Price for Bus Driver Scheduling with Complex Break Constraints (Lucas Kletzander, Nysret Musliu, Pascal Van Hentenryck)
   
• Detecting and Adapting to Novelty in Games (Xiangyu Peng, Jonathan Balloch, Mark Riedl)
   
• Differentially Private and Fair Deep Learning: A Lagrangian Dual Approach (Cuong Tran, Ferdinando Fioretto, Pascal Van Hentenryck)
   
• How to Motivate Your Dragon: Teaching Goal-Driven Agents to Speak and Act in Fantasy Worlds (Prithviraj Ammanabrolu, Jack Urbanek, Margaret Li, Arthur Szlam, Tim Rocktaschel, Jason Weston)
   
• Automated Storytelling via Causal, Commonsense Plot Ordering (Prithviraj Ammanabrolu, Wesley Cheung, William Broniec, Mark Riedl)
   
• Encoding Human Domain Knowledge to Warm Start Reinforcement Learning (Andrew Silva, Matthew Gombolay)
•  
• Weakly-Supervised Hierarchical Models for Predicting Persuasive Strategies in Good-faith Textual Requests (Jiaao Chen, Diyi Yang)

The AAAI Fellows program recognizes individuals who have made significant, sustained contributions – usually over at least a 10-year period – to the field of artificial intelligence (AI).

Goel’s research, which spans about 35 years, has connected fields of AI, cognitive science, and human cognition. Increasingly, it has merged the fields of AI and education, culminating in his lab’s groundbreaking work on Jill Watson, a virtual teaching assistant that can answer student questions in discussion forums for online classes. This trailblazing work has been recognized by numerous media outlets across the globe and has enormous long-term implications for the future of education.

“This is an exciting time for AI research into cognitive systems,” Goel said. “In one direction, my research uses the needs of human learning to ground and inspire novel AI techniques and tools. In the other, it uses AI theories and methods to provide new insights into human cognition and behavior.”

The team responsible for the advancement of Jill Watson and additional AI techniques for education, called emPrize, advanced to the semifinals of the international XPrize AI competition in 2020.

Howard, who was recently named the next Dean of Engineering at The Ohio State University, has performed similarly impactful research over her time in the field. As the director of the Human-Automation Systems Lab (HumAnS) at Georgia Tech, she has led research in conceptualizing humanized intelligence, the process of embedding human cognitive capability into the control path of autonomous systems.

Specifically, the lab studies how human-inspired techniques, such as soft computing methodologies, sensing, and learning can be used to enhance the autonomous capabilities of intelligent systems. This has impact in both virtual AI and robotics, and has led to enterprises like Zyrobotics, the company Howard co-founded that produces mobile therapy and educational products for children with differing needs.

Additionally, she has been a spokesperson for the importance of ethical research in the field.

“We’re at such a critical moment in the development of artificial intelligence,” Howard said. “There is incredible possibility, but equally daunting challenges. It’s an honor to be recognized for the work we are doing in this field, but it’s far from over. My hope is that I can inspire future researchers to pursue impactful and ethical advancements in the field.”

Eight others aside from Goel and Howard were also selected to the fellowship program for 2021 and will be recognized at the 2021 AAAI conference, being held virtually Feb. 2-9.

The award, bestowed by the Association for Computing Machinery, recognizes a woman whose early-career research has had a significant impact on the computing discipline, as measured by factors like society impact, frequent citation of work, or creation of a new research area.

De Choudhury will receive a framed certificate and a $1,000 stipend for the recognition, which is in its first year of existence and will be given out annually. She will be recognized for the award at a research conference to be named later.

“I feel deeply honored for this recognition and owe my successes to my wonderful students and collaborators, as well as the intellectual freedom provided by Georgia Tech’s College of Computing that has helped trailblaze interdisciplinary research in computing, like mine, for years,” she said.

De Choudhury’s work leverages large-scale online social data and advances in machine learning to help answer fundamental questions relating to our social lives. Chief among them lie within the field of mental health care – understanding mental health, improving access to care, and more. Her work has been recognized by a number of other awards, including 13 best paper and honorable mention paper awards from the ACM and AAAI, as well as publications such as the New York Times, BBC, NPR, and others.

In addition to the personal appreciation, De Choudhury stressed the importance of recognizing the work of under-represented researchers in the computing field.

“I’d like to commend the efforts of ACM-W for creating this new opportunity to celebrate the research of a group under-represented in the computing field,” she said. “There is a long way to go when it comes to computing making significant positive impact on a pervasive societal problem like mental health. Still, this award serves as a valuable encouragement for the next frontier of my research program.”

De Choudhury leads the Social Dynamics and Wellbeing Lab. Research from the lab, both past and current, can be explored in more detail on its website.

Being in the heart of Atlanta, however, the burgeoning heart of the film industry, comes with a few perks. Last year, Bryant was able to take advantage of one when she was contacted by representatives from the production crew of Superintelligence. The movie stars Melissa McCarthy as a woman who must prove to an artificial intelligence that humanity is worth saving and was recently released on HBO Max.

For the movie, Bryant was asked to program a Nao, a humanoid robot she uses in the Human-Automation Systems (HumAnS) Lab run by her advisor, School of Interactive Computing Chair Ayanna Howard. Read about Bryant’s experience programming the biggest star on the set.

How did this opportunity to work with Superintelligence come about, and what was the experience like?

The production team reached out to the College of Computing. They were interested in having a robot for a scene and needed someone who could program the Nao to match the scene they had written. They reached out to Dr. Howard because they knew she had that type of robot, and she reached out to me because I’m the person who does most of the customized programming for this particular robot. If there’s a script or movements or whatever, I’m the choreographer.

It was exciting. I was like, “Oh my goodness, this is for a movie.” I had no idea what it was about, but I was just excited to be a part of it. They asked if their ideas were possible and the production team was like, “We don’t know what it can do, but we think it looks cool. Can you make it do this?” We talked on the phone, and then I went to work.

How long did you have to program it?

I had about a week to get it ready. I had this idea of what they wanted, and I just tried to program it as best as I could.

So, tell me about the day of. What was it like being on set?

I took the robot to the Klaus Advanced Computing Building. They were filming in there. It was so exciting to see everything. I had to tell the robot to go on their cue, so I was sitting right behind the camera. I got to meet Melissa McCarthy and some of the other stars, and I got a few pictures with them that I’m excited to finally be able to share with everyone. Everyone was so welcoming and understanding that the robot needed some time. I like to say that the robot was the biggest superstar on the set. It had its moments where it was like, “I’m not ready yet. My joint isn’t quite ready to do this movement.” They were understanding and eager to learn. They wanted their own pictures with the robot and everything, and had their own questions that I was excited to answer.

A lot of non-roboticists or AI researchers’ first experiences with robots is in mass media like movies or TV shows, and normally its some dystopian or disaster scenario. How seriously did you take that responsibility or opportunity to portray the lighter, more realistic side?

I think for a lot of people, robots – especially these humanoid ones – have been largely portrayed negatively. They focus on disaster cases they may never happen in the next 100 years, if ever. There hasn’t been a lot of mass media attention that focuses on more positive use cases. I take that very seriously in our work, just knowing that we focus on people, on children that can benefit from the technology and have it improve their quality of life. It’s important to show those cases to affect the narrative. But we also want to highlight the concerns that are just. Things like bias and ethics of using robotics in certain domains. Those are real things that people are working to mitigate now, so we can bring people closer to what the field actually looks like by highlighting both.

Every time I teach kids or teach a class, I start out by showing what robots can actually do. I show videos of them falling over or something like that to illustrate that those terminators or killer robots, that doesn’t happen right now. But there are some other issues that are real and current and pressing, and here’s how we address them.

Being at Georgia Tech with movies filmed nearby has offered these kinds of neat opportunities. How neat is it to have this platform?

My friends think it’s so much cooler that I helped work on a movie that is going to be on HBO Max than for me to have some paper published at this really prestigious conference. The movie resonates with them more, so it’s an opportunity to have a connection. They can relate to the technology in a way that is natural to them and ask questions, and I can share more about robotics and my work. That’s how we get people interested in the field.

Large, heavy, and with the potential to suddenly fall over all mean that the risk to humans has remained too high to place this technology in homes, hospitals, retail spaces, or care facilities.

In 2016, however, researchers at UCLA posed a solution: What if we made robots that just couldn’t fall down? Now, researchers at Georgia Tech, in collaboration with UCLA and the University of Southern California, are working to develop a new class of locomotion systems that could enable this technology to become a larger part of our daily lives.

“We have lots of robots,” said Sehoon Ha, an assistant professor in Georgia Tech’s School of Interactive Computing and a co-principal investigator on the project. “But they aren’t in our house or in our stores. It’s mainly because of safety. I have a young daughter. I wouldn’t be comfortable with a full-sized humanoid robot in my house.”

Previously, UCLA developed a new class of robots called “buoyancy-assisted robots.” Instead of the human-like hardware that was bulky, heavy, and subject to the pitfalls of gravity, these legged robots remained erect thanks to a body made of helium balloons.

“Even though there is some mechanical or motor error, it never falls,” Ha said. “It never breaks. It’s super light. Even if it might collide with you, it doesn’t fall and it can’t hurt you.”

Creating a new class of locomotion systems has a couple of challenges: designing a new hardware that is cheap and safe and developing an algorithm that supports locomotion and collaboration. This grant will support development of novel frameworks that address a fundamentally new family of legged robots and empower them with reliable locomotion skills.

“The main philosophy is to deploy the reinforcement learning on real hardware,” Ha said. “This buoyancy-assisted robot is subject to a relatively larger magnitude of drag forces. It’s hard to simulate it. There’s a discrepancy between simulation and the real world. We want to collect real-world experience and limit the reality gap.”

The technology could help carry out a search and rescue in a disaster relief zone or answer a question in a retail space. The new project, funded by a $500,000 grant from the National Science Foundation’s National Robotics Initiative, will help create new locomotion control systems using reinforcement learning to improve the state of this technology.

Already cheaper than its bulkier counterparts, these robots could be as inexpensive as a couple hundred dollars produced at scale, Ha said.

“Now you might imagine a scenario where you could drop 1,000 of these into a disaster area to carry our search and rescue missions,” he said.

The grant runs for four years and research from the project will be open-source to encourage additional collaboration. The grant will also support a competition for middle and high school students using the low-cost platforms to foster student interest in the field.

Like carrying a can of pepper spray when walking down a dark alleyway – or avoiding the alleyway altogether – a new project out of Georgia Tech’s School of Interactive Computing (IC) aims to connect this abstract world of cybersecurity and privacy with concrete physical environments to promote better security behavior.

“In the real world, we have these corporeal sensations that give us cues on how to act,” said IC Assistant Professor Sauvik Das, the principal investigator on the project. “If you feel a cold breeze on your cheek, you may decide to wear a scarf. If you are walking down a dark alleyway, you may become more alert and aware of your surroundings. It’s a different story in the present state of cybersecurity and privacy.”

That current state is mostly limited to a warning when you’re leaving a secure network on your computer or a pop-up box that might caution against proceeding to a specific website. But what about the digital threats we face daily when proceeding throughout our daily routines, perusing the internet on our phones or walking through a crowded airport?

There are no corporeal sensory perception cues that indicate what is threatening or worthy of our attention. Similarly, we don’t have affordances that allow us to manipulate digital interfaces in ways that will better protect us against these threats that we find salient.

“That’s the idea here,” Das said. “We want to solve this abstraction problem by physically alerting people to threats and giving them means to defend against them.”

The project presents three solutions to the digital abstraction problem – Spidey Sense, Bit Whisperer, and Horcrux. Each aims to solves a specific branch of the problem: notifying you to threats, giving you more effective means to defend against threats, and providing means to better govern shared resources.

Spidey Sense

Spidey Sense uses a wristband that integrates with modern Apple watches that can squeeze the wrist in programmable patterns to notify the wearer of perceived digital threats.

The idea is that people might not feel the threat through visual communication design the same way they might when walking down a dark alleyway at night.

“How can we similarly communicate that threat?” Das poses. “This field of affective haptics was a good bridge.”

Bit Whisperer

So, what do you do when you know threats exist? In the real world, one might intuit that to block entry into a room they could place a heavy object in front of a door or that to communicate secure information they might need to whisper. This project aims to present similar options for digital information.

“It’s like whispering through the digital world,” Das said.

To transfer data from one smart device to another, one might use Bluetooth. But one can’t see the bits traveling through the air as they are communicated. Bit Whisperer uses physical objects, like a table, to communicate.

Using inaudible sound frequencies that can be generated through smartphones, data is transmitted through the physical surface from one device to other devices on the same surface. Anyone off the surface can’t receive the data without physically placing their device on it, making it much more challenging for would-be attackers.

Horcrux

Horcrux is a more abstract project at present. It aims to assist individuals governing shared digital resources. Current state of the art provides point-and-click resources, but those make it impossible to multitask and challenging to specify access controls.

This project, like the others, aims to provide physical tools that can be manipulated by hand to make it easier to specify access.

The idea now is a mat where play pieces like figurines can represent people or resources that people own.

“Think of a castle where you can move figurines through different accesses,” Das said. “These tangible interfaces allow for more interaction, more multitasking, and visible physical representations for what everyone has access to.”

These projects are being funded by a $500,000 grant from the National Science Foundation. IC Professor Gregory Abowd is a co-principal investigator on the grant, and Ph.D. student Youngwook Do is a key contributor.

It’s an age-old problem: We are inundated with informal forms of communication like phone calls, remote video conferences, text conversations on group messaging platforms like Slack or Microsoft Teams. Remembering key points of each discussion can at times be overwhelming, not to mention the stress caused by missing a meeting or seeing a couple hundred messages stack up while you were out for lunch.

This digital solution, developed by Georgia Tech researchers and being presented in a paper this week at the 2020 Conference on Empirical Methods in Natural Language Processing, can assuage those concerns by generating summaries of informal conversations. Using a subset of machine learning called natural language processing, the method identifies conversational structure using particular keywords.

“Think about informal conversational structure: It has an opening, problem statements, discussions, a conclusion,” said Diyi Yang, an assistant professor in the School of Interactive Computing and a co-author on the paper. “We want to mine those structures to teach the model what may be informative within the conversation for generating better summaries.”

Words like any variation of “hello” or “good,” for example, might indicate that it is a greeting. Other action words likely indicate some kind of intention, and dates or times a discussion and conclusion on plans. Knowing this, the model can represent the unstructured conversation better to craft an accurate summary.

These types of summaries are more important now than ever. More individuals all over the world are working or attending school remotely. More discussions are being handled over the phone or video conferencing, plans being made through applications like Microsoft Teams. Previous research on the subject has focused on formal content like books, papers, or news articles, but the existing body of work on informal language is relatively sparse.

“This is applicable now more than ever because of where we are,” Yang said. “There’s so much online and text conversation, and we have way too much information. We need help storing it in a shorter and more structured way. If you’re away from your laptop for 30 minutes, it’s important to be able to get a quick summary of what you missed.”

Challenges still exist. There are problems with referral in the conversation, or calling back to a previous discussion point later in a meeting. There are also typos or slang, repetition, interruption or changes in role, language changes that can interfere with the model’s ability to determine structure. These are items Yang and her collaborator are continuing to address moving forward.

“This is a great starting point,” Yang said.

The work is presented in the paper Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarization. The paper is co-authored by Yang and Jiaao Chen, a second-year Ph.D. student in the School of Interactive Computing.

The conference highlights research from universities, government, and industry around the world. It is comprised of three separate events: IEEE Visual Analytics Science and Technology (VAST), IEEE Information Visualization (InfoVis), and IEEE Scientific Visualization (SciVis). Like other conferences throughout the Covid-19 pandemic, VIS was held virtually.

Georgia Tech’s research was highlighted by one Best Paper Honorable Mention titled Mapping Researchers with PeopleMap. The paper – authored by Jon Saad-Falcon, Omar Shaikh, Zijie J. Wang, Austin P. Wright, Sasha Richardson, and Polo Chau – presents an open-source interactive tool that uses natural language processing to create visual maps for researchers based on their research interests and publications.

“Discovering research expertise at universities can be a difficult task,” the paper contends. “Directories routinely become outdated, and few help in visually summarizing researchers’ work or supporting the exploration of shared interests among researchers. This results in lost opportunities for both internal and external entities to discover new connections, nurture research collaboration, and explore the diversity of research.”

The paper also received a VAST Poster Research Award.

Also of note, new School of Computational Science & Engineering Chair Haesun Park received recognition for a 2010 IEEE VAST Paper. The paper received a Test of Time Award, recognizing it for continued contributions to the visual analytics and visualization community. The paper is titled iVisClassifier: An Interactive Visual Analytics System for Classification Based on Supervised Dimension Reduction and co-authored by Jaegul Choo, Hanseung Lee, and Jaeyeon Kihm.

School of Interactive Computing Ph.D. student Emily Wall, who is advised by Associate Professor Alex Endert, was also recognized with the VGTC Outstanding Dissertation Honorable Mention for her work Detecting and Mitigating Human Bias in Visual Analytics.

“People are susceptible to a multitude of biases, including perceptual biases and illusions; cognitive biases like confirmation bias or anchoring bias; and social biases like racial or gender bias that are borne of cultural experiences and stereotypes,” Wall contends. “As humans are an integral part of data analysis and decision making in many domains, their biases can be injected into and even amplified by models and algorithms.”

Her work aims to develop a better understanding of the role human bias plays in visual data analysis by defining bias, detecting bias, and mitigating bias.

Explore more about Georgia Tech’s contributions to IEEE VIS at the links below, or visit the Georgia Tech Visualization Lab. You can follow the lab on Twitter at @GT_Vis.

Georgia Tech at IEEE VIS 2020

Papers

• SafetyLens: Visual Data Analysis of Functional Safety of Vehicles (Arpit Narechania, Ahsan Qamar, and Alex Endert)
• NL4DV: A Toolkit for Generating Analytic Specifications for Data Visualization from Natural Language Queries (Arpit Narechania, Arjun Srinivasan, and John Stasko)
• Interweaving Multimodal Interaction with Flexible Unit Visualizations for Data Exploration (Arjun Srinivasan, Bongshin Lee, and John Stasko)
• What are Data Insights to Professional Visualization Users? (Po-Ming Law, Alex Endert, and John Stasko)
• Characterizing Automated Data Insights (Po-Ming Law, Alex Endert, and John Stasko)
• CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization (Zijie J. Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, Duen Horng (Polo) Chau)
• Bluff: Interactively Deciphering Adversarial Attacks on Deep Neural Networks (Nilaksh Das, Haekyu Park, Zijie J. Wang, Fred Hohman, Robert Firstman, Emily Rogers, Duen Horng (Polo) Chau)
• How Does Visualization Help People Learn Deep Learning? Evaluating GAN Lab with Observational Study and Log Analysis (Minsuk Kahng, Duen Horng (Polo) Chau)
• Mapping Researchers with PeopleMap (Jon Saad-Falcon, Omar Shaikh, Zijie J. Wang, Austin P. Wright, Sasha Richardson, Duen Horng (Polo) Chau)
• LEGION: Visually compare modeling techniques for regression (Subhajit Das, Alex Endert)
• CAVA: A Visual Analytics System for Exploratory Columnar Data Augmentation Using Knowledge Graphs (Dylan Cashman, Shenyu Xu, Subhajit Das, Florian Heimerl, Cong Liu, Shah Rukh Humayoun, Michael Gleicher, Alex Endert, Remco Chang)
• A Comparative Analysis of Industry Human-AI Interaction Guidelines (Austin P. Wright, Zijie J. Wang, Haekyu Park, Grace Guo, Fabian Sperrle, Mennatallah El-Assady, Alex Endert, Daniel Keim, Duen Horng (Polo) Chau)
• Toward A Bias-Aware Future for Mixed Initiative Visual Analytics (Adam Coscia, Duen Horng (Polo) Chau, Alex Endert)

Recognitions

• iVisClassifier: an Interactive Visual Analytics System for Classification Based on Supervised Dimension Reduction (Jaegul Choo, Hanseung Lee, Jaeyeon Kihm and Haesun Park)
• Detecting and Mitigating Human Bias in Visual Analytics (Emily Wall (Advisor: Alex Endert))

Workshops

• MoVIS '20 (Organizers: Clio Andris, Somayeh Dodge, Alan MacEachren)
• VISxAI '20 (Organizers: Adam Perer, Duen Horng (Polo) Chau, Fred Hohman, Hendrik Strobelt, Mennatallah El-Assady)

The new Center — pronounced “ethics” — will advance ethics-in-technology-centered research, education, and engagement at the Georgia Institute of Technology in collaboration with communities, government, non-governmental organizations, and industry. The office of the Executive Vice President for Research provided significant funds over a three-year period to seed the Center.

“We must foster Georgia Tech’s strengths in ethics, responsible research, and the development of emerging technologies in collaborative ways,” said Raheem Beyah, Georgia Tech’s vice president for interdisciplinary research. “ETHIC^x  will provide the necessary environment to support this work and Georgia Tech’s mission to advance technology and improve the human condition.”

The Colleges already have in-depth research and education experience addressing technology-related ethics questions. For instance, the School of Public Policy founded the Center for Ethics and Technology more than 12 years ago to foster a critical inquiry culture and deliberation about technology-related ethical issues. Faculty in that Center research ethical issues in the design of emerging contact tracing technologies, design ethics, social justice theory, and criticism broadly, and their relationship to emerging technologies such as smart cities, self-driving cars, and smart assistants, and a platform for fostering reflection and self-correcting reasoning in teaching and deliberation. The College of Computing also has created thriving research and educational initiatives such as the Ethical AI professional development course and the Law, Policy, and Ethics Initiative for Machine Learning @ GATECH.

The new Center will build on those strengths and position the Georgia Institute of Technology to become the leader in framing ethical concerns in technology, including fairness, accountability, transparency, social justice, and technological change.

Anticipating New Ethical Challenges

“ETHIC^x will be a place for robust, multidisciplinary research and a place to engage in systematic ethical analyses,” said Kaye Husbands Fealing, dean of the Ivan Allen College of Liberal Arts and co-director of the new Center. “It also will be a place for communities, corporations, governments, technologists, educators, and others to discuss and find solutions to complex ethical issues in science and technology.”

The Center will conduct research in ethics and emerging technologies, frame ethical questions, solutions in ethics and technology, and social justice and equity. Interdisciplinary and community-based research also will be emphasized.

Educational initiatives will include investigating and designing curricula for ethics training that can be woven throughout students’ educational journeys and for employees at affiliated companies.

”Responsibility is a core value of everything we do in the College of Computing at Georgia Tech. That means focusing on our communities and examining the impacts, both positive and negative, of our research and curricula,” said Charles Isbell, dean and John P. Imlay, Jr. chair of the College of Computing. “It means reaching across disciplines to collaborate with experts in other fields who can inform our own technological developments. We find solutions for tomorrow’s problems, which means we have to anticipate the new ethical challenges we will face. This Center will help us do that.”

New Center Builds on Deep Experience

Ayanna Howard, chair in the School of Interactive Computing, joins Husbands Fealing as co-director of the new Center.

“In the School of Interactive Computing, we encourage all of our faculty and student researchers to think critically about the new challenges their research presents and offer strategies to mitigate any potential negative impact on society,” Howard said. “Good innovation isn’t just about developing new technologies; it’s about developing solutions to problems that can make the world a better, more equitable, and more inclusive place.”

Georgia Tech launched the School of Interactive Computing in anticipation of the need for interdisciplinary research in computer science, liberal arts, and more. Faculty members examine diverse ethical challenges, including misinformation, content moderation, free speech on social platforms, data privacy and security, virtual reality, wearable computing devices, and robo-ethics.

Faculty and students throughout the Ivan Allen College of Liberal Arts engage in interdisciplinary research collaborations on ethics and emerging technologies, including in areas such as engineering, the environment, bioethics, responsible innovation, research ethics, the ethical and political dimensions of design and technology, and more.

“In the Ivan Allen College, careful consideration of the impacts of technology on people, and of people on technology, is a central part of our curriculum and values,” said Justin Biddle, an associate professor in the School of Public Policy, director of the Center for Ethics and Technology, and a member of the new Center’s leadership team. “With innovation today often outpacing our ability to understand its consequences, and widespread questions regarding the relations between technology, equity, and social justice, this kind of thinking is more important than ever.”

Faculty in both Colleges also have initiated discussions on the social and ethical implications of emerging technologies across campus and beyond. These include the TechDebates on Emerging Technologies, the Sparks Forum on Ethics and Engineering, the Machine Learning@GT Seminar Series, and the Ethics and Technological Futures series developed by Nassim Parvin and Susana Morris in the School of Literature, Media, and Communication. Ellen Zegura, a professor in the School of Computer Science, also leads a Mozilla grant aimed at embedding ethics in computer science classes through role play.

'Where the Best of Sciences and Humanities Meet'

Deven Desai, associate professor and area coordinator for Law and Ethics at Scheller College of Business, also will assume a key leadership role at ETHIC^x. He said the new Center will “build and deepen technology-related ethics scholarship and research across Georgia Tech.

“Scheller College’s focus on law and ethics is part of how we train future business leaders, the people who take innovation and bring it to market,” said Desai, who is also associate director for Law, Policy, and Ethics for Machine Learning at GA Tech (ML@GATECH), an interdisciplinary research center.

“ETHICx will be a place where the best of science and humanities meet to challenge and push to find the unasked, important questions. In that friction and fun, the best questions about the problems we face and the best answer about how to solve them so that everyone can benefit will come out,” he said.

Other members of the new Center’s key leadership team include Jason Borenstein, director of graduate research ethics programs in the School of Public Policy; Betsy DiSalvo, director of the human-centered computing Ph.D. program and associate professor in the School of Interactive Computing; Michael Hoffmann, a professor in the School of Public Policy; and Nassim Parvin, an associate professor in the School of Literature, Media, and Communication.

A launch event is planned for November, during Ethics Awareness Week, with a forum to identify key challenges in technology ethics. The Center will soon announce details.

For more information about ETHIC^x, contact Husbands Fealing at dean@gatech.edu or Howard at ah260@gatech.edu.

Factors like higher rates of underlying health problems, reduced access to health care, and structural inequities shape access to critical resources. These same populations, however, also often have less access to the types of online information designed to improve health outcomes.

This project, led by principal investigator Andrea Grimes Parker, an associate professor in the School of Interactive Computing and member of the Institute for People and Technology, will investigate how vulnerable and marginalized populations use technology for information seeking during the Covid-19 pandemic, as well as the impact of information exposure on their psychological wellbeing over time.

“The Covid-19 pandemic has brought further attention to systemic disparities in health that have long existed in the United States,” Parker said. “Within a public health crisis, the information that people are exposed to has huge implications for how attitudes around the pandemic are shaped, how people respond, and thus the course of the pandemic.

“Our work will provide both qualitative and quantitative evidence of the particular ways in which Covid-19 information exposure is tied to outcomes such as mental health in those most vulnerable to Covid-19 mortality and morbidity.”

Researchers will examine this information exposure over time. Their findings will help to shape recommendations for crisis information communication, particularly online, in the future. This work builds upon existing work by Parker and collaborators at Northeastern University.

Parker and colleagues Professors Miso Kim and Dr. Jacqueline Griffin began their collaboration investigation how well crisis apps – mobile apps designed to provide help during emergency situations – support older adults. This work was published at the 2020 ACM Conference on Human Factors in Computing Systems.

When the pandemic began, they expanded their focus to additional groups of vulnerable to poor health, such as low-income and racial and ethnic minority populations. The team, in collaboration with Professor Stacy Marsella, also expanded their focus beyond crisis apps, designing a survey to investigate information seeking practices in vulnerable populations amidst the pandemic.

This survey has been distributed to over 600 individuals in Massachusetts and Georgia to date. Parker’s new Google funding will enable the team to iterate on and expand the dissemination of this survey, conduct longitudinal analyses, and compliment the quantitative analysis with a qualitative component that will help unpack the nuances behind information-seeking practices and resulting Covid-19 attitudes, behaviors, and mental health outcomes.

This funding is part of Google.org’s $100 million commitment to Covid-19 relief efforts. Organizations receiving funds were selected through a competitive review. Funding focus areas include health equity, disease spread monitoring and forecasting, frontline health worker support, secondary public health effects, and privacy-preserving contact tracing efforts.

The investment follows a successful $1 million, nine-month pilot grant to the same team, which also includes Yale University, Cornell University, Vanderbilt University, and the Vanderbilt University Medical Center. Georgia Tech’s portion of the grant is $500,000.

Led by co-principal investigator Professor Jim Rehg of the School of Interactive Computing, Georgia Tech will develop methods for assessing nonverbal communication behaviors during face-to-face social interactions such as job interviews.

“Our innovative approach uses an unobtrusive wearable camera to record social behaviors, which are then analyzed using computer vision and deep learning models,” Rehg said. “Our automated analysis will allow job seekers to get feedback on their communication skills as part of our team’s integrated approach to job interview coaching.”

The project, which is part of the NSF’s Convergence Accelerator program, addresses an underutilized U.S. talent pool that poses a “critical but overlooked public health and economic challenge: how to include individuals with ASD” in the workforce, according to Vanderbilt Professor Nilanjan Sarkar who is leading the project team.

Consider:

• One in 54 people in the United States has ASD;
• Each year 70,000 young adults with ASD leave high school and face grim employment prospects; 
• More than 8 in 10 adults with ASD are either unemployed or underemployed, a significantly higher rate than adults with other developmental disabilities;
• The estimated lifetime cost of supporting an individual with ASD and limited employment prospects $3.2 million. 
• The total estimated cost of caring for Americans with ASD was $268 billion in 2015 and projected to grow to $461 billion in 2025.
• An estimated $50,000 per person per year could be contributed back into society when individuals with ASD are employed.

“We want to harness the power of AI, stakeholder engagement and convergent research to include neurodiverse individuals in the 21^st century workforce,” Sarkar said. “We feel that there is a big opportunity to turn great societal cost into great societal value.”

For this project, organizational, clinical and implementation experts are integrated with engineering teams to pave the way for real-world impact. The multi-university, multi-disciplinary team already has commitments from major employers to license some of the technology and tools developed.

Researchers will address three themes:

• Individualized assessment of unique abilities and appropriate job-matching
• Tailored understanding and ongoing support related to social communication and interaction challenge
• Tools to support job candidates, employees and employers.

Already, notable private-sector companies that employ people with ASD have committed to using at least one of the technologies developed under this program: Auticon, The Precisionists, Ernst & Young and SAP among them.

Two other companies, Floreo and Tipping Point Media, will make their existing VR modules available for adaptation to the program. Microsoft, which has a long-standing interest in hiring people with ASD, is involved as well and provided seed funding and access to cloud services for technology integration.

The five technologies can be used separately or as an integrated system, and the work has broader potential beyond ASD to expand employment access. In the U.S. alone, an estimated 50 million people have ASD, attention-deficit/ hyper-activity disorder, learning disability or other neuro-diverse conditions.

Sehoon Ha

Ph.D. in Computer Science, Georgia Tech 2015

Research interests: Robotics, Artificial Intelligence, Character Animation

Ha’s research lies at the intersection of computer graphics and robotics, including physics-based animation, deep reinforcement learning, and computational robot design. Specifically, he has published work that addresses the need for more intelligent control software in robotics to improve agility, robustness, efficiency, and safety. In the long term, he aims to develop robotic companions for the home, search-and-rescue robots for disaster recovery scenes, and custom medical surgery robots that are tailored to individual patients.

Jennifer Kim

Ph.D. in Computer Science, University of Illinois, Urbana-Champaign 2019

Research interests: Human-Computer Interaction, Interactive Systems, Health Care

Kim’s research investigates and develops interactive systems as communication artifacts to address various health-related challenges such as financial burdens of medical costs, difficulties in understanding behaviors of people with neurological disorders, and online health misinformation.

Chris Le Dantec

Ph.D. in Human-Centered Computing, Georgia Tech 2011

Research interests: Digital Media, Science and Technology Studies

Le Dantec is interested in developing community-based design practices that support new forms of collective action through production and use of civic data. Specifically, his research has direct impact on how policy makers and citizens work together to address issues of community engagement, social justice, urban transportation, and development.

Andrea Grimes Parker

Ph.D. in Human-Centered Computing, Georgia Tech 2011

Research interests: Human-Computer Interaction, Computer Supported Cooperative Work, Health Informatics

Grimes Parker designs and evaluates the impact of software tools that help people manage their health and wellness with a particular focus on equity. She studies racial, ethnic and economic health disparities, and the social context of health management. Through technology design, her research examines intrapersonal, social, cultural, and environmental factors that influence a person’s ability and desire to make healthy decisions.

Alan Ritter

Ph.D. in Computer Science and Engineering, University of Washington 2013

Research interests: Natural Language Processing, Information Extraction, Machine Learning

Ritter’s research aims to solve challenging technical problems that can help machines learn to read vast quantities of text with minimal supervision. Past work included a system that reads millions of tweets for mentions of new software vulnerabilities. This tool spotted critical security flaws in software. He is also interested in data-driven dialogue agents that can converse with people more naturally.

Sashank Varma

Ph.D. in Cognitive Studies, Vanderbilt University 2006

Research interests: Abstract Mathematical Thinking, Memory Systems Supporting Language Processing, Computational Models of High-Level Cognition

Varma’s research investigates complex forms of cognition that are uniquely human from multiple disciplinary perspectives. Primarily, this involves mathematical cognition, where he investigates how people use symbols systems to understand abstract mathematical concepts, how they develop intuitions about and insights into mathematics, and the mental mechanisms shared between reasoning and algorithmic thinking.

Wei Xu

Ph.D. in Computer Science, New York University 2014

Research Interests: Natural Language Processing, Machine Learning, Social Media

Xu’s recent work focuses on methods to understand the varied expressions in human language and to generate paraphrases for applications, such as reading and writing assistive technology. She has also worked on crowdsourcing, summarization, and information extraction for user-generated data, such as Twitter and StackOverflow.

School of Interactive Computing Ph.D. student Ceara Byrne, whose primary research focuses on instrumenting dog toys with various sensors to measure canine behavior, found a way to contribute to the cause when she was approached by a fellow Georgia Tech student for assistance in 3D printing.

Lee Whitcher, a Ph.D. student in the Daniel Guggenheim School of Aerospace Engineering, had already joined colleagues from the Georgia Tech Research Institute and George W. Woodruff School of Mechanical Engineering to design and manufacture personal protective equipment (PPE) like face shields to supplement the available supplies in the Atlanta area.

The work from GTRI and ME assisted in hospitals, and Whitcher’s work – a non-profit called Atlanta Beats COVID – aimed to design and produce masks and ventilators that could be produced by non-engineers wherever they are needed.

To do that, Whitcher and his partners needed a 3D printer that could cast the negatives for the masks. Georgia Tech’s GVU Prototyping Lab in the Technology Square Research Building had just what they needed. So did Byrne.

Byrne has been using the Prototyping Lab’s printer for a while now to develop negatives of the silicone dog toys she uses in her research. Byrne’s work involves studying behavior in canines to understand temperament for service animals.

“I was inspired by a friend from high school who grew up on a ranch,” Byrne said. “She and I got involved in 4-H. When I came back for a master’s degree, I started working with Thad Starner and Melody Jackson on the FIDO project. I started noticing these aspects of the data that were reflective of dog temperament like drive and how they tackle activities. It really interested me.”

Part of the research was to find good ways to measure that temperament beyond just visual observation. One solution was to place sensors into toys to take measurements as the dog played with it.

“I’ve used the Prototyping Lab to 3D print my negative molds so that I can silicone cast the positives like balls and tug toys,” Byrne said. “It’s a long process of finding the right silicones, materials, hardness.  For the toys, I went through three or four different molds to find the right way to actually cast the parts. It was a lot of experimenting.”

That experimentation made her uniquely prepared to chip in with Whitcher’s project when Covid-19 hit. Looking for a way to develop the right mold for easy do-it-yourself mask production, Whitcher turned to Byrne for assistance.

“There are a number of aspects to it,” Byrne said. “How do you de-gas some of the silicone? When you have a mask, you can’t have the bubbles in the mold because you need a seal. How do you do it with the vacuum? If there’s no vacuum available, what are some easier ways? How do we make these negatives properly, and how many can you cast at once? What are the environmental aspects when you do it from home?”

These are all questions Byrne has had to explore when it comes to her dog toys. The experience proved useful in the mask production, as well.

Byrne was happy to get involved in pandemic relief assistance. She has brothers and sisters-in-law who are doctors.

“They’ve been amazing in helping around the community,” she said. “My brother is making masks, which I think is fascinating. He’s a radiation oncologist and has built respiratory masks with the Pancreatic Cancer Foundation. So, I wanted to help out in any way that I could, as well.”

Being at Georgia Tech, she said, made the collaboration a natural occurrence.

“That’s what makes Georgia Tech unique, right?” she said. “We can collaborate across these disciplines that maybe don’t connect to each other on the surface.”

Read more about the relief effort, how to request PPE, and how to get involved at AtlantaBeatsCOVID.com.

Seim’s thesis, titled Wearable Vibrotactile Stimulation: How Passive Stimulation Can Train and Rehabilitate, presents a technique in which a vibrating wearable device is used to retrain motor function following debilitating occurrences of spinal fracture or stroke. Now a postdoc at Stanford University and a fellow with the National Institutes of Health, Seim is currently working with stroke survivors to develop accessible and functional wearable devices to reduce physical disability in both the upper and lower limbs.

“Lately, I have also developed new mechanical tools to assess hand and arm function when there are no quantitative clinical tests available,” Seim said. “I plan to continue research on wearable and ubiquitous systems for health, accessibility, and training.”

In Agrawal’s thesis, titled Visual Question Answering and Beyond, she explores a multi-modal artificial intelligence task called visual question answering. In this task, given an image and natural language question about it, a machine is programmed to automatically produce an accurate natural language answer. The applications of VQA include aiding visually impaired users in understanding their surroundings, aiding analysts in examining large quantities of surveillance data, teaching children through interactive demos, interacting with personal AI assistants, and making visual social media content more accessible.

Now at DeepMind and soon to be an assistant professor at the University of Montreal and Mila, an AI research institute, Agrawal intends to equip current VQA systems with better skills to move toward artificial general intelligence.

“In the long term, I am excited about science fiction becoming reality, when we all have smart virtual assistants that can see and talk and serve as an aid to visually impaired users,” she said.

The eight other recipients of the Georgia Tech Sigma Xi Best Ph.D. Thesis Award were Mingue Kim (ECE), Ming Zhao (Chemistry), Andres Caballero (BME), Ke (Chris) Liu (CEE), Monica McNerney (ChBE), Chris Sugino (ME), Hamid Reza Seyf (ME), and Eric Tervo (ME).

The NIH is issuing $5.9 million in funding for a new national biomedical technology resource center (BTRC), called the mHealth Center for Discovery, Optimization & Translation of Temporally-Precise Interventions (mDOT). Georgia Tech, one of seven collaborators on the project, will receive $500,000, and mDOT will be headquartered at the MD2K Center of Excellence at The University of Memphis.

One of the biggest drivers of the nation’s rising healthcare spending is providing care for patients with chronic diseases, many of which are linked to daily behaviors such as dietary choices, sedentary behavior, stress, and addiction. The mDOT Center will be a new national technology resource for improving people’s health and wellness. It will conduct cutting-edge AI research to produce easily deployable wearables, apps for wearables and smartphones, and a companion cloud system. mDOT’s innovative technology will enable patients to initiate and sustain the healthy lifestyle choices necessary to prevent and/or successfully manage the growing burden of multiple chronic conditions.

Led by Jim Rehg, a Professor in Georgia Tech’s School of Interactive Computing, Georgia Tech’s project will focus on analyzing streams of biomarker data to enable the development of more effective, personalized treatment approaches for chronic health conditions like smoking and physical activity. To achieve this, the team will develop machine learning methods that can discover important risk factors from sensor data and identify effective intervention targets.

“Consider developing an intervention to help people who are trying to quit smoking by providing personalized strategies for managing risk factors that are known to precipitate relapse,” Rehg said. “Researchers and practitioners would use our tools to analyze biomarker data and characterize the patterns that lead to relapse and identify potential intervention targets.”

The collaboration can then use the tools provided by the other teams to develop and tailor an effective personalized stress intervention and deliver it efficiently on a mobile device. Omer Inan, a faculty member in Georgia Tech’s School of Electrical and Computer Engineering, will also collaborate with the team, leveraging work on novel non-invasive biosensors that detect cardiovascular changes in heart failure. Working alongside the mDOT team will enhance the ability to develop and deploy interventions based on his novel wearable sensors.

“Researchers and industry innovators can leverage mDOT’s technological resources to create the next generation of mHealth technology that is highly personalized to each user, transforming people’s health and wellness,” said Santosh Kumar, the lead investigator of mDOT, who is the director of MD2K Center of Excellence and Lillian & Morrie Moss Chair of Excellence Professor of Computer Science at the University of Memphis.

To ensure mDOT’s innovative technology can be used by scientists to solve real-world problems, mDOT will be working closely with over a dozen other federally-funded projects to engage in joint technology development, testing, and large-scale real-life deployment. To ensure that mDOT’s technological resources can fuel innovation in the health technology industry, the mDOT Center is establishing a new industry consortium to provide access to mDOT’s latest research and seek feedback to inform its ongoing research.

The mDOT Center will be administered by the National Institute of Biomedical Imaging and Bioengineering (NIBIB).

“The mDOT Center will be the first BTRC focused on developing innovative mHealth technologies. It is positioned to empower scientists to discover, personalize, and deliver temporally-precise mHealth interventions and treatments, ensuring that health and wellness tools are delivered at the right moment, via the right personal device, and is optimized to have the most influence,” said mDOT’s program officer Tiffani Lash, director of the NIBIB program inConnected Health.

The multidisciplinary mDOT team consists of leading researchers in artificial intelligence (AI), mobile computing, wearable sensors, privacy, and precision medicine from Harvard University, Georgia Institute of Technology, The Ohio State University, The University of Massachusetts-Amherst, The University of Memphis (lead), The University of California at Los Angeles, and The University of California at San Francisco.

About MD2K: The Center of Excellence for Mobile Sensor Data-to-Knowledge (MD2K), headquartered (in FedEx Institute of Technology) at The University of Memphis, was established in 2014 by a grant from National Institutes of Health (NIH) under its Big-Data-To-Knowledge (BD2K) initiative. It has developed mobile sensor big data technologies to improve health and wellness. MD2K’s open-source software platforms for smartphones and the cloud are used across the nation to conduct scientific studies.

In 2010, a group of researchers at Georgia Tech, the CDC, and Ministries of Health in several African countries launched an open-source laboratory management system as part of the College of Computing’s Computing-for-Good (C4G) initiative. Designed to be ultra-configurable to meet variable needs of labs across developing countries with minimal training for staff, it quickly grew to become one of C4G’s biggest success stories.

More than 10 nations in sub-Saharan Africa adopted the program, called the Basic Laboratory Information System (BLIS), giving areas with little or poor internet connectivity an easy-to-use system for many who had minimal computing experience. These countries, which had over 1 million patients at the time, were using paper-based systems to manage information on disease spread, local illnesses, and much more. As information and communications technologies have expanded in the area, however, many labs gained a standardized reports system that could track prevalence rates of infections, slowing their spread.

But a lot can change in just 10 years. What was once designed for personal computing interfaces is now desired for a wide range of new platforms. Although laptops are still the device of choice for the majority of nurses – 79.6 percent reported in a study of a Nigerian hospital -- smartphones and tablets have seen a steady increase. The coming years will include many more innovations that render even those obsolete.

As users in the global south aspire to embrace mobile computing in clinical settings, a flexible interface, adaptable to everchanging applications, is needed.

Enter: Jung Wook Park and Aditi Shah, a Ph.D. student in Georgia Tech’s School of Interactive Computing (IC) and former master’s student in the School of Computer Science (SCS), respectively. Along with SCS Professor Santosh Vempala and IC Principal Research Scientist Rosa Arriaga, Park and Shah published research updating the current interface of C4G BLIS

Their updates focused on a handful of key areas, primarily mobile support. A responsive user interface framework supporting various screen sizes and resolutions was developed and evaluated by real users at hospitals in Africa currently using BLIS.

They compared user experience with the current interface on desktops and smartphones with a proposed interface on both and found that there was a significant improvement on both the desktop and smartphone.

“When you bring in a new system, they may feel uncomfortable with it,” Park said. “If we didn’t do a great job, you might get the same score or lower at the beginning. Over time, we saw improvements of 32 and 34 percent on desktops and smartphones.”

Shah, now at Microsoft, offered plenty of help in the development of the system, and her experience with a visual impairment allowed her to provide perspective on accessibility, as well.

The implications of this research extend far beyond ease of use for nurses, however. Park identified a growing problem across the globe in health care: communication. As the current pandemic can illustrate, viruses and diseases can spread quickly across many different populations. It isn’t sufficient to have just local data to mount an appropriate response; teams around the world must be able to rapidly share information.

A system like C4G BLIS, with its improved user interface that can be used across multiple platforms depending on the local needs of various communities, can help that communication.

“If you notice something locally and maybe other areas of the country or continent notice something, how do you know if it is a pandemic?” Park posed. “You need to be able to share that information to manage the spread. By turning these local systems into a standardized cloud-based system, we can improve communication.”

Already, Vempala said, he has heard reports from many labs that have adapted the flexible system to keep track of COVID-19 data in their communities.

The paper is titled Redesigning a Basic Laboratory Information System for the Global South, and was presented at the International Telecommunication Union Kaleidoscope conference, earning a Best Paper award.

Whereas humans might touch a hot pan on a stove once and never forget the lesson, it’s more complex to train robots to apply such universal knowledge across different situations.

The new technique, called CAGE, or Context-Aware Grasping Engine, takes into consideration a range of factors – such as the task the object will be used for, whether the object is full or empty, what it’s made of, and its shape – so that a robot can learn the right way to grasp various objects in a given context. For example, it allows a robot to learn not to hold a hot cup of tea by its opening, or to handle a cooking pot differently based on whether it just left a stovetop or a cabinet.

“In order for robots to effectively perform object manipulation, a broad sense of contexts, including object and task constraints, needs to be accounted for,” said Weiyu Liu, lead researcher on CAGE and Ph.D. student in robotics.

Using CAGE, a robot is able to apply what it has learned to objects it’s never seen.  For example, if trained to grasp a spatula by the handle to make a scooping motion, the robot is able to generalize this knowledge and know to grasp a mug by the handle and use it to scoop — if that was the programmed task — even if the robot has never encountered a mug before.

The research team, from the Robot Autonomy and Interactive Learning (RAIL) lab at Georgia Tech, validated their approach against three existing methods for teaching robots to handle objects. The team used a novel dataset consisting of 14,000 grasps for 44 objects, 7 tasks, and 6 different object states (e.g. objects contained solids, liquids, or were empty).

CAGE outperformed the other methods in a simulation by statistically significant margins, according to the researchers, highlighting the model’s ability to collectively reason about contextual information.

CAGE had an 86 percent success rate when averaged across tests looking at how well it identified context-aware grasps and if the model could generalize to new objects a robot had not seen previously. Among the existing methods, the highest success rate averaged 69 percent.

Liu said that the team’s model can rank grasp “candidates” for various contexts, ensuring that more suitable candidates are ranked higher than less suitable ones given a context. So a robot might, for example, learn to hand a sharp metal knife to a person handle-first, but hand over a plastic knife in any orientation due to its relative safety.

A final experiment evaluated the effectiveness of CAGE using a Fetch robot equipped with a camera, moving arm, and a parallel-jaw gripper. It performed almost perfectly in making a judgement on how to grasp objects for several distinct tasks, including scooping, pouring, lifting, and handing over an object, among others. If there was no suitable grasp for the given situation, the robot made no attempt in all cases.

The work, developed by Liu, Angel Daruna, and Sonia Chernova, was accepted into the  International Conference on Robotics and Automation, taking place virtually this June. The paper is titled CAGE: Context-Aware Grasping Engine and the research data is publicly available at https://github.com/wliu88/rail_semantic_grasping.

This work is supported in part by NSF IIS 1564080, NSF GRFP DGE-1650044, and ONR N000141612835. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.

On June 24, ML@GT faculty members Srijan Kumar (School of Computational Science and Engineering,) Aditya Prakash (School of Computational Science and Engineering,) Munmun De Choudhury (School of Interactive Computing,) Nicoleta Serban (H. Milton Stewart School of Industrial and Systems Engineering,) and OMSCS student Kenneth Miller will participate in a virtual panel discussing their work. The panel will be moderated by ML@GT executive director Irfan Essa.

Panelists will give individual presentations before participating in a general question-and-answer segment with audience members.

• Kumar and De Choudhury will share details of their work regarding the psychological impact of Covid-19. Kumar will also discuss his work examining hate and counter-hate messages on Twitter against Asian Americans during the pandemic.
• Prakash is a member of the Center for Disease Control and Prevention’s (CDC) forecasting team, and will share their new data-driven approach to disease forecasting.
• Serban’s presentation will focus on her work creating an agent-based simulation forecasting model. This model captures the progression of the disease in an individual and in households, schools, communities, and workplaces.
• A lawyer by day and OMSCS student by night, Miller participated in a Kaggle challenge using natural language processing and machine learning to help doctors and scientists read the most important studies related to Covid-19.

The panel will take place virtually via a Bluejeans Event at 11 a.m. on June 24 and is open to the public. Registration is required.

Over 220 academics at the 14^th annual event are convening and discussing work that is especially relevant during a time of an ongoing global health crisis and social unrest that has taken root across the United States.

Research in the conference proceedings include many topics directly addressing social ills and injustices that are magnified online as well as potential ways to better understand and mitigate them.

Several College of Computing faculty, current and former students, and postdoctoral researchers are part of the organizing committee. Munmun De Choudhury (Interactive Computing) is serving as the general chair of the conference this year. Former Human-Centered Computing PhD student Stevie Chancellor is workshop chair, former Computer Science PhD student Tanushree Mitra is tutorials chair, current CS PhD student Koustuv Saha is web chair, and current postdoc Talayeh Aledavood is local/social chair. CoC faculty Diyi Yang (Interactive Computing) and Srijan Kumar (Computational Science and Engineering) are data challenge chairs.

One of the two keynotes at the conference is by IC faculty Amy Bruckman.

Georgia Tech has three papers in this year’s program: 

• A study in causal inference by CS PhD student Koustuv Saha that tests what leads to favorable psychosocial outcomes in mental health forums.
  Link: https://aaai.org/ojs/index.php/ICWSM/article/view/7326
• A paper by HCC PhD student Ian Stewart, with advisors Diyi Yang and Jacob Eisenstein, that intends to gather a sharper view of “collective attention” on social media. Looking at descriptive details for a crisis event, researchers find that the information needed to describe that event changes as time goes on.
  Link: https://aaai.org/ojs/index.php/ICWSM/article/view/7331
• A socially-inspired approach to detect cyberbullying online, by incoming PhD student Caleb Ziems. The paper proposes new criteria for cyberbullying (e.g. harmful intent) and finds that both text and social features help prediction. This paper has been recognized with an Honorable Mention Award, given to a total of eight papers this year.
  Link: https://aaai.org/ojs/index.php/ICWSM/article/view/7345
   

For details about more research and to read the organizing committee’s full statement on the commitment to Black Lives Matter, fighting structural racism, and promoting inclusion and equity, go to https://www.icwsm.org/2020/index.html. In the wake of current events in the United States, the conference made 20 registration fee waivers available for Black scholars and individuals from other marginalized groups throughout the world, and provided scheduling flexibility to speakers and attendees participating in the Shutdown STEM walkout on June 10.

The conference is sponsored by the Association for the Advancement of Artificial Intelligence.

Georgia Tech is a leading contributor with 42 papers that include research covering more than 60 subfields within robotics. Deep learning in robotics and automation is one of the top areas of research among authors from the College of Computing and College of Engineering, the two largest contributors to Georgia Tech’s research at ICRA.

Deep learning is computing’s strongest area while mechanism design is where engineering excels based on the number of authors in the areas.

Top 3 subfields for computing authors:

• Deep Learning
• Semantic Scene Understanding
• Physically Assistive Devices and Network Devices (tie)

Top 3 subfields for engineering authors:

• Mechanism Design
• Learning and Adaptive Systems
• Multi-Robot Systems

The diversity of Georgia Tech robotics research at ICRA is also represented in the number of academic units and disciplines involved in advancing the field. Multidisciplinary teams often come together to tackle challenges that require a unique combination of skill sets.

There are nearly 100 Georgia Tech authors with work at ICRA. Explore the people, research, and trends from our community at https://icra.cc.gatech.edu/

The award recognizes the most influential ICRA paper published between 1998-2002 and selected Monte Carlo Localization for Mobile Robots as this year’s recipient. Dellaert conducted this work during his Ph.D studies at Carnegie Mellon University with Dieter Fox, Wolfram Burgard, and Sebastian Thrun.

“It is a great honor to be recognized, but receiving a ’20 years on’ milestone award also makes you feel old!” said Dellaert.

The paper was accepted to ICRA in 1999 and introduced the Monte Carlo Localization (MLC) method or particle filter localization, which represents the probability density involved in maintaining a set of samples that are randomly drawn from it. This method is faster, more accurate, and less memory-intensive than earlier grid-based methods and allows a robot to be localized without knowledge of its starting location.

MCL is simple to apply to the robotics domain, leading to its popularity. It is now taught in every robotics 101 class around the world. Many mobile robots, including commercial efforts, rely on MCL for localizing.

“Simplicity is key for acceptance and you cannot predict which of your research will have the most impact. This paper was a result of me procrastinating on my Ph.D. thesis which is a paper almost nobody read. It is an enormous honor that MCL has made a lasting impact on our field,” said Dellaert.

Like the virus itself, however, stress and anxiety stemming from overconsumption of news or other media can spread through social networks. As the mental health fallout becomes clearer, are some similar social media distancing recommendations needed to stem the flow through the online world?

A multidisciplinary team of researchers at Georgia Tech, Washington University-St. Louis, and the University of Wisconsin-Madison argue that these mental health implications of the pandemic are equally important, and a new grant from the National Science Foundation (NSF) has recently funded new research to that effect.

“It’s not just the fear and anxiety that I might get infected or I might infect or know someone who is infected,” said Munmun De Choudhury, an associate professor in Georgia Tech’s School of Interactive Computing and the co-principal investigator on the project. “It’s all of these things around it that are furthering the psychological impact. It’s very different from other kinds of illnesses or pandemics because of the uncertainty of the crisis. We simply don’t know how long we are into it.”

The grant is funded by the NSF’s Rapid Response Project program, which is intended for research that addresses an immediate need within society. It has provided $200,000 toward the yearlong project.

The research will combine investigations in two separate environments: the online world, where news, personal posts, videos, and other media are shared rampantly across social networks, and the offline real world, where the epidemiological data about the spread of the virus or economic data about the financial fallout can be measured.

For the former, they will use social media data from various popular social platforms – Twitter, Reddit, and YouTube – to measure the spread of information and how consumers of it express themselves in terms of anxiety or fear, or what they are saying about their own psychological wellbeing.

“How often are people expressing anger or fear or blaming someone through their posts?” said Srijan Kumar, an assistant professor in Georgia Tech’s School of Computational Science and Engineering and the other co-principal investigator. “We’ll develop new classifiers using natural language processing that will help us classify social posts into two categories: either anxiety-inducing or anxiety itself.”

This is new territory, according to De Choudhury. Although there have been other pandemics such as the 1918 influenza epidemic, none of this magnitude have taken place during the digital/social age. And while social media provides an important mechanism for staying informed and remaining in contact with friends and loved ones during the difficult social distancing measures, overexposure could result in negative mental health consequences.

“There is probably a sweet spot,” De Choudhury said. “Just like we need physical distancing in the real world, we probably need to practice distancing from social media or online information to an extent to avoid consuming too much anxiety-inducing media, while also staying informed.

“If I say something, it doesn’t just affect me. It affects all the people who read my posts. If they share it or if they post something, then it affects all of their social neighbors. It can be an outward ripple that affects people. We want to measure that, how they spread through social networks.”

They’ll compare that data with the other element: the offline world. Currently, people in New York City are likely more stressed and anxious in a different way than people in Georgia. New York has been the epicenter of the viral outbreak in the United States, meaning that much of the anxiety locally stems from the virus itself.

Will I contract the virus? Will someone I know contract the virus? Can I go to the store for groceries? How much disinfecting is required when I return home?

And then, you can tease out that geographical data. How are higher-income individuals stressed in comparison to lower-income? What about differences along racial lines? Data has shown higher mortality rates in African-Americans, for example, which leads to different fears than those in other communities.

In U.S. cities where there is also sufficient social media data, they will examine this offline data to see rates of infection, fatalities, when shelter-in-place was imposed, and more.

The final piece will be what they will do with this information. The goal is to create tools for social platforms to provide coping techniques or guidelines for use.

“Maybe that might include encouraging you to limit the amount of time you spend on social media,” Kumar said. “Or, maybe you step out and do something with family members. Some kind of physical activity. Then we can begin to examine how people react to these messages. Do we see that their anxiety levels are coming down, or not?”

“In this time, we have a very unique lens to study this pandemic in a whole new light as opposed to other events of a global scale,” De Choudhury said. “There is no guarantee this won’t come back. And even if it doesn’t, something else will. Being able to have these tools built and available will better prepare us for the future.”

For more coverage of Georgia Tech’s response to the coronavirus pandemic, please visit our Responding to COVID-19 page.

“The experience that these health workers had in these communities provided unique perspectives and ideas that produced the kinds of ideas that could be relevant,” said Ismail, now a Ph.D. student in Georgia Tech’s School of Interactive Computing.

“But because they were the lowest rung on the health infrastructure and were low income or low social class, those ideas weren’t recognized and represented.”

Around the same time, College of Computing alumnus Aditya Vishwanath, now a doctoral student at Stanford University, had a similar realization. He was working with Asha Mumbai, a non-profit in a low-resourced slum in India’s biggest city, using virtual reality to see how students appropriated and made sense of it. Like Ismail, he recognized a group of students who had unique viewpoints and drive, but too few resources to realize them.

Knowing how important it is to support innovation from those who understand the specific needs of a community, the two of them founded MakerGhat, a non-profit with the mission to take ideas from concept to creation and application where they are needed most: the communities they serve.

Situated in an impoverished neighborhood in Mumbai, MakerGhat is a community lab in which local students, young and old, can join to receive education and resources to put their ideas into practice. Makers join through subscription or scholarships if they are unable to afford membership. In exchange, they receive access to support ranging from an electronics room, a 3D printing and PC workstation, a science lab, a woodworking shop, and a design and workshop studio.

The space is intentionally unsophisticated. Enter the space, and you may find a mish-mash of supplies and painting on the walls, a far cry from the labs of the nearby Indian Institute of Technology-Bombay, one of the top technological universities in India.

“We want people to be encouraged to try things and not afraid to break it,” Ismail said. “We don’t want something that people are afraid to use.”

If a maker can’t find what they are looking for, they can turn to connections within the community to meet the need. Heavier equipment, for example, might require a trip to the local smith for welding.

“Students coming in have family members in these other industries, so it sets up an informal infrastructure where the students know where to go for a specific need,” Ismail said.

The model has resulted in a number of tangible outputs. In Summer 2019, a handful of interns from Georgia Tech, Stanford, and Smith College were able to take advantage of the Denning Global Engagement Seed Fund to fund their travel to India. Interns were there not just to teach or run the lab, but to co-learn with locals. Collaborations between the technical expertise of the interns and the locally-significant knowledge of the makers resulted in a handful of innovations that addressed local needs.

One collaboration resulted in a system that could compact plastic bottles to assist in a waste management challenge in Mumbai. Workers who collect waste locally and transport to recycling plants to sell to companies or government institutions face challenges transporting plastic bottles, the most common waste item, which take up a lot of space.

Another created a community mapping platform to help identify local resources. Makers and interns went into the community and conducted surveys to find needs specific to different geographies.

“A big part of this is engaging with the community to identify needs, current status quos, and how to approach the challenge,” Ismail said. “This happens in the schools too. What are the gaps that need to be addressed, and how can we help address them?”

MakerGhat serves about 300 students weekly, ranging from young to old – it is open to any age or background. Many come from STEM fields, but others may be interested in math or art or fashion design.

“It’s a melting pot,” Ismail said.

The goal is to turn MakerGhat into an incubator. As the first class of students graduates from the program, they will move on to other sources of education or work. Ismail said that she and her collaborators – which includes Vishwanath, a team programmer, local leaders in finances and project resources, and a group of 10 or so volunteers – want to help build companies from the ideas and innovations that formed at MakerGhat.

“The mission is to actually transform these students and community members into entrepreneurs,” Ismail said. “We want to take these creations to the next level and help them scale beyond their own community."

That might mean launching new MakerGhat centers elsewhere. The goal is to make the model of the original open-source so that other communities can replicate – in India and beyond. While it may play out different in each location depending on the community’s needs, the organizational structure would be the same.

“There’s a misconception that great innovation only comes from these big tech companies or big universities,” Ismail said. “But we want to challenge that narrative. Many of the great ideas that can make significant impacts on society come from the people in these communities of need themselves.”

Other members of the Georgia Tech community have contributed to the project. Neha Kumar, an assistant professor joint between the School of Interactive Computing and the Sam Nunn School of International Affiars, is an advisor. Students involved in a Makers-in-Residence program last summer were Ritesh Bhatt, Solum Onwuchekwa, and Josiah Mangiameli. Vishal Sharma, an incoming IC Ph.D. student, was also involved.

Wearable devices, such as the Fitbit and Apple Watch, offer less intrusive and more cost-effective sleeping monitoring, but the tradeoff can be inaccurate or imprecise sleep data.

Researchers at the Georgia Institute of Technology are working to combine the accuracy of sleep clinics with the convenience of wearable computing by developing machine learning models, or smart algorithms, that provide better sleep measurement data as well as considerably faster, more energy-efficient software. 

The team is focusing on electrical ambient noise that is emitted by devices but that is often not audible and can interfere with sleep sensors on a wearable gadget. Leave the TV on at night, and the electrical signal - not the infomercial in the background - might mess with your sleep tracker.

[Related News: New Deep Learning Approach Improves Access to Sleep Diagnostic Testing]

These additional electrical signals are problematic for wearable devices that typically have only one sensor to measure a single biometric data point, normally heart rate. A device picking up signals from ambient electrical noise skews the data and leads to potentially misleading results. 

“We are building a new process to help train [machine learning] models to be used for the home environment and help address this and other issues around sleep,” said Scott Freitas, a second-year machine learning Ph.D. student and co-lead author of a newly published paper.

The team employed adversarial training in tandem with spectral regularization, a technique that makes neural networks more robust to electrical signals in the input data. This means that the system can accurately assess sleep stages even when an EEG signal is corrupted by additional signals like a TV or washing machine.

Using machine-learning methods such as sparsity regularization, the new model can also compress the amount of time it takes to gather and analyze data, as well as increase energy efficiency of the wearable device.

The researchers are testing with a product worn on the head but hope to also integrate it into smartwatches and bracelets. Results would then be transmitted to a person’s doctor to analyze and provide a diagnosis. This could result in fewer visits to the doctor, reducing the cost, time, and stress involved with receiving a sleep disorder diagnosis.

Another issue that the researchers are looking at is reducing the amount of sensors needed to accurately track sleep. 

“When someone visits a sleep clinic, they are hooked up to all kinds of monitors and wires to gather data ranging from brain activity on EEG’s, heart rate, and more. Wearable tech only monitors heart rate with one sensor. The one sensor is more ideal and comfortable, so we are looking for a way to get more data without adding more wires or sensors,” said Rahul Duggal, a second-year computer science Ph.D. student and co-lead author.

The team’s work is published in the paper REST: Robust and Efficient Neural Networks for Sleep Monitoring in the Wild, accepted to the International World Wide Web Conference (WWW), scheduled to take place April 20 through 24 in Taipei, Taiwan.

Jake Abernethy has been promoted to associate professor in the School of Computer Science and granted tenure. Abernethy’s research focus is machine learning, where he enjoys discovering connections between optimization, statistics, and economics.

In 2011, he completed his Ph.D. at the University of California, Berkeley before becoming a Simons postdoctoral fellow for the following two years. After the water crisis in Flint, Mich., Abernethy worked on detecting lead contamination and infrastructure remediation. Prior to studying and teaching machine learning, Abernethy performed comedy and juggling shows, opening for Sinbad and Dave Chappelle.

Munmun De Choudhury has been promoted to associate professor in the School of Interactive Computing and granted tenure. De Choudhury is also affiliated with the GVU Center and Institute for People and Technology (IPaT) and leads the Social Dynamics and Wellbeing Lab (SocWeb Lab.) De Choudhury studies problems at the intersection of computer science and social media, building computational methods and artefacts to help understand human behaviors and psychological states and how they manifest online.

Prior to joining Georgia Tech in 2014, De Choudhury was a postdoctoral researcher in the nexus group at Microsoft Research, Redmond. In 2011, she received her Ph.D. from Arizona State University, Tempe. After graduate school, De Choudhury spent time at Rutgers University and was a faculty associate with the Berkman Center for Internet and Society at Harvard University.

Yajun Mei has been promoted to professor in the H. Milton Stewart School of Industrial and Systems Engineering. Mei's research interests include change-point problems and sequential analysis in mathematical statistics and sensor networks and information theory in engineering. Mei also examines longitudinal data analysis, random effects models, and clinical trials in biostatistics.

Mei received his Ph.D. in mathematics from the California Institute of Technology in 2003. He has also worked as a postdoc in biostatistics at the Fred Hutchinson Cancer Research Center. In 2010, Mei was awarded the National Science Foundation (NSF) CAREER Award and in 2008 was awarded Best Paper at FUSION. Mei was awarded the prestigious Abraham Wald Prize in Sequential Analysis in 2009.

Alex Endert has been promoted to associate professor and granted tenure in the School of Interactive Computing. Endert directs the Visual Analytics Lab where he and his students apply fundamental research to domains including text analysis, intelligence analysis, cybersecurity, and decision-making, and explore novel user interaction techniques for visual analytics.

Endert earned his Ph.D. from Virginia Tech in 2012, and in 2013 his work on Semantic Interaction was awarded the IEEE VGTC VPG Pioneers Group Doctoral Dissertation Award, and the Virginia Tech Computer Science Best Dissertation Award. In 2018, Endert received the NSF CAREER Award.

Editors Note: Molei Tao has been promoted to associate professor with tenure in the School of Math. Tao is an applied and computational mathematician, designing algorithms for faster and more accurate computations and developing mathematical tools to analyze and design engineering systems or answer scientific questions.

He earned his Ph.D. in control and dynamical systems with a minor in physics from the California Institute of Technolgy where he also worked as a postdoctoral researcher. He is the 2011 recipient of the W.P. Carey Ph.D. Prize in Applied Mathematics and a 2019 NSF CAREER Award.

Does watching too much television rot your child’s brain? How much is too much when it comes to video games? Is our time spent on social media impacting our mental health?

These are all important questions, but how they are asked matters to the ultimate conclusions we can draw. It is well-established that the most commonly used method in this area of research – user self-reports and survey questions – are prone to error. Now, new research from collaborators at Georgia Tech, Facebook, and the University of Michigan have shed light on the nature of error – that is to say whether user over or underestimate their data, who and which questions are more prone to error, and more.

Error in the data, said School of Interactive Computing Ph.D. student Sindhu Ernala, can impact the inferences drawn from the data itself.

“We know survey questions have several well-documented biases,” Ernala said. “People may not remember correctly. They can’t keep up with their time. They remember recent things more accurately than those further in the past. All of this matters because error in measurement might impact the downstream inferences we make. Accurate assessments of social media use is critical because of the everyday impact it has on people’s lives.”

Indeed, Ernala and her collaborators found that these biases held up in many surveys. In a paper accepted to the 2020 ACM Conference on Human Factors in Computing (CHI), they picked 10 of the most common survey questions in prior literature that investigate time spent on Facebook. The questions were asked in a variety of ways: open ended or multiple choice, the frequency of visits or the total time spent. They asked these 10 questions in a survey to 50,000 random users in 15 countries around the world.

With self-reported data in hand, they compared it to the actual server logs at Facebook to see how it stacked up. Interestingly, people most often overestimated the time they spent on the platform and underestimated the number of times they visited. Specifically, in the 18-24 demographic, a common age range for research done at universities, there was even more error in self-reports.

“This is important, because a lot of our research is done with these age samples,” Ernala said.

With this information in mind, the researchers made a handful of recommendation in order to improve the data and, thus, the research around the data itself:

1. As a researcher, if you are investigating time spent, consider using time tracking applications as an alternative to self-report time spent measures. These applications include things like Apple’s screen time feature or Facebook’s “Your Time on Facebook.”
    
2. If researchers want to use surveys, which often makes sense, consider using the phrasing with the lowest error or multiple-choice questions.

The researchers caution against using time spent self-reports directly, but rather interpret reports as noisy estimates of where someone falls on a distribution. More important when determining wellbeing outcomes is how users actually spend their time on the platform.

“Social platforms change and user habits change over time,” Ernala said. “The questions now might not be the best questions five or 10 years from now. This is fluid, and we need to continue to look at this to make sure our past and future research is well-informed.”

She and her collaborators hope to contribute positively to this ongoing process by providing some validated measures that can be used across studies, while understanding that these methods may change over time as user habits transform.

First-year Ph.D. students Daniel Bolya (advised by Judy Hoffman) and Joanne Truong (advised by Dhruv Batra and Sonia Chernova) were recognized by the program, which supports graduate students pursuing research-based Master’s and doctoral degrees at United States institutions.

The NSF GRFP provides financial support for three years, comprised of a $34,000 stipend per 12-month fellowship year, as well as a direct payment of $12,000 to Georgia Tech toward the cost of education for each of the three years of fellowship funding.

Bolya’s work is in machine learning and computer vision. Recent work at Georgia Tech has focused on error profiling in instance segmentation and object detection models. His method, building upon previous work at MIT, is unique in that it captures all possible sources of error in a model, while properly weighing the importance of each. He plans to continue pursuit of faster methods of instance segmentation that can he make accessible. Current methods are not practical for many applications due to limits in speed, accuracy, and data efficiency. His research addresses this challenge.

“This is not just about computer vision,” he said in his research statement. “Improving instance segmentation would impact the tech we use every day.”

Like his work at MIT, called YOLACT, he plans to fully release the project open source once it is ready.

Truong’s long-term research goal is to develop robots that can see, talk, reason, and act in complex human environments. Specifically, she will focus on a method called “sim2robot transfer,” which develops efficient domain adaptation techniques to enable pre-training of AI agents in simulators while ensuring that the learned skills generalize to a real robotic platform.

“The overall goals of my research plan are to, one, break down the possible errors in simulation-to-reality transfer that result in a reality gap, and, two, close the loop between simulation and reality by using data collected on a real robot to finetune and optimize parameters in simulation,” she said.

She worked on the first goal last fall, achieving optimization in simulator settings for sim2real predictivity. Currently, she is working on the second goal, developing domain adaptation techniques to enable low-shop adaptation between simulation and reality.

Below, you’ll find just a couple of the tools you can interact with online, giving you opportunities from learning how to code to creating art. But this is only just a start – we’d love to hear from you.

If you’re a Georgia Tech student or faculty member, submit your interactive tools to communications officer David Mitchell at david.mitchell@cc.gatech.edu. We’ll add to the list, share with our audience, and help everyone find some enjoyment during a difficult time.

Create Your Own Generative Art Pieces – submitted by Devi Parikh

Looking for a new piece of art for your wall? With this tool, you can flex your creative muscles. Choose a style, adjust the values, colors, and properties, and generate a piece that would fit in nicely in your home.

This work demonstrates a broader area of research into machine learning and creativity. The first piece of AI-generated art to go to auction sold for $432,500 in 2018.

LINK: https://cc.gatech.edu/~parikh/art.html

Interact with Visual Chatbot – submitted by Devi Parikh

Parikh’s lab is doing research in an area called visual question answering. Developed in 2017, this demo allows you to upload an image and have a conversation with a chatbot about it. Pick out an image you’ve taken or just grab one from the web and ask questions to see just how quickly and accurately this AI can perform the task. This research is key to developing agents that can reason about specific tasks in the real world.

LINK: http://demo-visualdialog.cloudcv.org/

Learn to Code Using EarSketch and TunePad – submitted by Brian Magerko

Have you been dying to learn how to code? There’s no time like the present. Without the benefit of a classroom setting to learn all the ins and outs, you might find a usable tool like EarSketch beneficial. EarSketch uses music to guide the learner. With sounds from the EarSketch library or your own uploads, along with Python or JavaScript to code, you can produce quality music online.

Like EarSketch, TunePad – developed in collaboration with Northwestern University – is a tool for creating music using the Python programming language. No knowledge in music or coding is required to get started. Get those musical juices flowing, and start creating.

LINK: earsketch.gatech.edu

Learn About Grasping Tasks Using this Online Tool – submitted by Samarth Brahmbhatt

This tool allows people to interactively explore how we grasp household objects. So, why is this important? Grasping is a key capability in the development of household robotics. In order to train robots how to grab and use items in the house, we need to identify the most efficient approach. Explore this tool, which includes items from an apple to a doorknob to a video game controller.

LINK: https://contactdb.cc.gatech.edu/contactdb_explorer.html

Maddux, a meticulous documenter of pitch sequences and batter results throughout his career, once explained to Mazzone in between innings that the leadoff batter in the following frame would pop out to third base on the fourth pitch of the at-bat. He’d start him with a fastball, change speeds for strike two, waste a pitch outside, and then induce the popup on a one-ball, two-strike count. Sure enough, a few minutes later, Maddux did exactly as he’d said.

There are a couple of lessons here: One, Maddux was a wizard. Many pitchers over time have tried to replicate his impeccable approach to the game, but few have ever succeeded at that level; two, pitch sequence matters – perhaps more than how overpowering your fastball is or how sharp the break is on your curve.

Capitalizing on this intuition, a group of undergraduate students at Georgia Tech are working with the softball team to provide an automated upgrade to players’ training. Using the wealth of statistics kept by the team – pitch-by-pitch data for balls, strikes, types of pitches thrown, and results – they have trained an algorithm that can select the best pitch to throw in any given situation.

The tool is used by the coaches and pitchers for game planning purposes, generating daily reports after every game and practice to help inform coaches of trends in sequences and results.

“In baseball and softball nowadays, data analytics has become such an incredibly important part of the game,” said Jack Bennett, a third-year Industrial Engineering student. “Anything that can get them data to go into games more prepared. Technology is at the forefront of this.”

They began using the approach during the 2019 season. Bennett and partners Zach Panzarino (third-year Computer Science) and Ron Kushkuley (third-year Computer Engineering) had demonstrated a similar capability at last year’s Sports Innovation Hackathon using data for Atlanta Braves pitcher Mike Foltynewicz, finishing in third place. Doug Allvine, assistant athletics director for innovation at Georgia Tech, put the team in touch with softball coach Aileen Morales. Morales was interested, and the students were able to begin testing the approach.

It works like this: The softball team keeps track of its own data – not just player statistics, but pitch selections and results for every pitcher in every game throughout the season. That’s a lot of data and can offer a lot of information. What happened when Pitcher X threw a 3-2 changeup to a lefthanded batter?

But it goes a little deeper than that. Panzarino, Bennett, and Kushkuley found that the pitch sequence is what matters most. That follows the standard strategic thinking – a slider away can be more effective if set up by an inside fastball on the previous pitch, for example. What the algorithm does, however, is consider the order each pitch is thrown in the at-bat and provide a score for which pitch will be most effective based on past data.

“We leverage sequences, the count, outs, everything,” Panzarino said. “Looking at the current state and the previous pitches, it will score all the potential future routes a pitcher can choose. We give them reports before each game so that they can prepare, and then we look at success or failure after the game.”

After a test run a year ago, the students have honed the technology and are working with the team again this year. Qualitatively speaking, they said they noticed results throughout the year.

“When we first gave them our analysis, it would recommend certain stuff in certain situations,” Bennett said. “Maybe it would say a changeup should be thrown more in this situation. Then, when we’d get postgame data later, we’d see that more changeups were being thrown and were continuing to be effective.”

“When I first saw what they were developing, I was beyond impressed,” Morales said. “We are very meticulous with collecting data in our program and trying to find ways to learn more about what is and what is not working for our athletes. It’s remarkable to see how they can take the data we had and leverage it in a way that allowed us to finetune our training.”

Recently, at the 2020 Sports Innovation Hackathon, the group developed a similar solution for baseball. They received runner-up in the competition, and hope to connect further with the Georgia Tech baseball team in the future.

“Tons of theory has been written on how pitchers should approach sequencing in games, but this is a model that can show you the data about how well that works,” Panzarino said.

“You’ve got to start somewhere,” said Bryant, a robotics student in the school. “I feel like whenever I’m going through my research, the way I approach it is I have these grand ideas, and I have to break it down to this and this and that. I’m the type who’s normally working on four or five things at the same time. Dr. Howard always tells me: ‘Okay, slow down. We have to start somewhere. We have to start somewhere so we have something to move toward.’”

It’s an appropriate metaphor for Bryant, who began her career in computer science with no previous experience as an undergraduate student at the University of South Carolina. It also applies to all the other women in the field who, like Bryant, rely heavily on those who come before them and pass the torch to those who come after.

Unlike many robotic students, who have stories about being introduced through Lego Mindstorms that allow them to build and program their own Lego robots, the concepts of computer science and robotics were the furthest things from Bryant’s mind. They were completely foreign ideas that she had never given a second thought during her time in middle school and high school.

“That wasn’t me,” Byrant said.

Instead, she happened to take an Intro to Java class in her first year. There, she met her first collegiate mentor, Karina Liles. Liles was a graduate student who worked in a robotics lab and, after first semester, invited Bryant to come work with her as an undergraduate assistant. Bryant saw it as a part-time job and a place she could have her own desk. She wasn’t thinking about it as much more than that.

“I had no idea what her research was,” Bryant said. “I knew it was a robotics lab, so that was cool. And she was in education for low-resource communities. I came from a school that didn’t offer computer science at all, so I found that appealing.”

Once Bryant was introduced to the research process, asking and answering new questions, she was hooked. Collecting data, programming and testing robots then seeing children interact with them face-to-face.

“It made all the difference,” she said.

It was a start, but it was still a new world. Neither of her parents had earned four-year degrees, and her dad had passed away when she was in middle school. When she told her mom and grandmother that she was interested in computer science, she received some hesitancy back. But, while neither had experience in technology, they had raised her to be inquisitive and to seek out mentorship. That’s exactly what Bryant did.

Thrown into the deep end, she relied on Karina and a handful of other women she came across at South Carolina or conferences like Human Robot Interaction and Grace Hopper.

“I was drawn to women in the field, because the nurturing and the support from people who are also in an underrepresented group – whether it’s gender or race or whatever – they can talk to you about those specific challenges that you might come across,” Bryant said.

Eventually, that led her to Howard.

After her junior year in Columbia, Bryant applied to a program called Distributed Research Experiences for Undergrads (DREU). For minority undergraduate students, DREU matches students with mentors who have signed up to take on undergrads into their labs over the summer. Although students can be matched with anyone in the United States, Bryant’s mentor happened to be Howard.

“I was so excited,” said Bryant, who knew of Howard’s research through her own work at South Carolina. The work she was doing with social robots for kids with autism aligned with Howard’s, and it wasn’t uncommon for the Georgia Tech professor’s name to be cited in one of their papers. “There was a student matched with a mentor in Hawaii, and everyone thought that was the luckiest one. I was like, ‘No, I’m pretty sure I got the best deal.’”

Bryant worked on a project in Howard’s lab that summer with three other undergrads. Howard was immediately impressed with Bryant because of her unique programming ability.

“I remember needing someone to program the robot, and she was just like, ‘Oh, I can do it,’” Howard said. “She impressed me right away, and when it was time for her to choose a graduate program I knew she’d fit perfectly in our lab.”

Their work together now is impacting individuals with disabilities, making technology work for everybody including those with motor or visual or hearing impairments. They are investigating robot gendering and its impact on human trust, and work toward inclusivity with programs like AI4All.

Bryant is using the inspiration that Howard has provided to her and feels a responsibility to continue that for the next generation of women roboticists. She is humbled by people who now look up to her like she did to Howard, being left speechless by a young student who featured her for a school project on Black History Month.

When she goes to Grace Hopper, Bryant loves meeting the undergrads and passing on her advice about academics and the challenges women face in the field. She also watches to make sure they are asking questions or calls on those who look like they have a question, but are afraid to ask.

“I remember being that person in the room,” she said. “Women don’t just need representation, they need a voice. I want to be their champion, connect them to the right people.”

And her biggest advice to them might sound familiar.

“Just start,” she said. “You’ve got to start somewhere.”

The ASF opening weekend at Georgia Tech also includes one of the most unique experiences in music performance on March 7. The Guthman Musical Instrument Competition Concert showcases nine global finalists playing unique instruments created for the competition. It's free and open to the public.

New this year as part of the Guthman event is the Music, Art, and Technology Fair, hosted by the Georgia Tech School of Music and Cycling ’74. It's a unique opportunity to share projects at the intersection of art and technology in a hands-on, interactive, science-fair format.

The GVU Center at Georgia Tech created interactive graphics to explore the two-week festival and find specific events connected to Georgia Tech. The institute is one of the founding members of ASF.

ASF events presented by or taking place at Georgia Tech:

March 6

2100: A Climate Odyssey

Presenting Partners: Science ATL, The Weather Channel, the National Weather Service, Peachtree City, Out of Hand Theater. 

Time and Location: 8 p.m., Ferst Center for the Arts at Georgia Tech

March 7

STEAM at Tech Day   

Presenting Partner: Georgia Tech CEISMC

Time and Location: 12 p.m., Clough Undergraduate Learning Commons at Georgia Tech

Pitch Your Future

Presenting Partners: Institute for Electronics and Nanotechnology at Georgia Tech, Jimmy Carter Presidential Library

Time and Location: 1 p.m., Carter Presidential Library & Museum

Guthman Music, Art & Technology Fair

Presenting Partners: Georgia Tech’s School of Music, Cycling ‘74

Time and Location: 4 p.m., Ferst Center for the Arts at Georgia Tech

Guthman Musical Instrument Competition 

Georgia Tech’s School of Music

Time and Location: 7 p.m., Ferst Center for the Arts at Georgia Tech

March 10

Project Change: STEM Teachers @ Tech Day

Presenting Partner: Georgia Tech CEISMC    

Time and Location: 9 a.m., Georgia Tech Student Center

Playing Mother Nature: A Night of Simulating Earth Science Phenomena!

Presenting Partner: Georgia Tech’s School of Earth and Atmospheric Science       

Time and Location: 7 p.m., Manuel's Tavern

March 12

Sober Science Speakeasy  

Presenting Partner: Georgia Tech’s STEM Comm VIP team 

Time and Location: 7:30 p.m., Coda Building, Midtown

Science Riot   

Presenting Partners: Georgia Tech, Science Riot

Time and Location: 7:30 p.m., Highland Inn Ballroom

March 14

8th Annual Latino College & STEM Fair

Presenting Partners: Georgia Tech CEISMC Go-STEM          

Time and Location: 9 a.m., Georgia Tech Student Center

Science of Star Wars 

Presenting Partner: Institute for Electronics and Nanotechnology at Georgia Tech

Time and Location: 10 a.m., Marcus Nanotechnology Building Atrium

Investigating the Nanoscale 

Presenting Partner: Institute for Electronics and Nanotechnology at Georgia Tech

Time and Location: 11 a.m., Marcus Nanotechnology Building Atrium

March 18

Science Improv          

Presenting Partner: Georgia Institute of Technology

Time and Location: 7:30 p.m., Whole World Improv Theater

Despite the enormous strides made in artificial intelligence to develop systems that can answer simple questions and requests, the kinds of natural conversational language humans have with each other when giving more complex directions or telling stories has thus far been out of reach.

Research from Georgia Tech’s School of Interactive Computing, however, provides a novel approach that improves the combination of automated story generation with natural language. The development is an important step in providing AI assistants the capability to more naturally converse with humans.

“Let’s think of a future version of Siri or Alexa, where you have a complex task that’s not just ‘Look this thing up on the internet,’ or ‘Tell me what the weather is outside,’” said Mark Riedl, an associate professor at Georgia Tech and the faculty lead on the research. “Maybe you want to plan your day or a birthday party. Think of the response like a little story, a narrative that conveys the requested information.

“It’s a missing capability in AI – they just don’t understand us or communicate with us in the same ways that we understand each other.”

Riedl and his team approached the challenge by viewing the exchange of information as stories – a series of events, one after the other, that leads to some conclusion. Past research on the topic identified patterns in language to identify how stories are constructed – namely that a verb generally changes the action and conveys a new event in a story.

“By boiling down these stories drawn from the internet to essential verbs and actions, we can extract patterns from stories better,” Riedl said. “There are a lot of ways to talk about marriage, but at the end of the day someone is marrying someone else.”

This paper, the third in the series, took the next step: If you take away all the words to identify the patterns in a story, you need to be able to put them back in naturally and intelligently in a way that humans are accustomed to. Put simply, it’s like building an outline and then filling in the details.

The system works by building the outline through a neural network trained on sequencing events. With the help of story examples drawn from the internet, it applies machine learning to produce a series of events, one leading to the most likely next outcome. That outline guides a second neural network that applies natural language – grammar, syntax, spelling, everything else you need to make the story intelligible – to produce more elaborate sentences.

“If you’re asking for directions for how a birthday party should go, you don’t want just ‘Jill eats cake; Jill opens presents,’” Riedl said. “You want something more akin to the stories we share as humans. It’s actually more difficult for us to process information when it’s delivered in a way we’re not accustomed to.”

The researchers found that an ensemble approach works the best. They use a series of five algorithms, each with different capabilities in accuracy and natural language generation, produces the best stories. Because one algorithm isn’t uniformly better at all aspects of the task, it will be run through all five to find the highest confidence level of the sentence.

“One technique might provide bland sentences, but is accurate with the actual content,” Riedl said. “Another might be very good at putting in a narrative flourish, but they fail more often. You want that nicer sentence, but you also want it to be able to catch mistakes in the content.”

The ensemble approach scored significantly higher in human studies than the individual algorithms alone. Human trust in their AI and robot assistants, Riedl said, was key to adoption in the future.

“The key is that you want to place that trust in your machine counterpart, but it has to earn that trust on correctness and accuracy,” he said.

The paper is titled Story Realization: Expanding Plot Events into Sentences, and will be presented at the 34^th AAAI Conference on Artificial Intelligence on Feb. 7-12 in New York City. The research is funded under a grant from DARPA.

Jill Watson – Georgia Tech’s AI designed explicitly for this purpose – turned four years old this January, with the AI’s birthday coinciding with the announcement of the 10 semifinalists for IBM’s AI XPrize competition. Georgia Tech’s emPrize team, led by Professor of Interactive Computing Ashok Goel and utilizing Jill Watson as the key technology, was named as one of the semifinalists.

The competition started in 2016, the year of Jill’s arrival in a graduate computer science course at Georgia Tech, and has “sought to accelerate the adoption of AI technologies and spark creative, innovative, and audacious demonstrations of the technology that are truly scalable to solve societal grand challenges.” After nearly four calendar years, XPrize will name a winner in April.

As part of the GT emPrize team’s work, the Jill Watson TA not only answers student questions about course requirements but can answer questions about another AI named VERA, or the Virtual Ecological Research Assistant.

Jill helps users learn how to use VERA, a system which enables students in GT’s Intro to Biology course (and online science seekers) to create their own ecological models from a web browser. Unlike the Jill Watson TA, which is currently used only by GT students, VERA is open to anyone with an internet connection.

Another part of emPrize is the Jill Social Agent, whose lead designer, Ida Camacho, is a recent alumna of Georgia Tech’s Online Master of Science in Computer Science program (OMSCS) and understands the pressures and uncertainties of online learning.

The Jill Social Agent in essence gives students just starting online courses a chance at “speed friending”. If online students feel they have more peer support and connections from the start, this might translate into success in the course. Hear from Camacho on the Tech Unbound podcast with GVU Center as she reveals some of her AI’s design and the educational experience that informed her work on emPrize.

Learn more at http://emprize.gatech.edu/ or explore a timeline of Jill's evolution. 

Students and faculty would be forgiven in thinking Jill Watson is a single teaching assistant. Each course that utilizes the Jill TA has its own custom “knowledge base” that the AI leverages to answer basic student questions 24/7.

Explore the Timeline of Jill’s Growth

In addition, a new AI, the Jill Social Agent, was designed and launched in 2019 to explicitly connect students quickly and get them working together. The agent was developed in part as a response to high attrition rates that plague online learning in general.

The lead architect for the Jill Social Agent, Ida Camacho, OMSCS ’19, discusses the AI on an episode of the Tech Unbound Podcast from the GVU Center. It’s a fascinating inside look at Camacho’s approach to building social structures for online education and her own journey as an OMSCS student.

Other major milestones from the Jill TA in 2019:

• Introduced in residential classroom for first time.
• Deployed in first non-CS course (Intro to Biology).
• Customized to train users on the VERA AI, an ecology modeling system.
•  

The new decade promises more educational advances made possible by the Jill Watson AI framework. Learn more at emprize.gatech.edu.

Desai, an associate professor in the Scheller College of Business, will be the center’s first associate director for Legal, Policy, Ethics, and Machine Learning. Not a technologist by training, Desai will draw from his experience working at Princeton's Center for Information Technology Policy and Google as Academic Research Counsel to help policy makers, legal scholars and technologists work better together. This includes helping each party understand how a given technology works and what issues it might raise.

“I am excited to be part of ML@GT because of the opportunity to be part of a world class group of thinkers and to connect our work to the world.  I believe there is a need to bridge the worlds of technology and law, policy, and ethics,” said Desai. “ML@GT is poised to increase not only machine learning insights and breakthroughs but also the way in which machine learning is built and used to serve society. I am honored and thrilled to be part of building that future.”

Xie, an associate professor in the H. Milton Stewart School of Industrial Systems Engineering (ISyE), is the first woman to join the leadership team. She will serve as the associate director for machine learning and data science where she will create better synergy between the ongoing research and education efforts between data science and machine learning as Georgia Tech builds a leading program in these areas.

“I am particularly excited to work with the broader community of students and faculty on campus who are interested or involved with machine learning and data science and foster their participation,” said Xie.

Lan, also an associate professor in ISyE has been appointed as the associate director for machine learning and statistics. In this role, Lan will promote research at the intersections between optimization, statistics, and machine learning and how they also apply in engineering. He will also help better facilitate communications for students coming from different home colleges or schools across campus.

“I am excited to be joining the team with active and dynamic academic leaders. I look forward to working with them to address a diverse set of challenges that ML@GT faces, e.g., being adaptive to the priorities and criterions for our affiliated faculty members and students across different academic units,” said Lan.

As the associate director for machine learning and artificial intelligence, Riedl, an associate professor in the School of Interactive Computing, will coordinate ML@GT’s strategy with respect to the broader field of artificial intelligence.

“Artificial intelligence and machine learning have the potential to radically change virtually every aspect of our lives. With thought and care, these technologies can be a force for good. Georgia Tech is well-positioned to be a major voice in how technology and policy shape the future,” said Riedl.

With more corporations integrating machine learning and artificial intelligence into their businesses, the center’s need for managing those relationships has increased significantly. Chau, an associate professor in the School of Computational Science and Engineering, will lead those relationships as the associate director for corporate relations for machine learning.

“I enjoy bringing people together, connecting industry with Georgia Tech researchers, bridging disciplines and innovating at their intersections. I’m excited to begin my new role as it will be a great way to help Georgia Tech further expand its national and global footprint,” said Chau.

As the associate director for community and students, Davenport is charged with creating a tight-knit community among faculty and students. Davenport, an associate professor in the School of Electrical and Computer Engineering, will work closely with the center staff to coordinate events and other opportunities to increase discussion and collaboration between research units.

The six new members will join existing leadership members Irfan Essa, Justin Romberg, Zsolt Kira, and Le Song.

About the Machine Learning Center at Georgia Tech

The Machine Learning Center at Georgia Tech is an interdisciplinary research center bringing together more than 190 faculty members and 60 machine learning Ph.D. students from across the institute for meaningful collaboration and innovation in machine learning and artificial intelligence. Students and faculty are experts in areas including, but not limited, to computer vision, natural language processing, robotics, deep learning, ethics and fairness, computational finance, information security, and logistics and manufacturing. For more information, visit www.ml.gatech.edu

Students will use computer science and coding to build their remix using musical samples from Ciara’s song “SET,” as well as other sounds from the EarSketch library. Students will learn looping (repeating) to extend the length of their song, use strings to create new beats, create custom functions representing different song sections, and learn to upload their own sounds to the EarSketch library.

“We are excited to support the innovative and unique work Georgia Tech and EarSketch are pioneering to give students across the country more access to computer science, coding, and music,” said Jeff Wilke, CEO Worldwide Consumer, Amazon. “This competition will give thousands of students from underserved and underrepresented communities the opportunity to try something new and fun. It will build their confidence and, most importantly, encourage them to think creatively.”

Students and teachers can learn more about the competition details at www.amazonfutureengineer.com/earsketch - all high school students are encouraged to participate, either in class or on their own. The top three student winners will each receive an all-expenses-paid trip to Amazon’s headquarters in Seattle, Washington to be an “Amazon Future Engineer” for the day. Additional winners will receive a PreSonus Audiobox 96 Studio and Amazon.com Gift Cards. The competition opens today and will close on Monday, January 20 at 11:59PM EST.

The Bureau of Labor Statistics projects that by 2020 there will be 1.4 million computer-science-related jobs available and only 400,000 computer science graduates with the skills to apply for those jobs. Computer science is the fastest-growing profession within the Science, Technology, Engineering and Math (STEM) field, but only 8% of STEM graduates earn a computer science degree, with a small percentage from underserved backgrounds. In multiple research studies, students using the EarSketch platform significantly increased their positive attitudes towards computing and their intentions to persist in computing, with particularly significant impacts on students from groups historically underrepresented in the field. Female students expressed even greater gains in computing confidence, motivation, and identity as compared to their male counterparts.

“Music and coding are both highly prevalent today, but coding is less apparent—EarSketch provides an amazing way to experience these creative disciplines simultaneously,” said Dr. Roxanne Moore, Senior Research Engineer at Georgia Tech and project lead for the remix competition.

“We know that creativity is the key to success in both music and computer science,” said Professor Jason Freeman, Chair of the School of Music at Georgia Tech and co-creator of the EarSketch platform. “We’re thrilled to partner with Amazon to support more students as they unlock their creative potential through EarSketch.”

To date, more than 375,000 students in all 50 states and over 100 countries have used EarSketch. Programs like EarSketch serve Georgia Tech’s mission to meet the demand for STEM (science, technology, engineering, and mathematics) professionals by opening the eyes of more students to these engaging and important subjects.

Launched in November 2018, Amazon Future Engineer is a four-part childhood-to-career program intended to inspire, educate, and prepare children and young adults from underrepresented and underserved communities to pursue careers in the fast-growing field of computer science. Each year, Amazon Future Engineer aims to inspire millions of kids to explore computer science; provides over 100,000 young people in over 2,000 high schools access to Intro or AP Computer Science courses; awards 100 students with four-year $10,000 scholarships, as well as offers guaranteed and paid Amazon internships to gain work experience. Amazon Future Engineer is part of Amazon’s $50 million investment in computer science/STEM education. In addition, Amazon Future Engineer has donated more than $10 million to organizations that promote computer science/STEM education across the country.

AMAZON CONTACT:

Amazon.com, Inc.
Media Hotline
Amazon-pr@amazon.com
www.amazon.com/pr

FairVis, presented at IEEE Vis 2019 in Vancouver, is the first system to integrate a novel technique that allows users to audit the fairness of machine learning models by identifying and comparing different populations in their data sets. 

According to School of Computational Science and Engineering (CSE) Professor and co-investigator Polo Chau, this feat has never been accomplished by any platform before, and is a major contribution of FairVis to the data science and machine learning communities.

“Computers are never going to be perfect. So, the question is how to help people prioritize where to look in their data, and then, in a scalable way, enable them to compare these areas to other similar or dissimilar groups in the data. By enabling comparison of groups in a data set, FairVis allows data to become very scannable,” he said.

In order to do accomplish this, FairVis uses two novel techniques to find subgroups that are statistically similar. 

The first technique groups similar items together in the training data set, calculates various performance metrics like accuracy, and then shows users which groups of people the algorithm may be biased against. The second technique uses statistical divergence to measure the distance between subgroups to allow users to compare similar groups and find larger patterns of bias.

These outputs are then viewed and analyzed through FairVis’ visual analytics system, which is designed to specifically discover and show intersectional bias. 

Intersectional bias, or bias that is found when looking at populations defined by multiple features, is a mounting challenge for scientists to tackle in an increasingly diverse world.

“While a machine learning algorithm may work very well in general, there may be certain groups for which it fails. For example, various face detection algorithms were found to be 30 percent less accurate for darker skinned women than for lighter skinned men. When you look at more specific groups of sex, race, nationality, and more, there can be hundreds or thousands of groups to audit,” said Carnegie Mellon University Ph.D. student Alex Cabrera.

Cabrera is the primary investigator of FairVis and has been pursuing this problem since he was an undergraduate student at Georgia Tech.

“During the summer of my junior year I had been researching various topics in machine learning, and discovered some recent work showing how machine learning models can encode and worsen societal biases. I quickly realized that not only was this a significant issue, with examples of biased algorithms in everything from hiring systems to self-driving cars, but that my own work during my internship had the possibility to be biased against lower socioeconomic groups.”

This is when Cabrera reached out to Chau who then recruited the help of CSE alumni Minsuk Kahng, CSE Ph.D. Fred Hohman, College of Computing undergraduate student Will Epperson, and University of Washington Assistant Professor Jamie Morgenstern.

Morgenstern is the lead researcher for a number of projects related to fairness in machine learning, including the study Cabrera mentioned about self-driving cars. This particular study shows the potentially fatal consequences of algorithmic bias which highlights the severity of software created without fairness embedded into its core. 

FairVis is one of the first systems that helps us achieve a dramatic step towards understanding and addressing the problem of fairness in machine learning, and prevents similar headlines from making their way to reality in the future.  

However, Cabrera stressed that the solution does not simply end with better data practices.

“Fairness is an extremely difficult problem, a so-called ‘wicked problem’, that will not be solved by technology alone,” he said. 

“Social scientists, policy makers, and engineers need to work together to make inroads and ensure that our algorithms are equitable for all people. We hope FairVis is a step in this direction and helps people start the conversation about how to tackle and address these issues.”

“Giving natural language instructions to a robot is a fundamental research problem on the critical path to developing more flexible domestic robots that can work with people,” said Peter Anderson, a research scientist at Georgia Tech.

In a recent paper, Georgia Tech has introduced a new way for robots to reason about navigation instructions in an unknown environment.

The team created a semantic map representation that updates each time the robot moves or sees something new. To reason about navigation instructions using this map, the lab found a way to leverage an algorithm used in classical robotics and apply it to artificial intelligence. The algorithm, called Bayesian state estimation, usually tracks the location of a robot from sensor measurements like lidar and wheel odometry. By manipulating the algorithm, Georgia Tech says their robots can use it model language instruction inputs instead.

The paper got its title "Chasing Ghosts: Instruction Following as Bayesian State Tracking" because rather than tracking a robot from sensor measurements, the team is tracking the likely trajectory taken by an ideal agent or human demonstrator in response to the instructions. In this approach, the sensor measurements are the instructions themselves.  This algorithm allows the agent to “reason” about all the different trajectories it could take and the probability of each trajectory when completing a task. By using an explicit map, researchers are easily able to inspect the model to see where the agent thinks the goal is and where it is likely to move next.

Currently, the robots move in simulated reconstructions of buildings, and communication is through written text, though some applications and off-the-shelf speech-to-text systems could work in conjunction with the existing system, according to researchers.

“Spoken language would definitely be more natural in many situations, so we might in the future investigate models that go directly from speech to robot actions,” said Anderson.

Anderson particularly likes to think about this work in regards to telepresence robots, though it could be applied to any robot.

“Telepresence robots are a great idea, but they are not as popular as they could be. Maybe we need smarter, more natural robots that just go where you tell them to go and look at what you ask them to look at,” said Anderson.

Think about all of the time that is lost commuting to work and walking to meetings. Imagine how climate change might be positively impacted if people needed to travel less for business. Anderson hopes that this work will allow people to focus more on their meetings, conversations with people, and, perhaps, help with climate change, rather than micromanaging a robot or jetting off around the world. 

This work will be presented in December at the Thirty-third Conference on Neural Information Processing Systems (NeurIPS) 2019 in Vancouver, British Columbia.

The conference fosters the exchange of research on the theoretical, technological, biological, and mathematical aspects of neural information processing systems. ML@GT research spans all of the categories, including work on neural data, fairness in machine learning algorithms, and teaching artificial intelligence to work in changing environments.

“NeurIPS continues to be an exciting conference to attend because of the diverse research that is being presented each year. It is one of the most sought-after and anticipated conferences every year, and it’s good to see ML@GT have a good variety of papers being accepted,” said Tuo Zhao, an assistant professor in the H. Milton Stewart School of Industrial and Systems Engineering (ISyE). Zhao has three accepted papers.

NeurIPS also continues to be a hotspot for major technology companies like Google, Microsoft, Facebook and to recruit new talent. 

To see a full list and recaps of ML@GT’s accepted papers click here.

We’ve all been there. For the first time in months, you’ve been logged out of your social media account and need to log back in. The problem is it’s been so long since your last log in that you don’t remember your password. You try every combination of baby and pet name, sister’s birthday, childhood street address – nothing works, and now you’re locked out.

If only there was a better way to remember these passwords after extended periods of disuse.

Luckily, researchers at Georgia Tech have come up with a novel solution to this longstanding problem, applying an old memory technique to new technology to offer users a more effective authentication method. Known as ‘the Memory Palace, the new tool is a three-dimensional virtual labyrinth navigated in the first-person perspective.

In cases of infrequent authentication, the Memory Palace works in place of an account’s password. Users create their own personal path with multiple left or right turns through a maze that must then be recreated to log in to their account. If the user makes it through the maze, similar to the one found in the old Windows three-dimensional labyrinth screensaver, they gain access.

Studies evaluating the technique showed that visual-spatial secrets were most memorable if navigated in the three-dimensional first-person perspective. They also showed that, in comparison to Android’s 9-dot pattern lock, the Memory Palace was significantly more memorable after one week, was harder to break through shoulder surfing (capturing passwords by looking over someone’s shoulders), and were not significantly slower to enter.

VIDEO: Explore 'The Memory Palace'

“Humans have evolved with remarkably persistent and fast-imprinting spatial memories, owing in no small part to our nomadic history,” said School of Interactive Computing Assistant Professor Sauvik Das, the lead researcher on the project. “Many people can, for example, clearly visualize and mentally walk through their childhood homes, even if they haven’t stepped foot in it for decades. They may only need to be shown once or twice how to drive to a new part of a familiar city.

“Our key insight was simple: Why not co-opt this incredibly strong spatial memory system for infrequent authentication?”

This visual-spacial authentication is based upon an old memory technique of the same name, also called the “method of loci.” That approach uses visualizations with the use of spatial memory, familiar information about one’s environment, to quickly and efficiently recall information. World Memory champions have applied this technique in competition for years, associating vivid images along a specific path with digits, letters, or playing cards they are required to memorize. In fact, the technique dates all the way back to ancient Greeks and Romans.

When developing their program, researchers focused on a few keys to their method. In addition to security against common attacks like random guessing or shoulder surfing, they needed the authentication secret to be memorable without much practice or reinforcement and they needed it to be deployable to the public.

“Users are unlikely to accept a solution that requires significant upfront training or effort,” said Das, an expert in a field dubbed social cybersecurity that examines social norms that impact the adoption or rejection of security techniques. “Also, the solution should be cost-effective and not require specialized hardware. Many authentication solutions have been proposed, but most fail to be widely adopted for these reasons.”

Existing solutions fall short in these requirements. Biometrics, like a thumb print or facial recognition, require specialized hardware that can be expensive for infrequent use cases. PINs and graphical passwords have problems in long-term memorability without frequent reinforcement, or are otherwise vulnerable to shoulder surfing.

“The Memory Palace addresses each of these concerns with a proven memory technique that can hold up over time but is not easily stolen,” Das said.

Das provided a handful of potential instances of infrequent authentication. Perhaps a session persists for a long period of time, like social media accounts, or a user must log in on a different device than normal, like a Netflix account on a web browser versus a smart TV. Other situations include occasionally-accessed resources, like a conference room secured with a smart lock, or as a fallback authentication method where a secondary secret is needed to recover access to an account where the primary secret has been compromised.

To deploy to the public, an app could implement the Memory Palace as a means of authenticating users. Alternatively, an operating system like Android could implement it as a means of authenticating into a device and automatically handle authenticating into any existing apps on the device.

This work was presented in a paper, titled The Memory Palace: Exploring Visual-Spatial Paths for Strong, Memorable, Infrequent Authentication (Sauvik Das, David Lu, Taehoon Lee, Joanne Lo, Jason I. Hong), at the ACM Symposium on User Interface Software and Technology (UIST 2019), which was held Oct. 20-23 in New Orleans.

Doing research in an environment like Georgia Tech’s, however, where entrepreneurship and risk-taking is not only encouraged but required, has a way of making even the clearest of plans veer off in varying unforeseen directions. Thus, out of her lab came Zyrobotics, a technology company that develops educational technologies for children with differing abilities.

For the past six years, Zyrobotics has developed personalized technologies that stimulate social, cognitive, and motor skill development using fun and educational applications. Now, there are five products, three hardware and two software. The software comprises about 15 different programs in math, robot, and coding education. There have been over 600,000 downloads and about 80 distributors using or distributing the products in clinics and school systems.

“As researchers, we’re not only concerned with development,” said Howard, now the Chair of Georgia Tech’s School of Interactive Computing (IC). “We want to know the impact. What Zyrobotics has done is allowed the research we were doing in the lab to touch so many more people than we otherwise would have done.”

A proof of concept

It started as the work of one of her graduate students in ECE. Hae Won Park was finishing up her Ph.D. when she came to a bit of a crossroads. Trying to decide whether to pursue a career in academia or to, perhaps, go into industry, she looked to Howard for some guidance.

“There was an opportunity with the National Science Foundation I-Corps grant where you have to write a proposal, put your ideas down, defend to a program manager, et cetera,” Howard said. “It seemed like a good program that would allow her to experience all of these aspects in a low-risk way. If it didn’t work out, oh well.”

Park’s research examined methods for utilizing touchscreen interfaces for accessible human-robot interaction. It was a project called TabAccess, an assistive technology that provides alternative switch inputs to control smartphones and tablets for users with motor impairments.

VIDEO: How does TabAccess work?

Throughout the course of customer discovery, where Park and Howard spoke with varying professionals and potential users, Howard said she realized just how big of a difference the technology could make outside of the lab.

“A year later, it was enough of a concept,” Howard said. “It looked like we could design something that made sense. The company was founded, and then it went off and did its own thing.”

A broader impact

It was the impact that led Howard to push forward on the project as a startup. At the time, she had been doing robotics educational STEM camps focused on children with special needs. Students, who had primarily visual and motor impairments, were taught how to code robots.

The camps were successful, but the touch points, as Howard called them, were relatively few.

“The touch points were just the kids I was able to recruit along with my clinical collaborator,” she said. “My touch point was: If I show up, I touched. If I didn’t, there was nothing going on. Whereas, in customer discovery, you weren’t necessarily speaking with the people you were impacting – the kids – but you were speaking with the teachers who interact with kids.”

Suddenly, the impact in her mind shifted from the 1-to-1 relationship of STEM camps to 1-to-100, 1-to-1,000, and more.

“My workshop on a good day had maybe 10 kids,” she said. “I did these in a good year maybe twice. So, maybe like 20 kids in a year. You can’t possibly do what we’re doing now without Zyrobotics.”

'For students, (entrepreneurship) is a no-brainer'

Howard said it’s this mindset that sets Georgia Tech apart.

“Students have always thought about the impact of what they’re doing,” she said. “Socially-responsible engineering. That’s always been the core mission. Being an entrepreneur has this aspect of knowing the exact problems you want to attack, versus maybe going into industry and working on someone else’s.”

It’s important, she said, that academics continue to have a place in the technological market. If left to major tech conglomerates, we up against group think and hesitating to take necessary risks.

“For students, it’s a no-brainer to engage in some entrepreneurial pursuit,” she said. “That mindset of thinking about problems and impacts allows you to view it differently.

“We need to be willing to make mistakes. The probability is that your startup will fail. But students understand that and still do it. We need to get rid of that fear of failure, or else we’ll never make significant change.”

But online convenience can come at a cost to building social connections. The lack of human interaction and support can be a direct cause for high dropout rates in web courses.

To directly address social barriers in virtual classes, an artificially intelligent system from Georgia Tech has been designed to connect online students quickly to their peers. It is being deployed in the institute’s Online Master of Science in Computer Science program (OMSCS) as well as two campus classes fall semester 2019.

Using the Jill Watson AI framework, the social agent is intended ultimately to help support students from different walks of life adapt more quickly to rigorous course requirements and foster a community where students can build their own support structures.

“In previous semesters, we had what we called an introduction agent that responded to student introductions and greeted students. Now we have a more fully realized social AI agent that can help students connect virtually and in real life,” said Ida Camacho, the lead engineer for the redesigned AI.

Encouraging social interactions among students using the Jill social agent demanded Camacho rethink the AI’s construct. Questions of privacy came up early on, and researchers found out from testing that if the social agent was too personal, students might get distracting playing with it.

We wanted students to feel like they are part of the community without giving up their anonymity. - Ida Camacho, Lead AI Designer

Using student introductions in the online forum, researchers prompted students to share personal details in order to help build a model for the agent. Using this unstructured data presented its own challenges, such as when the system encountered certain words, such as Paris, and had to parse out whether it was a location or it was referring to a certain blonde celebrity.

One of Camacho’s insightful designs centered on creating summaries of student information that is viewable by those enrolled.

When students enter the forum and introduce themselves now, the Jill social agent can immediately share top results by percent of classmates based on location, timezone, other courses being taken, and primary interests.

“We wanted students to feel like they are part of the community without giving up their anonymity,” said Camacho. “Increasing student engagement and creating micro-communities are two of our primary goals.”

Students can also choose to join conversations based on any area of interest (location, hobbies, etc.) using the hashtag #ConnectMe, which allows them to see and click on individual student names for those who opt in.

Based on the responses, students have already taken to the new and improved Jill.

“One thing that surprised me was that students started trying to connect IRL, or in real life,” said Camacho. “They wanted to set up study groups and meet each other. This was happening all over the place, like New York City, Austin, and Tokyo.”

Camacho suspected there might be a hunger for more social interactions in online courses, and she is fully committed to delivering the best student experience on this front.

“I envision the social agent being used more than just at the start of classes,” she said. “It’s already creating some social glue, getting students to talk right away so they don’t feel like they’re in this all alone.

“Jill can keep the conversation going, and I’m planning for the AI at the end of the semester to share recommendations by students on courses they’ve taken.”

Camacho in one sense is the ideal person to head the Design & Intelligence Lab’s new Jill social agent initiative. The Fresno, Calif., resident is a recent alumna of the OMSCS program and knows how important it was to stay engaged with her peers.

“I feel like if I hadn’t met anyone I might not have been as successful. My community-building started when others invited me into their study groups and I became a TA.”

She jokes that when she was a student, she enrolled in the program’s Knowledge-Based AI course – where the Jill Watson virtual TA was deployed using a pseudonym – to see if she could pick out the AI amongst the human TAs helping students in the online forums.

“I guessed wrong,” she said laughing. “I ended up thinking that it was the head TA. He was online all the time answering student questions. How can someone be online that much?”

Jill Watson being indistinguishable from its human counterparts might be taken as a good sign by the lab’s researchers as they continue to build the future of AI and help students from around the world succeed in pursuing online learning.

The virtual ecological research assistant, known as VERA, was developed at Georgia Tech and enables students to perform virtual experiments to explain existing ecological systems or to predict possible outcomes based on variables they input into the tool.

Getting to Know VERA

“People using VERA have access to the EOL and can test a hypothesis using countless organisms, make as many changes to variables as they want, and study the effects on any ecosystem through real-time modeling,” said Sungeun An, human-centered computing Ph.D. student and lead developer of the AI system.

“This is a unique opportunity that doesn’t exist anywhere else.”

Although the EOL has extensive data entries for more than two million species, An says that VERA has an intuitive user interface and design that is relatively easy to use.

“Students don’t need extensive scientific knowledge or programming and math skills to use VERA. They can build a conceptual model with simple visual cues on the computer screen, such as dragging elements or selecting input options,” said An.

Combining the Strength of Two AIs

However, to get the most out of VERA, An says there can be a learning curve.

To flatten the curve and help students optimize their experience with VERA, An and her fellow researchers turned to Jill Watson, the famed AI-enabled virtual teaching assistant (TA) that premiered in 2016 supporting Georgia Tech’s online Master of Science in Computer Science (OMSCS) program.

Jill Watson answers student questions about VERA via the collaborative messaging app, Slack. These range from technical questions about the tool – “How do I add a new project” – to subject matter questions – “What is consumption rate?”

“Leveraging the Jill Watson virtual TA and VERA together is a powerful demonstration of how to scale technology to serve more populations and provide access to the world’s scientific knowledge,” said Ashok Goel, professor of Interactive Computing and director of the Design & Intelligence Lab, which created both AI agents.

Combining the strength of the two AI agents, said Goel, is part of an intentional approach to rethinking instructional design for online learning.

“VERA is a significant advancement for artificial intelligence in science education and meant to be used anywhere by anyone interested in science exploration, so making it as accessible as possible is key to the system’s adoption,” Goel said.

Students and others using VERA – it’s also publicly available and linked on the Smithsonian’s EOL homepage ­– can learn more through a video series produced by Georgia Tech.

The videos demonstrate VERA’s capabilities using kudzu growth in the southeastern United States as an example. The videos are co-hosted by Emily Weigel, School of Biological Sciences instructor for the biology course using VERA, and College of Computing faculty member Spencer Rugaber.

VERA research is funded by a grant from the National Science Foundation, #NSF-1636848.

For more information about Georgia Tech's emPRIZE, contact Joshua Preston, research communications manager.

The International Conference on Computer Vision (ICCV) takes place from Oct. 27 to Nov. 2 and brings together researchers from Georgia Tech and around the world to discuss recent breakthroughs and research in the field. Researchers in the Machine Learning Center at Georgia Tech (ML@GT) have ten accepted papers in the conference.

School of Interactive Computing (IC) and ML@GT associate professor Devi Parikh leads with seven research papers. Her work spans from using artificial intelligence (AI) to help people make more stylish outfit choices to embodied visual recognition.

IC assistant professor Judy Hoffman and professor James Rehg are 2019 area chairs.

“As the computer vision field continues to expand and create novel ideas, conferences like ICCV become increasingly important. There was a lot of impressive work submitted to the conference this year. With computer vision being one of ML@GT’s strongest areas, I’m thrilled to see the center’s presence in this premier conference,” said Hoffman.

Other work from Georgia Tech includes papers on lessening the need for additional annotation in videos, making vision and language models more grounded, and agents learning to move to better perceive objects.

"Having a paper accepted, especially as an oral presentation, especially in a top conference gives me lots of confidence and encouragement for my Ph.D. research. I can't wait to attend ICCV to share my work, talk with other talented people, and learn other interesting topics in both academic and industrial areas," said Min-Hung Chen, a sixth-year electrical and computer engineering Ph.D. student.

Organized by IEEE, ICCV is one of the premier international computer vision conferences and will take place at the COEX Convention Center in Seoul, South Korea.

For more information on ML@GT’s involvement with the conference, visit http://bit.ly/339BYaS

A new study from Georgia Tech shows how advanced computing tech worn by firefighters impacts the nature of work for emergency responders, and how front-line firefighters and their commanders view its usefulness.

Researchers sought to establish the effects of a wearable device used by firefighters on the job as well as how companies might better design devices for more physical types of labor.

“For emergency responders, work is carried out with the pressure of someone’s life or property on the line based on how well the job is done,” said Alyssa Rumsey, PhD student in Digital Media, who conducted the study. “Firefighting is unlike office work, which is typically examined in human-computer interaction research; simply put, the stakes are higher.”

The findings showed that the wearable device went through many iterations – originally it was a heads-up display (HUD) that projected situational information onto a firefighter’s mask. This proved too distracting, so then only firefighter’s biometric information was displayed.

Ultimately the tool became a wrist-worn device that gave the biometric information to the on-scene commander, rather than the front-line firefighters, to see the vital signs of the emergency responders in real-time through a web interface.

“Firefighters reportedly didn’t have time to react to the information during the fire suppression because their attention was focused solely on the task at hand,” Rumsey said.

The final device could be used to remotely measure the physical conditions of firefighters in the field. The device was not tested in any live fires but was used extensively on “job duty courses” at two Georgia fire departments where routine training exercises by firefighters in full gear took place. Exercises included a series of obstacles such as ladder runs, hose drags, tire pulls, and crawling.

The device allowed managers to identify personnel who were pushing too hard and those who weren’t pushing hard enough based on heartrate spikes and overall activity during exercises. Supervisors could then call out those individuals and issue them commands either in person or via radio communications.

With overexertion and stress accounting for more than 50 percent of all firefighter deaths, the real-time biometric data potentially allows commanders to assess who on their teams is in trouble and be able to pull them from a scene.

Rumsey found that this clashed with some firefighters’ sense of identity, one that valued getting the job done and putting the safety of others above their own. Some participants rejected the idea of using the wearable tech in a real emergency fire.

As one participant put it: “It would be a way for the chief to see someone’s heart rate, and be like, ‘Yeah I know this person, he's gung ho, he's not gonna quit. It's time to pull him out.’”

Firefighters discussed how overreliance on technology sometimes makes even seasoned pros forget the basics. One participant vividly described an incident where firefighters relied on a thermal imaging camera to view temperatures in a room and assess its safety:

“He scans the room, sees the floors there. Takes off walking through the middle of the living room floor and never went back to the basics of ‘sounding the floor.’ Him and his other two guys with him, fell through the floor and burned to death.”

While the wearable device was viewed favorably in training as a way to improve physical fitness, reduce obesity, and generate comradery, how it might be implemented beyond that was not clear.

“We’ve seen that introducing this technology impacts identity, power dynamics, and organizational structures within fire departments,” Rumsey said. “Smart technology in safety-critical settings, such as fire scenarios, can exacerbate risks rather than lessen them. Understanding an organization is key to implementing technology in some of these more physically demanding jobs.”

Rumsey and co-investigator Chris LeDantec, assistant professor of digital media, published their findings in the paper Clearing the Smoke: The Changing Identities and Work in Firefighting in the Proceedings of the Association for Computing Machinery’s 2019 Conference on Designing Interactive Systems.

A new four-year, $1.2 million grant from the National Science Foundation to a team of researchers from Georgia Tech, Emory University, and the University of Rochester will help bridge this gap by funding the development of a computational assessment toolkit for PTSD patients and clinicians, called PE Collective Sensing System (PECSS). PECSS, which will sit atop the PE Coach App developed by the Veterans Health Administration and the Department of Defense, will aim to improve current treatment practices and increase the number of veterans who seek treatment.

“PECSS will allow clinicians to use automated predictions to deliver better therapeutic treatment and individualized feedback, and patients to better understand the progress they are making and how to improve their exposure exercises,” said Rosa Arriaga, a Senior Research Scientist in Georgia Tech’s School of Interactive Computing and the principal investigator on the project.

[THE INTERACTION HOUR PODCAST: IS TECHNOLOGY A GAME CHANGER FOR CARE OF PTSD PATIENTS?, FEATURING DR. ROSA ARRIAGA]

Currently, the most common and empirically-supported treatment for PTSD is “prolonged exposure” (PE) therapy. The treatment consists of imaginal exposure, where patients imagine themselves and narrate their traumatic event, and in-vivo exposure to real-world stimuli in safe but challenging environments.

There are, however, challenges in data collection and extraction, which is often subjective and narrow. This project will address those challenges by developing a novel, user-tailored sensing system that can record and transfer information from exercises, continuously monitoring patients and clinicians

“Clinicians are in urgent need of methods, tools, and data to efficiently track, assess, and respond to mental health needs throughout the treatment process,” Arriaga said.

The project will involve insights from experts in multiple fields – ubiquitous computing, human-computer interaction, applied machine learning, psychology, and more. When complete, the system will be deployed at the Emory Healthcare Veterans Program, a nationally-renowned initiative that treats members of the military suffering from PTSD.

“The Trauma and Anxiety Recovery Program that includes the Emory Veterans Program has been on the cutting edge in using technology to advance the care of people suffering with anxiety since it was founded by Dr. Barbara Rothbaum over 25 years ago,” said Sheila Rauch, an associate professor in Emory’s Department of Psychiatry and Behavioral Sciences and a co-principal investigator on the project.

“As a team of international experts in PTSD treatment, we integrate technology to speed response to treatment and help patients to visualize the changes as they respond to care. Our aim is to use this real-time data to find tune practice for the individual patient and learn across patients how we can improve care.”

“Mental health clinicians and their patients are in urgent need of 21^st-centural methods, tools, and objective data to optimize therapy,” added Emory Assistant Professor Andrew Sherrill, another co-principal investigator. “This partnership will bring together innovators in HCI and evidence-based psychotherapy to transform mental health care for PTSD patients.”

This grant is provided under the NSF Smart and Connect Health Funding Program in its Division of Information and Intelligent Systems.

Now, the largely techno-utopian view has been adjusted, and moderation of content has become the norm. The question is: how can you moderate, while also maintaining the promise of free speech? Also, how can you avoid discouraging posters whose content was moderated or removed while encouraging them to remain a part of the public discourse?

These are just a few of the questions being posed and pursued by Shagun Jhaver, a Ph.D. student in Georgia Tech’s School of Interactive Computing (IC), whose papers at the upcoming Computer-Supported Cooperative Work and Social Computing (CSCW) conference provide some context and, perhaps, solutions.

Fairness, accountability, and transparency

Jhaver is a computer scientist at heart. He earned his bachelor’s degree in India in electrical engineering and then studied computer science for his master’s at the University of Texas at Dallas. Like most in IC, though, his primary focus is on humans.

“One of the main attractions to our School was that, although it is a computer science school, I am able to do interviews and surveys with people,” Jhaver explained. “What good are technological developments if they don’t work for humans, if they don’t improve society? In order to understand the interactions between technology and society, I wanted to develop a mixed-methods background, and the resources and faculty here are perfect for that.”

One of his first projects as a graduate student was investigating communication on social media around the Black Lives Matter movement.

“I wanted to understand the emergent collective participation around this movement and what people were feeling on the ground in the moment,” he said. “That’s how I entered this area of social computing.”

Social computing is an area of computer science that focuses on the intersection between social behavior and computational systems. Integral to Jhaver’s study was how social media and the data gathered within those systems reflected what was happening within society as a whole.

There may be no more adequate reflection of this phenomena than on Reddit and Twitter, two communities his research has looked at. At CSCW, he’ll present a handful of studies that have examined the topic of content moderation. One of the papers, titled Does Transparency in Moderation Really Matter?: User Behavior After Content Removal Explanations on Reddit, earned a best paper award. Another, titled Did You Suspect the Post Would be Removed?: Understanding User Reactions to Content Removals on Reddit, earned an honorable mention.

How, he wonders, do you develop good moderation practices that enforce community rules while also maintaining the free expression of ideas? And, what practices improve how posters feel about their moderated content and encourage them to continue participating in these forums?

“Content moderation is more nuanced than just editing and removing content,” Jhaver said. “It’s about the overall experience of the user and the community and how they interact.”

His research came to a few conclusions:

One, fairness matters; two, accountability is important; three, the platforms should be transparent in their decisions. From the perspective of end users, that means that rules are clear and easy to follow, and when the post is removed they are notified and given a clear explanation of why. If they appeal, they are given an appropriate response.

But there are multiple stakeholders involved in the exchange, and who determines what is fair?

“These Reddit moderators are volunteers,” Jhaver said. “Is it fair for us to expect them to take on these increased responsibilities for providing explanations?”

In other words, these issues are much more nuanced than they would seem to many casual participants. Amy Bruckman, a professor in IC and Jhaver’s co-advisor (with IC adjunct faculty Eric Gilbert), said she can’t think of other research that has examined this aspect of social communities.

“I don’t think it has been studied – okay, your content was just removed, so how do you feel about that?” she said. “Taking that other side of it is unique.”

Giving everyone a voice

So, why do these explanations even matter? Why not just remove bad content and move on?

“But free speech is interesting,” Jhaver said. “There’s this dichotomy where if you are free to harass certain people over their race, gender, or other aspects of identity, then you are preventing them from having the voice to speak their truth. So, you are infringing on their freedom of speech. That’s why there’s this need.”

Whatever the case, these issues are not going away. Methods of communication will continue to change over time, particularly as technology continues to advance. But, Jhaver said, these conversations aren’t anything new either.

“These are age old problems,” he said. “Harassment, free speech, suppression of free speech. These topics have always been discussed, but the internet has changed the way we see them and changed how they manifest themselves.

“I want my research to help minorities and other vulnerable groups have a greater voice in society,” Jhaver said. “I want to contribute to the design of more equitable, inclusive, and participatory technologies.”

“When I got that email, I responded within 30 milliseconds and said I’d have a website up in two days,” recalls Harvey, who is now well into a four-month internship at Google’s Switzerland campus in Zurich.

Making good on his promise helped Harvey secure a spot in the interview process, which spanned several early-morning video calls to Zurich (six hours ahead of Atlanta) and two rounds of vetting over about 45 days.

Harvey is now experiencing firsthand the culture that such a selective hiring process helps cultivate. He jokes about trying not to become too “googley” – an enigmatic term that hints at Google’s sometimes utopian ideals – but he was soon struck with the weight of the responsibility given to him.

“It’s incredibly humbling to be working on Google Flights,” Harvey says, referring to the product that has been his focus. The tool is part of Google Travel, designed to be a comprehensive resource for planning trips.

“Releasing a bad product impacts a lot of people. When designing, I might think of grandparents who want to fly out to visit their grandchildren,” Harvey says. “How do I help them?”

“What I’m designing can either make the experience easier or harder. Imagine making something that will ruin the day for a million grandmothers. I don’t want to do that.”

Harvey’s internship as a UX designer – short for user experience designer – is what Harvey himself wants to make of it. The Google culture that includes 24/7 free food, nap rooms, and flex schedules (just a few of the perks) is designed to “let you maximize your full potential,” as Harvey puts it.

UX designers, as the name implies, often focus on how to design an experience that differentiates a product from its competitors. Harvey approaches his work by starting from a place of empathy and figuring out how he would respond to a product. He then evaluates evidence-based designs to quantify what works, and finally, he sets out on the long, chaotic journey to build something truly special.

There are no shortcuts in the process, especially not at Google. A glimpse into Harvey’s experience shows this.

“I listen – and I listen hard – to the user researchers and product experts. I turn off the part of my brain where I want to talk over people because I think I have something brilliant to say. I switch off my ego.”

Analyzing Google’s unmatched volume of user data is helping Harvey to identify some of the most vexing problems in online flight planning. He’ll let the information he gathers marinate on his brain for a long time, then start sketching out on paper as many solutions as possible – literally anything that might solve the given challenge.

“Three percent of the ideas are gonna make it out of the furnace,” he jokes, referring to the process where team members kindly discard the concepts that don’t pass muster.

What survives undergoes even closer scrutiny, and the intensity of the process leaves only those product designs that might work well as part of a sprawling Google ecosystem operating around the clock.

Harvey still marvels that a team of some of the most talented people he’s ever met – working together on the same challenges – doesn’t create a toxic culture of competing alphas. Rather, it’s the opposite.

“This place is conducive to making some awesome stuff and not making people feel small,” he said. “My favorite part of being here is being pushed every day and striving to be a contributing member. I’d recommend coming to Google just for the growth potential.”

Harvey mentions the benefit of Georgia Tech’s MS-HCI program, the guidance of program director Dick Henneman, and how these both prepared him for Google. Taking the summer job delayed graduation for him, but Harvey is OK with this, realizing that this rare opportunity will shape the rest of his career.

In treating mental illness, clinicians are often forced into a trial-and-error approach to prescribing medication to patients. Each patient may react differently – oftentimes with negative outcomes – to drugs that have been matched with conditions based on incomplete and potentially biased data from clinical trials.

“In most non-mental health treatment where particular symptoms like a fever or chronic pain might indicate a specific physical condition, there exists a more definitive matching approach to prescription,” said Koustuv Saha, an IC Ph.D. student who led the study. “In psychiatric care, that matching approach is unknown.”

Patients taking the wrong medication could experience increased depression or anxiety, suicidal ideation, or other symptoms like fluctuations in sleep and weight. In many cases, they are forced to return to their clinician for a change in medication or, in worse cases, may lose trust in the medication entirely and stop using it.

“Considering that five of the top 50 drugs sold in the United States are psychiatric medications, it’s extremely important to understand how they actually work on individuals,” Saha said.

In the past, clinical trials have taken a disease-centered approach that attempts to prescribe specific medications with psychiatric symptoms, neglecting those psychoactive effects of the drug. Trials are conducted for smaller cohorts over shorter periods of time, eliminate some individuals who experience more extreme symptoms, and are often biased, being conducted by the drug companies themselves.

Adopting a “patient-centered” model that considers individual outcomes for patients using a specific medication, this study leveraged longitudinal and large-scale social media data to achieve a form of digital-based matching of patients to medications.

The researchers collected a list of medications approved by the Food and Drug Administration, then collected Tweets that mentioned these medications between 2015-16. From that, they collected over 600,000 Tweets that identified users of a specific medication. Interestingly enough, their data matched the top four prescription psychiatric medications in that period: Sertraline (Zoloft), Escitalopram (Lexapro), Fluoxetine (Prozac), and Duloxetine (Cymbalta).

Using a control group of random Twitter users that did not take the medication and building on prior work that showed the ability for language found in social posts to predict mental health conditions, researchers could match specific medications with their outcomes, positive or negative, after use.

The findings indicated that Selective Serotonin Reuptake Inhibitors (Sertraline, Escitalopram, Fluoxetine) – three of the most popular prescription medications – are actually associated with worsening symptoms. Tricyclic Antidepressants like Dosulepin, Imipramine, and Clomipramine, by comparison, were more associated with improving conditions.

“Clinically, our findings reveal signals of the most common effects of the psychiatric medications over a large population, with the potential for improved characterization of their occurrence,” Saha writes in the paper. “Technologically, we show the potential of novel technologies in digital therapeutics.”

This research, Saha said, exists as a proof of concept to show levels of a specific condition – before and after medication use – using digital data. He stressed it is not a replacement for clinical care, only a way to help augment treatment using additional available data.

The work was presented at the 13^th International AAAI Conference on Web and Social Media in a paper titled A Social Media Study on the Effects of Psychiatric Medication Use (Koustuv Saha, Benjamin Sugar, John Torous, Bruno Abrahao, Emre Kiciman, Munmun De Choudhury). It was awarded Outstanding Study Design Paper at the conference. It is funded in part by a grant from the National Institutes of Health.

In treating mental illness, clinicians are often forced into a trial-and-error approach to prescribing medication to patients. Each patient may react differently – oftentimes with negative outcomes – to drugs that have been matched with conditions based on incomplete and potentially biased data from clinical trials.

“In most non-mental health treatment where particular symptoms like a fever or chronic pain might indicate a specific physical condition, there exists a more definitive matching approach to prescription,” said Koustuv Saha, an IC Ph.D. student who led the study. “In psychiatric care, that matching approach is unknown.”

Patients taking the wrong medication could experience increased depression or anxiety, suicidal ideation, or other symptoms like fluctuations in sleep and weight. In many cases, they are forced to return to their clinician for a change in medication or, in worse cases, may lose trust in the medication entirely and stop using it.

“Considering that five of the top 50 drugs sold in the United States are psychiatric medications, it’s extremely important to understand how they actually work on individuals,” Saha said.

In the past, clinical trials have taken a disease-centered approach that attempts to prescribe specific medications with psychiatric symptoms, neglecting those psychoactive effects of the drug. Trials are conducted for smaller cohorts over shorter periods of time, eliminate some individuals who experience more extreme symptoms, and are often biased, being conducted by the drug companies themselves.

Adopting a “patient-centered” model that considers individual outcomes for patients using a specific medication, this study leveraged longitudinal and large-scale social media data to achieve a form of digital-based matching of patients to medications.

The researchers collected a list of medications approved by the Food and Drug Administration, then collected Tweets that mentioned these medications between 2015-16. From that, they collected over 600,000 Tweets that identified users of a specific medication. Interestingly enough, their data matched the top four prescription psychiatric medications in that period: Sertraline (Zoloft), Escitalopram (Lexapro), Fluoxetine (Prozac), and Duloxetine (Cymbalta).

Using a control group of random Twitter users that did not take the medication and building on prior work that showed the ability for language found in social posts to predict mental health conditions, researchers could match specific medications with their outcomes, positive or negative, after use.

The findings indicated that Selective Serotonin Reuptake Inhibitors (Sertraline, Escitalopram, Fluoxetine) – three of the most popular prescription medications – are actually associated with worsening symptoms. Tricyclic Antidepressants like Dosulepin, Imipramine, and Clomipramine, by comparison, were more associated with improving conditions.

“Clinically, our findings reveal signals of the most common effects of the psychiatric medications over a large population, with the potential for improved characterization of their occurrence,” Saha writes in the paper. “Technologically, we show the potential of novel technologies in digital therapeutics.”

This research, Saha said, exists as a proof of concept to show levels of a specific condition – before and after medication use – using digital data. He stressed it is not a replacement for clinical care, only a way to help augment treatment using additional available data.

The work was presented at the 13^th International AAAI Conference on Web and Social Media in a paper titled A Social Media Study on the Effects of Psychiatric Medication Use (Koustuv Saha, Benjamin Sugar, John Torous, Bruno Abrahao, Emre Kiciman, Munmun De Choudhury). It was awarded Outstanding Study Design Paper at the conference. It is funded in part by a grant from the National Institutes of Health.

Using a novel capability to reason about shape, function, and attachment of unrelated parts, researchers have for the first time successfully trained an intelligent agent to create basic tools by combining objects.

The breakthrough comes from Georgia Tech’s Robot Autonomy and Interactive Learning (RAIL) research lab and is a significant step toward enabling intelligent agents to devise more advanced tools that could prove useful in hazardous – and potentially life-threatening – environments.

The concept may sound familiar. It’s called “MacGyvering,” based off the name of a 1980s — and recently rebooted — television series. In the series, the title character is known for his unconventional problem-solving ability using differing resources available to him.

For years, computer scientists and others have been working to provide robots with similar capabilities. In their new robot-MacGyvering work, RAIL lab researchers led by Associate Professor Sonia Chernova used as a starting point a robotics technique previously developed by former Georgia Tech Professor Mike Stilman.

In this latest work, a robot trained using the team’s novel approach is given a set of optional parts and told to make a specific tool. Much like its human counterparts, the robot first examines the shapes of each part and how one might be attached to another.

Using machine learning, the robot is trained to match form to function – which object shapes facilitate a particular outcome – from numerous examples of everyday objects. For example, by learning that the concavity of bowls enables them to hold liquids, it makes use of this knowledge when constructing a spoon. Similarly, the robots were taught how to attach objects together from examples of materials that could be pierced or grasped.

In the study, researchers successfully created hammers, spatulas, scoops, squeegees, and screwdrivers.

“The screwdriver was particularly interesting because the robot combined pliers and a coin,” said Lakshmi Nair, a Ph.D. student in the School of Interactive Computing and one of the researchers on the project. “It reasoned that the pliers were able to grasp something and said that the coin sort of matched the head of a screwdriver. Put them together, and it creates an effective tool.”

Currently, the robot is limited only to the shape and attachment. It cannot yet effectively reason about particular material properties, a crucial step in advancing to a real-world scenario.

[RELATED: Robot Able to Instantly Identify Household Materials Using Near-Infrared Light]

“People reason that hammers are sturdy and strong, so you wouldn’t make a hammer out of foam blocks,” Nair said. “We want to reach that level of reasoning in our work, which is something we’re working on now.”

The inspiration for the work comes from the popular story of Apollo 13, the doomed seventh crewed flight of the Apollo space program. After an oxygen tank in the ship’s service module exploded two days into the mission, crew members were forced to make makeshift modifications to the carbon dioxide removal system.

Despite a dangerously tight window of time and extremely high tension among all aboard and at mission control, the rescue proved successful. Nair and collaborators hope this research will prove foundational to future robotics technology that could reason faster and without the burden of stress.

“They were able to make this filter, but the solution took a long time to come up with,” Nair said. “We want to make robots that can assist humans in these kinds of scenarios to take the pressure off of them to come up with innovative solutions and potentially save their lives.”

This work was presented at the 2019 Robotics: Science and Systems conference in a paper titled Autonomous Tool Construction Using Part Shape and Attachment Prediction (Lakshmi Nair, Nithin Shrivatsav, Zackory Erickson, Sonia Chernova). It is supported in part by grants from the National Science Foundation and the Office of Naval Research.

Since 2013, undergraduates from colleges across the country come to campus for the 10-week program, where they learn how to use data science to tackle civic problems. This year, CDS paired with Smart Cities’ Smart Communities, an initiative that integrates technology-based research with a community’s goals.

This year’s CDS projects were:

• Smart Sea Level Tools for Emergency Planning and Response, in which students found a better way to conduct maintenance for 30 smart sea level sensors that are part of a program run by School of Computer Science and Institute for People and Technology Senior Research Scientist Russell Clark in Savannah, Georgia.
• Albany Housing Data Initiative, where students cleaned city data from disparate sources and created a database to help the city of Albany, Georgia, understanding the effect of programs to reduce energy costs.
• Connected Vehicle Technology Master Plan, in which students analyzed data to better handle the flow of traffic in Gwinnett county for emergency vehicles.

The program’s co-director and SCS Professor Ellen Zegura believes students connected to these projects more because of their real-world application.

“It’s a pleasure to watch the work progress from the early first days to getting to see how much you all have learned and how much you all understand the context of the projects you’re doing,” she said during the finale ceremony held in the Technology Square Research Building.

“It’s not just that you built a database, but here’s what a sensor looks like and here’s how it can go wrong.”

The students agree. Angela Lau, a rising sophomore at Cornell Unviersity, wanted an internship that coud help the community.

“I was really interested in this program because of the local applicability of the projects,” she said. “It surprised me how real it was and how we could help a community over a few weeks.”

Working with real data also presented unique learning experiences that students wouldn’t normally encounter in a classroom setting.

“There were a lot of challenges working with a real data,” said Kutub Gandhi, a rising senior at Rice University. “Our entire project was figuring out what was wrong with the data collected from sea level sensors.”

For many students, this was their first time learning data science skills that they can now use throughout their career.

“I had heard of data visualization but didn’t know much about it,” said David Li, a rising senior at Stony Brook University. “But by the end I realized, ‘Wow I learned this and I never knew I could do this before!’”

Through two studies with more than 100 game hobbyists and practicing game developers, the Georgia Tech team found that people varied significantly in how they used the AI.

“We did not explicitly structure any roles into our machine learning models, but we still found that users naturally projected different roles onto the same AI and took corresponding roles,” said Matthew Guzdial, Ph.D. student in computer science and lead researcher.

According to researchers, after refining the machine learning model, the AI agent was capable of picking up on users’ preferences for level structures. A majority of game developers reported that they would use the AI co-designer in the software, which was developed in Unity.

Researchers observed four major categories of roles that people assigned their virtual partners.

Some participants viewed the AI as a friend. One participant prompted the AI to begin the level design, forfeiting her own turn and stating, “Let’s see what my friend comes up with.”

Some participants wanted an equal design partner (collaborator), others seemed to expect the AI to adhere to their specific design beliefs or instructions (student), and some designers followed the AI’s lead or expected to be evaluated on their design (manager).

“Human designers in the study demonstrated a willingness to adapt their own design practices to the AI, sometimes as a means of attempting to determine how best to interact with it,” said Guzdial.

Conversely, every participant had at least one interaction where the AI adapted to the human designs. For some, this was the exception rather than the rule. “The [AI] agent placed objects fairly arbitrarily, in places where it didn’t really affect gameplay, just looked weird,” said another participating professional designer.

The AI agent embedded in the game design software was trained on implicit feedback from the user. If a user kept the AI’s game level additions, the AI received a “reward,” and if the user removed them a “penalty” was given to the AI. The AI was not allowed to remove human-generated elements.

One designer said, “It was nice to be surprised by the AI partner. It prompted conversation/discussion in my head.” Another said, “I was running out of ideas, then prompted the AI for help, and I said, ‘Oh yeah I forgot about these things!’”

Despite mostly positive feedback, not everyone found the tool to be consistently valuable. As one participant put it, “I could see using this tool as a way to give myself inspiration. But, if I had more specific goals in mind... I would have found it more inhibiting than useful.”

Guzdial says MorAI Maker is intended as a design aide, not as a replacement for designers.

“The AI system is developed in favor of augmenting, not replacing, creative work,” he said.

The full research, Friend, collaborator, student, manager: How design of an AI-driven game level editor affects creators, is published in the 2019 Proceedings of the ACM Conference on Humans Factors in Computing Systems.

The research is based upon work supported by the National Science Foundation under Grant No. IIS-1525967. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

AI-assisted tech is increasingly common, but actions by these autonomous programs are often hard to spot in people’s daily use of devices and online services.

Georgia Tech’s Expressive Machinery Lab has developed exhibitions where the AI agents are front-and-center and people are able to create with them in public spaces. These AIs have included a dance partner, visual storyteller, music maker, and comedic improv performer.

“There are common misconceptions about what AI is, what it is capable of, and how it works,” said Brian Magerko, professor of digital media and director of the Expressive Machinery Lab. “AI systems in public spaces that can engage as active participants in co-creative activities have the potential to serve as avenues for AI literacy. We believe this work pushes these efforts forward considerably.”

The exhibitions involving live interactions between people and AIs – what the researchers call co-creative experiences – have taken place across the country since 2013 at academic conferences, art festivals, museums, and other venues.  

The multi-year endeavor has resulted in a design blueprint developed by the researchers that shows how to build AI experiences for public spaces where audiences or performers can create with an AI partner.

“Museums and other public spaces can serve as alternative venues for AI literacy initiatives, complementing formal education and broadening access to opportunities to interact with and learn about AI by both adults and children who may not have AI devices in their homes or schools,” said Duri Long, human-centered computing Ph.D. student at Georgia Tech and a researcher involved in the work.

Researchers encountered challenges unique to making “creative AIs”, such as how to build systems that engage people with different tastes, AIs that perform over sustained periods of time, and AIs being able to adapt to unpredictable human behavior.

For example, the AI dance partner, known as LuminAI and the oldest of the group, doesn’t have fingers so any naughty hand gestures aren’t processed in the AI’s dance routine.

“Our AI agents are unlike many other AIs, which usually have a specific task to accomplish,” Long said. “Our work involves open-ended co-creative AI installations where there is not a single clear goal or other reward function to optimize the AI’s behavior. Our AIs are meant to create or collaborate with a human counterpart, and that looks different every time.”

While AIs in general often have large databases of sensor data (images, temperature readings, etc.) to improve their understanding of the world, in creative areas such as dance, theater, and other performing arts there is limited data from which AIs can pull.

The researchers overcame this in part by having their AIs learn from human partners in real-time and decide what might be a suitable action. For professional performers, who want a greater degree of control, they could perhaps take turns with the AI partner to have a more structured performance. Conversely, an AI as part of a museum exhibit might guide participants on how to start an activity in order to engage people early on.  

Social interaction was also important to consider and, counter to some technology trends, the researchers discovered that human-to-human interaction could increase as a result of AI involvement.

LuminAI, the dancing AI, prompted a couple to do the salsa, two friends to start a synchronized dance routine, and a group of teenagers to perform in a dance circle.

The comedic AI in the roster, called Robot Improv Circus, allows an audience to watch someone interacting in VR with the AI agent and provide feedback to the person by using voice prompts and gestures to trigger in-game reward systems. This led to several groups of friends encouraging each other to try different actions with the comedic AI.

The research was published in the Proceedings of the Creativity & Cognition Conference 2019. The paper Designing Co-Creative AI for Public Spaces was co-authored by Duri Long, Mikhail Jacob, and Brian Magerko.

Speakers come from industry and academia, giving attendees exposure to problems being solved by both entities. Talks touch on current topics in machine learning and artificial intelligence, applications for technologies, and related insights and experiences. Past speakers have included the likes of Pieter Abbeel, Magic Leap’s Ashwin Swaminathan and Prateek Singhal, Hugo Larochelle, and Manuela Veloso.

“We are proud to be able to bring world-class researchers to our campus to further explore different areas of machine learning and artificial intelligence. Talks like these are important for continuing to grow the ML community and broadening the public’s awareness about where the field is headed. We’re looking forward to another great semester of exciting talks,” said Irfan Essa, director of ML@GT.

The series kicks off on Sept. 4 with Galen Reeves, an assistant professor from Duke University. Talks will be given every other Wednesday at 12:15 p.m. in the Marcus Nanotechnology Building unless otherwise noted. All talks are open to the public.

Fall Seminar Series Schedule

Sept. 4 – Galen Reeves, Duke University

Sept. 18 – Chandrajit Bajaj, University of Texas

Oct. 2 – Vijay Suvramamian, University of Michigan

Oct. 23 – Aleksandra Faust, Google Brain Robotics

Nov. 20 – Speaker to be announced soon

For the most up to date information on the seminar series, visit http://ml.gatech.edu/seminars

Many Georgia Tech faculty, students, and alumni participated among the leadership for the conference.

• Professor Ashok Goel served as the conference’s co-chair;
• Professor Keith McGreggor was the sponsorship chair;
• Wendy Newstetter of the College of Engineering and Center for 21st Century Universities served on the awards committee;
• Georgia Tech alum Jim Davies was co-chair for publication-based talks;
• Georgia Tech alum Maithilee Kunda was co-chair for member abstracts;
• Georgia Tech alum Swaroop Vattam served on the workshops and tutorials committee.

School of Interactive Computing adjunct professors Brian Magerko and Gil Weinberg, primarily of the Ivan Allen College of Liberal Arts and School of Music, respectively, were also part of a panel on Creativity in the Arts.

Ph.D. students Sungeun An presented a poster paper at the conference titled Learning by Doing: Supporting Experimentation in Inquiry-Driven Modeling (Sungeun An, Robert Bates, Jennifer Hammock, Spencer Rugaber, Emily Weigel, Ashok Goel), and Marissa Gonzales another titled Why are Some Online Education Programs Successful: Student Cognition and Success (Marissa Gonzales, Ashok Goel).

For more information about this year’s conference and to stay up-to-date on news about future conferences, visit https://cognitivesciencesociety.org/.

Using near-infrared light, similar to what’s used in TV remotes, the robot can identify common materials used in household objects to better inform its actions. This might allow intelligent machines to understand, for example, the right bowl (paper versus metal) to put in a microwave or how hard to grasp a cup made of glass versus plastic.

To classify materials, the researchers first determined hundreds of light wavelengths reflected from five common materials – paper, wood, plastic, metal, and fabric. With this information, they trained a neural network on 10,000 examples in order to create a machine-learning (ML) model that could be used by a robot to quickly identify a material.

According to the researchers, a robot using their new ML model can identify materials without it first having to touch an object, a useful function for handling potentially fragile items. To do so, the robot holds a small spectrometer near an object to get a quick light measurement, which is then processed to identify the material.

“Robots currently use conventional cameras or haptic sensing - the sense of touch - to estimate a material type,” said Zackory Erickson, the first author on the research paper detailing the new work and Georgia Tech robotics Ph.D. student.

“This is the first time that we know of that spectroscopy and machine learning have been used for material classification in robotics research, and our accuracy is on par with existing methods.”

The team’s new ML model yielded the best results using spectrometer measurements from near-infrared light. In fact, the accuracy was 99.9 percent with the full dataset of 10,000 measurements from 50 objects that the model had been trained on.

“While human eyes typically use three color receptors to see the world, our robot can be thought of as using hundreds of color receptors to recognize materials,” said Charlie Kemp, associate professor in the Wallace H. Coulter Department of Biomedical Engineering at Georgia Tech and Emory University and part of the research team. “Instead of a conventional color camera that measures red, green, and blue light, our robot uses a spectrometer that measures light at hundreds of different wavelengths, some outside of the range of human vision.”

To see how results would compare using only a single light reading from each object, the team also trained the model on just 50 measurements, one from each object. Interestingly, accuracy in identifying the correct material only dropped to 95 percent. When using a spectrometer reading from objects the machine learning model had never seen, the robot still achieved an 81.6 percent success rate.

“Spectroscopy presents a reliable and effective way for robots to estimate materials of household objects,” Erickson said. “We’ve demonstrated how a robot can use near-infrared spectroscopy to infer the materials of everyday objects like cups, bowls, and garments.”

The research is published in the Proceedings of the 2019 International Conference on Robotics and Automation (ICRA) in the paper titled Classification of Household Materials via Spectroscopy co-authored by Zackory Erickson, Nathan Luskey, Sonia Chernova, and Charlie Kemp.

For more Georgia Tech research published at ICRA, as well as the entire conference program, explore this interactive visualization from the GVU Center at Georgia Tech.

Batra is one of three Georgia Tech faculty members this year to earn the award, giving the Institute a total of 18 in its history. The other two awardees in this class are Associate Professor Mark Davenport of the School of Electrical and Computer Engineering and Assistant Professor Matthew McDowell of the School of Materials Science and Engineering.

Along with the Department of Defense, the White House Office of Science and Technology Policy will provide $1 million over the course of five years to support Batra’s research to make artificial intelligence (AI) systems more transparent, explainable, and trustworthy. The award comes as a result of Batra’s selection for a similar early-career award by the Army Research Office Young Investigator Program in 2014.

The research Batra’s lab will pursue with the funding addresses a fundamental challenge in development of AI – their “black-box” nature, the consequent difficulty humans face in identifying why or how AI systems fail, and how to improve upon those technologies. When a self-driving car from a major tech company, for example, suffered its first fatality in 2015, legal and regulatory agencies understandably questioned what went wrong. The challenge at the time was providing a sufficient answer to that question.

“Your response can’t just be, ‘Well, there was this machine learning box in there, and it just didn’t detect the car. We don’t know why,’” Batra said.

Batra’s research aims to create AI systems that can more readily explain what they do and why. This could come in the form of natural language or visual explanations, both of which – computer vision and natural language processing – are central areas of focus in Batra’s lab. The machine could, for example, identify regions in image that provide support for its predictions, potentially assisting a user’s understanding of what the machine can or cannot do.

It’s an important area of study for a few reasons, Batra said. He classifies AI technology into three levels of maturity:

• Level 1 is technology that is in its infancy. It is not near deployment to everyday users, and the consumers of the technology are researchers. The goal for transparency and explanation is to help researchers and developers to understand the failure modes and current limitations, and deduce how to improve the technology – “actionable insight,” as Batra called it.
   
• Level 2 is when things are working to a degree, enough so that the technology can and has been deployed.
  
  “The technology may be mature in a narrow range, and you can ship the product,” Batra said. “Like face detection or fingerprint technology. It’s built into products and being used at agencies, airports, or other places.”
  
  In such cases, you want explanations and interpretability that helps build appropriate trust with users. Users can understand when the system reliably works and when it might not work – face detection in bad lighting, for example – and make efforts to use in a more appropriate setting.
   
• Level 3 is typically a fairly narrow category where the AI is better – sometimes significantly so – than the human. Batra used chess-playing and Go-playing bots as an example. The best chess-playing bots convincingly outperform the best humans and reliably hand a resounding defeat to the average human player.
  
  “We already know bots play much better than humans,” he said. “In such cases, you don’t need to improve the machine and you already trust its skill level. You want the machine to give you explanations not so that you can improve the AI, but so that you can improve yourself.”

Batra envisions scenarios where the techniques his lab develops could assist at all three levels, but the experiments will take place between Levels 1 and 2. They will work in Visual Question Answering, which are agents that answer natural language questions about visual content, and other areas of maturity that may reach the product level in five or more years.

Batra has served as an assistant professor at Georgia Tech since Fall 2016. Visit his website for more information about his research.

Chosen from a shortlist considered by the IJRR Executive Committee, the paper, Continuous-time Gaussian Process Motion Planning via Probabilistic Inference, was recognized for its technical rigor, relevance, and potential for impact in the robotics research community. The research comes from IC Ph.D. students Mustafa Mukadam and Jing Dong, master’s student Xinyan Yan, and advisors Professor Frank Dellaert and Assistant Professor Byron Boots.

This paper introduces a novel formulation of motion planning that treats the problem of finding an efficient, feasible path between two points as probabilistic inference with Gaussian Processes. Motion planning is a hard problem, and state-of-the art sampling-based and trajectory optimization algorithms have well-known drawbacks. The former can effectively find feasible trajectories but often exhibits jerky and redundant motion, and the latter requires a fine approximation of the trajectory to reason about thin obstacles or tight constraints.

In their paper, the team of researchers adopts a continuous-time representation of trajectories, viewing them as functions that map time to robot state. Combing this representation with fast approaches to probabilistic inference, they developed a computationally-efficient gradient-based optimization algorithm called a Gaussian Process Motion Planner that can overcome large computational costs associated with fine discretization, while still maintaining smoothness of motion in the result.

With the award comes a $1,000 prize. Boots attended the Robotics: Science and Systems (RSS) conference in the Freiburg, Germany, this week, where he accepted the award on behalf of his team.

Another paper involving Boots was also awarded a Best Student Paper Award at RSS. Titled An Online Learning Approach to Model Predictive Control, the paper was written by Robotics Ph.D. students Nolan Wagener, Ching-An Cheng, and Jacob Sacks, along with Boots.

It shows that there exists a close connection between model predictive control (MPC), a popular technique for solving dynamic control tasks, and online learning, an abstract theoretical framework for analyzing online decision making. This new perspective provides a foundation for leveraging powerful online learning algorithms to design MPC algorithms. Toward this end, the researchers propose a generic framework for synthesizing new MPC algorithms called Dynamic Mirror Decent Model Predictive Control.

The framework exposes key design choices that can help practitioners easily develop new control algorithms tailored to the challenges of their specific task. The approach is validated by developing new MPC algorithms that consistently match or outperform the state-of-the-art on several tasks including an aggressive driving problem with the goal of racing an autonomous car around a dirt track under computational resource constraints.

A team of Georgia Tech master’s and Ph.D. students, advised by Chernova, won the challenge by successfully assembling three kits with its robot in 39 minutes. It was the only team in the competition to complete the task, with the second-place finisher failing to score a point.

For its victory, the RAIL lab will receive a new mobile manipulation robot from Fetch Robotics, its second. Along with the other robots already in the lab’s possession, the newcomer will provide RAIL researchers new opportunities to pursue multi-robot applications. The prize package also includes items from the event’s co-sponsors EandM Robotics, Schunk, SICK Sensor Intelligence, and The Construct, to go with the $100,000 robot.

[VIDEO::https://youtu.be/G_ur71h4CNQ]

“This is a long-term benefit,” said Chernova, an associate professor in the School of Interactive Computing. “This is one of the most capable mobile manipulation platforms out there, and to now have two of them will enable us to enhance the capabilities of the robot and pursue new lines of research in our lab.”

The allure of a new state-of-the-art robot would be enough to entice most teams to take part in the competition, but for Chernova and her participating students it was more about the opportunity to explore specific applications that aligned with their research initiatives, past and present.

The lab has done past work in grasping, semantic reasoning and mapping, and fault diagnosis, the latter of which has become a focus over the past six months. The competition, Ph.D. student David Kent said, came at a good time because the particular challenges it presented are often in this domain.

“This particular setup was particularly challenging because there was just enough variability where it wasn’t going to work every time,” he said. “There would always be something going wrong, so fault recovery ended up being very central.”

To win the competition, not only did Georgia Tech’s team have to come in first place, it had to do so by scoring at least 14 points. To put that into context, Georgia Tech was the only team in the competition to finish with any points. Teams scored points by successfully collecting items laid out at different stations to assemble three kits. They were awarded eight points for each completed kit. Any kit that was missing a piece, however, resulted in zero points awarded, and any kit with extra pieces would have points deducted.

“If you drop one screw along the way and you don’t notice – which is actually very easy to do – you go away with nothing,” Chernova said. “In the real world, a partial kit is useless.”

Georgia Tech achieved its first 15 points and elected to complete its third kit without official scoring to ensure it wouldn’t drop below the threshold needed to win the robot. Officially the team scored 15, but a completed third kit gave it an unofficial 23 points, after bonuses were added.

“It was a lot of fun to be able to work with my lab on a single project and see it come together,” said Ph.D. student Weiyu Liu, another member of the team. “It was a really great opportunity to try out some of the code we had written and also to see others’ code and other research projects.”

Already, the team has turned the experience into a submitted paper, which they hope to have accepted and published in the future. The focus is on mobile manipulation, which is a particularly challenging aspect of robotics because of what Chernova calls “an explosion of uncertainty.”

“Manipulation in many ways is a solved problem,” she said. “Navigation in many ways is a solved problem. When you put those two solved problems together, though – when you take the wheels and put the arm on it – it becomes a much more challenging problem, one our research will continue to tackle with the aid of Fetch in the coming years.”

Members of the team included: Chernova, Kent, Liu, Siddhartha Banerjee, Angel Daruna, Jonathan Balloch, Abhinav Jain, Akshay Krishnan, Muhammad Asif Rana, Harish Ravichandar, Binit Shah, and Nithin Shrivatsav.

The paper, titled A Social Media Study on the Effects of Psychiatric Medication Use, was presented by IC Ph.D. student Koustuv Saha and included fellow IC Ph.D. student Benjamin Sugar and IC Assistant Professor Munmun De Choudhury. Collaborators from Microsoft Research, Harvard Medical School, and New York University-Shanghai were also involved with the research.

The research addresses a challenge in understanding the effects of psychiatric medications during mental health treatment. While clinical trials help to evaluate effects of the medication, there are challenges in generalizing trials to broader populations. Using a list of common approved and regulated psychiatric medications and a Twitter dataset of 300 million posts from 30,000 individuals, researchers developed machine learning models to first assess effects relating to mood, cognition, depression, anxiety, psychosis, and suicidal ideation and then, based on a score, observe how use of specific drugs are associated with characteristic changes in an individual’s psychopathology.

The goal of this research is a deeper understanding of effects and how to situate those with treatment outcomes.

ICWSM is a forum for researchers from multiple disciplines to come together to share knowledge, discuss ideas, exchange information, and learn about cutting-edge research in diverse fields with the common theme of online social media. This includes social theories, as well as computational algorithms for analyzing social media. In its 13^th year of existence, the conference has become one of the premier venues for computational social science.

Georgia Tech researchers will present 18 research papers at this year’s event. The papers touch on a variety of aspects of machine learning including blended unconditional gradients, clustering with fairness constraints, and observational agents.

School of Interactive Computing assistant professor, Byron Boots is a 2019 area chair. Boots is also the co-organizer of the Real-World Sequential Decision Making: Reinforcement Learning and Beyond workshop and a guest speaker at the Generative Modeling and Model-Based Reasoning for Robotics and AI.

“ICML is globally renowned as one of the best conferences for machine learning research. Year after year, cutting edge research is presented and published and it’s a sign of ML@GT’s strength that Georgia Tech is consistently a top contributor in the accepted papers.” Justin Romberg, School of Electrical and Computer Engineering Schlumberger Professor and associate director of the Machine Learning Center at Georgia Tech (ML@GT).

Hosted June 9 through 15 at the Long Beach Convention and Entertainment Center, ICML is one of the fastest growing conferences in the world. It will bring together over 8,000 participants including entrepreneurs, engineers, graduate students, postdocs, and academic and industrial researchers.

Along with Georgia Tech papers, other accepted papers will include work in closely related fields like statistics, data science, and artificial intelligence, and important application areas like speech recognition, robotics, and machine vision.

For a full list of Georgia Tech’s research papers and more information about Georgia Tech’s presence at the conference, please click here.

Jennifer Mankoff, one of Professor Gregory Abowd’s first of 30 Ph.D graduates in 2001, was inducted into the prestigious CHI Academy this week, and Gillian Hayes (2007), also advised by Abowd, was awarded the Social Impact award.

Mankoff, who was Abowd’s third ever Ph.D. graduate, joined an exclusive community that includes eight Georgia Tech faculty members. Most recently, Professor Amy Bruckman was inducted a year ago.

Mankoff noted the mentors, like Abowd, she had along the way to give her that opportunity. Abowd provided the introduction for Mankoff at the awards ceremony for the CHI academy. She credited her research community and the CHI community for giving her the freedom to pursue the kind of research that she was passionate about.

“The openness to let people be able to work on whatever they’re passionate about and see that has value is something that’s been important to me over the years,” Mankoff said. “More than once, I’ve shifted to another area that I wasn’t working in before and maybe a lot of others weren’t either. It’s a sign of how open the community is.”

Seated at a reunion party for the Abowd “family” – academics who were part of a lineage that began as doctoral students in Abowd’s lab – she noted the importance of having a vibrant community like that.

“We were very lucky to be there at the beginning, helping to form his group and to learn from him and all the energy he brings to this group,” she said. “It’s one of the strongest networks I have at CHI.”

Hayes received her Social Impact award just 12 years after Abowd received his own in 2007. She said it was an especially proud honor to have the distinction of following in the footsteps of her advisor.

“The way he has instilled in us an ethos of being able to give back, being able to bake in community outcomes with our research outcomes and define good, interesting research problems that also really solve real-world problems, and work in partnership with communities,” Hayes said.

Hayes, whose 30-minute talk at the conference focused on ways in which the community needed to do better in thinking about issues of accessibility, access, racial and gender inequities, and much more, said she thought the CHI community was leading the way as a standard-bearer for diversity, inclusion, and service.

“But we still have a long way to go,” she said.

Her talk, she hoped, would be a call to action to the rest of the community.

“This is our time, and we can control our destinies and we can create truly community-driven innovation,” she said.

Under a new $2.7 million grant from the National Institutes of Mental Health (NIMH), she and a team of researchers at Northwell Health will apply that new information in a clinical setting in hopes of improving treatment.

“In our past research, we have gained a number of new insights, but I see an opportunity to influence real world people and outcomes,” De Choudhury said. “Going beyond just academic and empirical findings, how do you take that information and make a difference in people’s lives? What research challenges do such translations pose to the computing domain?”

This grant offers the researchers that opportunity. It will be one of the first in which computing researchers and leading experts in psychiatry research are coming together to influence how treatment can be delivered harnessing patient-contributed data. The grant is funded through a new NIMH program designed to inform and support delivery of high quality mental health services.

The idea is to build machine learning algorithms based on data that mental health patients voluntarily share with the research team, including both clinicians at Northwell Health and researchers in De Choudhury’s lab at Georgia Tech. With these algorithms, they hope to identify different risk markers and symptom changes that appear in social media posts to identify changes and trends in an individual over time.

By combing a number of different social media sources, primarily Facebook and Twitter, they will look at the use of words or patterns of words an individual uses. In mental illnesses like schizophrenia, the main population they will explore, that is important information to know.

“If they are feeling delusional or experiencing paranoia, what is it that they are saying,” De Choudhury said. “We can look at social interactions and see whether they might be feeling isolation, which can have a negative impact on mental health. Nuances of language styles, like the way people use articles or pronouns, can say a lot about their psychological state, as well, which has been shown in our and co-investigator (University of Texas Professor) Jamie Pennebaker’s prior work.”

The population they will focus on comprises younger individuals, largely teens and early 20s, who have had a first episode of schizophrenia. Most will have only recently been diagnosed and admitted to a specialized treatment facility directed by the collaborators on the project in New York. The goal is to use the information gathered in their digital traces to identify risk markers that signal a potential relapse.

“Schizophrenia is a challenging and debilitating illness,” De Choudhury said. “Even people under treatment have a high chance of relapse with negative outcomes on quality of life, productivity, and functioning. Symptoms often come back, and most mental illnesses are only managed, not cured.”

Better management means that the treatment is timely and highly adaptable to the patient’s needs, De Choudhury said. Unfortunately, that’s a challenge because, in clinical settings, there is very little knowledge about a patient’s day-to-day life. Unlike a disease such as cancer, which has an objective screening that can identify its presence and severity, mental illnesses are based on what is reported. These self-reports are often skewed, based on what a patient wants to tell or remembers.

“In some ways, the treatment paradigm right now is not very evidence based,” she said. “But to prevent relapse, it’s important that we try to be as precise and proactive as possible.”

The project will span four years and began on April 15.

Seim, who is advised by IC Professor Thad Starner, was chosen by a committee of leaders from across campus, including the Office of the Dean of Students, various faculty, and commencement officials. The process included an audition of a speech written by Seim.

Having recently defended her dissertation for her degree in Human-Centered Computing, Seim said that she is honored by her selection and opportunity to share the stage with Georgia Tech President Bud Peterson and Vice Provost for Graduate Education and Faculty Affairs Bonnie Ferri.

“I am so thrilled to represent the graduating class, and I can’t wait to share my message about the importance of research,” Seim said. “I love Georgia Tech so much. After all my time here, I still enjoy it as if it were my first day on campus.”

Seim, whose research in wearable computing and passive haptic rehabilitation has been covered extensively by external media, said that in her speech she hopes to help graduates think about a recent realization that she had.

“That is the significant role we have in society’s progress,” she said. “It’s about the formation of knowledge and how Ph.D. students are uniquely trained to evaluate fact and expand what society can achieve. My training in the Human-Centered Computing program actually helped me to begin recognizing this by introducing me to the concept of epistemology.”

Looking back, Seim will remember Georgia Tech as a unique student body and a beautiful campus.

“For me, I have to put special emphasis on the academic community,” she said. “The faculty made learning a great experience, and as a graduate student I felt like I was really part of a community. The student researchers who I mentor continue to impress me and consistently show curiosity, respect, and dedication. It has been a pleasure working with everyone.”

The Ph.D. commencement ceremony will take place at 9-10:30 a.m. Friday, May 3, at McCamish Pavilion. Ferri will also speak. No tickets are required for the event.

Building upon nearly a decade of research, Georgia Tech’s Child Study Lab, which was established in 2010 by a $10 million grant from the National Science Foundation, and collaborators at Weill Cornell Medical College were awarded last year with a $1.7 million grant from the National Institutes of Health. The grant will help researchers collect new data, using the datasets created over the past decade to develop automated tools that better and more efficiently characterize behaviors that are present and important in typical child development but are often considered to be core, early-emerging markers of autism spectrum disorder (ASD) when absent.

[VIDEO::https://youtu.be/jVldx01ENHM::aVideoStyle]

[RELATED: Using Computer Science to Augment Autism Research at Georgia Tech (VIDEO)]

Psychologists have long understood that there were links between early childhood development and the likelihood of typical language and behavior outcomes throughout life. What they weren’t able to do, however, was to study childhood behavior at a granular level similar to that of a microscope. Given the importance of early detection to inform proper interventions, the tedium of human coding and analysis poses a significant challenge.

“That process is manual and driven by humans specifying what happens in a frame of a video,” said Jim Rehg, a professor in the School of Interactive Computing and the principal investigator on the NIH award. “It takes hours upon hours of data collection and analysis.”

Computing could alter that reality, and this work being done at Georgia Tech is a significant reason why.

“Given enough video, we can model the details of behavior,” Rehg said. “Deep learning, married with the ability to collect the data, allows us to build out how our algorithms work in much the same way computer science has been applied to genetics and imaging to make those more powerful and scalable.”

That has long been the mission of the Child Study Lab, and the latest grant will continue to move the needle forward in autism research at Georgia Tech and beyond. Unlike many other conditions, autism spectrum disorder can’t be found by taking a blood test or viewing images of the brain. Doctors must analyze behavior through developmental screenings and comprehensive diagnostic evaluations.

During screenings, doctors might talk or play with a child to see how they learn, speak, or behave. Do they exhibit typical communicative skills like joint attention, in which two people use gestures or gaze to share their attention with respect to other objects or events? The skills a child demonstrates in these areas are known to be strong indicators of how they will develop throughout childhood and adolescence.

The challenge here is that, given how important it is to detect ASD at an early age and thus tailor interventions and education to meet the child’s specific needs, the manual labor that comes with these screenings and evaluations makes it far less efficient than detection of other developmental challenges. Autism spectrum disorder affects one in 59 children in the United States alone, and not all who are screened are ultimately determined to be one of those individuals.

The need for objective, automated measurements of behavior is clear, and Rehg – along with IC Research Scientist Agata Rozga, Child Study Lab coordinator Audrey Southerland, collaborators at Weill Cornell, and more – are taking steps in that direction.

“For us, the goal is to use these computational capabilities to extract the important key moments and information to give clinicians or psychologists the ability to more easily examine a child’s behavior,” Southerland said. “If we can provide additional details through technology about the quality or coordination of important social and communicative behaviors, we can hopefully provide behavioral experts with the capability of exploring these behaviors in much greater detail than currently possible.”

The first grant from the NSF funded the creation of the Child Study Lab, which has over the years developed an extensive dataset of behaviors in typically developing children. At the time, it was the first large-scale investment in technology that would assist in modeling and sensing behaviors that underlie developmental conditions like autism spectrum disorder. Additional grants have assisted in studies that use computer vision to measure and detect gaze shifts or wearable technology and machine learning to detect and differentiate between types of problem behaviors.

The NIH grant brings all the past research together to compare what the sensory data says in relation to human coding, and how that might ultimately serve to develop reliable, objective, automated tools for measuring early, nonverbal communication behaviors.

“The important thing is for us to make sure that whatever we produce is good enough so that we can actually push it out into the field to people who are specializing in this area,” Southerland said. “We never want to get rid of the human expert in this field, but we want to build technology they can use to augment and streamline their analysis of behavior.”

In addition to the National Institutes of Health and the National Science Foundation, the Child Study Lab has also received funding from the Simons Foundation and has partnered with external entities like the Marcus Autism Center.

Southerland and the Child Study Lab are actively seeking families with young children to participate in this study to further develop their automated tools. Anyone interested in playing a part in this exciting work can visit the lab’s website to learn more.

Miranda Parker was early on in her time as a Ph.D. student in the School of Interactive Computing (IC) when she began her first quantitative study. She wanted to see whether they could model the variables that influence whether a school would or would not adopt computer science (CS) as a class for its students.

Prior to the study, the hypothesis was that variables like median income or enrollment numbers or the population of students who qualify for free and reduced cost lunch programs could be an indicator of whether or not computer science was implemented. Lower income levels, for example, might correlate to schools that just didn’t have the resources to deploy such programs.

Somewhat to Parker’s surprise, the short answer to that question was – no. No, a higher median income didn’t mean more computer science; no, schools with lower free and reduced lunch numbers didn’t teach computer science at a higher rate; no, higher enrollment didn’t necessarily mean more young students yearning to learn how to code.

On the surface, that first study might have felt like a failure. If the goal was to prove that income disparity equated to a disparity in who was gaining exposure to a key part of their education, then it may be fair to describe it as such. However, Parker looks back on that study as a key component of what has guided her research at Georgia Tech ever since.

It wasn’t a failure, she said. It just helped open her eyes to some realities she may not have noticed otherwise.

“Part of me wanted my first study to fail because part of me didn’t want to be able to say, ‘Oh, yes, these three things mean more computer science,’” she said. “Sure, it’s snazzy. It’s easy to put on a Facebook post. But it’s so much more complicated than that. And I’m glad that it’s more complicated than that.”

Over the years, Parker, who studies human-centered computing with a focus on computer science education, has gained a deeper understanding of what might influence a public high school in Georgia to offer computer science education. None of the above items are among them. What she said has shown some correlation is a bit more complex.

“If a school had computer science in 2016, the correlation was that it also had computer science in 2015, 2014, and 2013,” she said.

Okay, but how did it get started in 2013? That’s part of the question her research is trying to uncover.

“That’s an endless cycle,” she explained. “You had it before, now you still have it. But how did you get it to begin with?”

One thing she’s learned, which can be said for a majority of research in IC, is that it comes down to the people. Who is involved with a school and what connections do they have to a particular subject? If a connection has worked in CS in the past or may be passionate about adding that to the school, the results indicate the school is much more likely to employ that subject.

Makes sense, right?

“If a school has someone who can teach computer science and there are parents saying we need to teach computer science, then whether it’s rural or urban or high or low income, it doesn’t matter,” Parker said. “They will have computer science. But if there’s no one there to push them, it’s much less likely.”

It’s not just a person, either. Organizations like Georgia Tech’s Constellations Center for Equity in Computing and the Center for Education Integrating Science, Math, and Computing are also championing K-12 CS educational opportunities.

But, Parker said, being successful is a bit more complicated than just serving CS up to the masses in communities that are unfamiliar with these and other organizations.

“Computer science isn’t the end all, be all,” she said. “If a school is in a more agricultural-based county, that may benefit the school more than a heavy computer science program would. It’s about finding how computer science can most benefit students in different ways for different areas.”

The most encouraging thing about that research, Parker said, was that the failure of her original study showed her one important piece of information.

“You don’t need high income to have computer science,” she said. “It really can be for everyone. That’s an important piece of information to know.”

Parker is aiming to finish her Ph.D. work in the fall and will decide between pursuing a faculty position, which she is leaning toward now, or other opportunities that may present themselves down the road. Former Georgia Tech Professor Mark Guzdial, now a faculty member at the University of Michigan, is Parker’s advisor.

James Hays, an associate professor in the School of Interactive Computing and the Machine Learning Center at Georgia Tech, and researchers from TU Berlin and Universität Regensburg, are looking at how these patterns, known as gaze patterns, can be used to retrieve images from memory so that it’s easier to find that same image – like an adorable dog photo – stashed away in the digital cloud.

Through a controlled lab experiment and a real-world scenario, Hays and his co-authors have developed a matching technique using machine learning to help computers understand what image someone is thinking of, and accurately retrieve it from a computer folder – based solely on eye movements.

Using eye-tracking software in the lab, the researchers recorded the eye movements of 30 participants as they looked at 100 different indoor and outdoor images, ranging from picturesque lighthouse scenes to cozy living rooms. Participants were then asked to look at a blank screen and recall any of the 100 images they just saw.

The researchers also conducted a realistic scenario by putting together a mock museum with 20 posters of various sizes and orientations spread throughout the “museum.” They outfitted each participant with a headset complete with a Pupil mobile eye tracker, complete with two eye cameras, and one front-facing camera. Participants were asked to walk around the museum and look at all of the images, taking however long they liked, and in whatever order they preferred. They took anywhere from a few seconds to over a minute looking at each poster.

After looking at all of the images, participants were asked to look at a blank whiteboard and recall as many of their favorite images as possible, in any order. Participants remembered between 5 and 10 of the total 20 poster images.

The results from both experiments indicated that the gaze patterns of people looking at a photograph contain a unique signature that computers can use to accurately determine the corresponding photo.

Using the data collected from the experiments, researchers created spatial histograms, or heat maps, that could be analyzed by their new machine learning technique to determine which photo someone was thinking about. Hays and Co. also used a Convolutional Neural Network (CNN) to look at the 2,700 collected heat maps.

“The ability to retrieve images using eye movements would be beneficial to those who are disabled or unable to search for images using their hands or voice,” said Hays. “Also, wearable technology is a huge industry right now, and we believe that tracking motion with the eyes would be a natural by-product of that boom.”

In Hays’ previous research, SketchyGAN, people are able to draw (rather than type) what they are looking for to get image search results. But, if images are mislabeled or people can’t draw that well, search results are not useful. Other attempts at image retrieval have included various types of brain scans, but those are often too expensive and impractical for everyday use.

While this new research may prove helpful to people, it does not come without limitations, researchers note. The scalability of the model in part depends on image content and how many images are in the database. The more images the database holds, the more likely it is that several different photos will produce extremely similar gaze patterns.

One proposed workaround to this potential issue is asking people to make more extensive eye movements than they normally would. At the moment, participants are not asked to do anything more intentional or out of the norm when looking at the images. Researchers think that by putting a small amount of effort back on the user, this would help the computer find the correct image.

Another foreseen problem is working with people’s memories. As people’s memories grow weaker with time or age, it will be harder to get a crisp gaze pattern and accurately return the right image. The team plans to explore these potential issues in the future by looking into the influence on memory decay and how it affects image retrieval from long-term memory.

The authors are also looking into combining gaze tracking with a speech interface, as that could be a rich resource for information. No matter which direction they go, the team believes that eye-movement image retrieval is not only possible but also a significant next step to improving human and computer interaction.

One might even say that before long, people will be able to find that favorite dog photo in the blink of an eye.

Further details on this approach to image retrieval can be found in the paper, “The Mental Image Revealed by Gaze Tracking,” which has been accepted at the ACM Conference on Human Factors in Computing Systems (CHI 2019), May 4-9.

Tomlinson is involved in the Graduate Women@CC group, helping to organize events. She has been involved in some capacity with the group since she came to Georgia Tech six years ago. The group is a collection of female graduate students who strive for professional success for their members. They meet once each month for coffee, where they discuss current projects they are working on, and also help to organize various workshops throughout the year.

“It’s great to hear that people think my impact on GradWomen has been a good one, and the work to keep it going has been useful for the greater campus community,” Tomlinson said. “I’m hoping that it will actually help others learn about GradWomen and encourage them to get involved.”

Tomlinson is working toward her Ph.D. in human-centered computing. Her current work is on evaluating effective methods for studying engagement, learning, and transfer for multimodal interactive systems. This includes collaboration on a grant to develop and evaluate accessible auditory displays for PhET Interactive Simulations, a non-profit open educational resource project at the University of Colorado that creates and hosts explorable explanations.

She is advised by Professor Bruce Walker, who is jointly appointed in the School of Interactive Computing and the School of Psychology.

The agent also uses everyday language that non-experts can understand. The explanations, or “rationales” as the researchers call them, are designed to be relatable and inspire trust in those who might be in the workplace with AI machines or interact with them in social situations.

“If the power of AI is to be democratized, it needs to be accessible to anyone regardless of their technical abilities,” said Upol Ehsan, Ph.D. student in the School of Interactive Computing at Georgia Tech and lead researcher.

“As AI pervades all aspects of our lives, there is a distinct need for human-centered AI design that makes black-boxed AI systems explainable to everyday users. Our work takes a formative step toward understanding the role of language-based explanations and how humans perceive them.”

Researchers developed a participant study to determine if their AI agent could offer rationales that mimicked human responses. Spectators watched the AI agent play the videogame Frogger and then ranked three on-screen rationales in order of how well each described the AI’s game move.

Of the three anonymized justifications for each move – a human-generated response, the AI-agent response, and a randomly generated response – the participants preferred the human-generated rationales first, but the AI-generated responses were a close second.

Frogger offered the researchers the chance to train an AI in a “sequential decision-making environment,” which is a significant research challenge because decisions that the agent has already made influence future decisions. Therefore, explaining the chain of reasoning to experts is difficult, and even more so when communicating with non-experts, according to researchers.

The human spectators understood the goal of Frogger in getting the frog safely home without being hit by moving vehicles or drowned in the river. The simple game mechanics of moving up, down, left or right, allowed the participants to see what the AI was doing, and to evaluate if the rationales on the screen clearly justified the move. 

The spectators judged the rationales based on:

• Confidence – the person is confident in the AI to perform its task 
• Human-likeness – looks like it was made by a human
• Adequate justification – adequately justifies the action taken
• Understandability – helps the person understand the AI’s behavior

AI-generated rationales that were ranked higher by participants were those that showed recognition of environmental conditions and adaptability, as well as those that communicated awareness of upcoming dangers and planned for them. Redundant information that just stated the obvious or mischaracterized the environment were found to have a negative impact.

“This project is more about understanding human perceptions and preferences of these AI systems than it is about building new technologies,” said Ehsan. “At the heart of explainability is sensemaking. We are trying to understand that human factor.”

A second related study validated the researchers’ decision to design their AI agent to be able to offer one of two distinct types of rationales:

• Concise, “focused” rationales or
• Holistic, “complete picture” rationales

In this second study, participants were only offered AI-generated rationales after watching the AI play Frogger. They were asked to select the answer that they preferred in a scenario where an AI made a mistake or behaved unexpectedly. They did not know the rationales were grouped into the two categories.

By a 3-to-1 margin, participants favored answers that were classified in the “complete picture” category. Responses showed that people appreciated the AI thinking about future steps rather than just what was in the moment, which might make them more prone to making another mistake. People also wanted to know more so that they might directly help the AI fix the errant behavior.

“The situated understanding of the perceptions and preferences of people working with AI machines give us a powerful set of actionable insights that can help us design better human-centered, rationale-generating, autonomous agents,” said Mark Riedl, professor of Interactive Computing and lead faculty member on the project.

A possible future direction for the research will apply the findings to autonomous agents of various types, such as companion agents, and how they might respond based on the task at hand. Researchers will also look at how agents might respond in different scenarios, such as during an emergency response or when aiding teachers in the classroom.

The research was presented in March at the Association for Computing Machinery’s Intelligent User Interfaces 2019 Conference. The paper is titled Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptions. Ehsan will present a position paper highlighting the design and evaluation challenges of human-centered Explainable AI systems at the upcoming Emerging Perspectives in Human-Centered Machine Learning workshop at the ACM CHI 2019 conference, May 4-9, in Glasgow, Scotland.

The free event will bring more than 400 high school students from 60 schools across the metro Atlanta area. Catalyst aims to expose underserved students to various branches of science, technology, engineering, art and math (STEAM) education and ignite a spark to pursue such interests in the future.

“HackGT 5, BuildGT, Horizons, and many other hackathon related events are built for college students. Given the educational disparities that exist within certain parts of Atlanta, HackGT understands the importance of reaching out to communities beyond Georgia Tech and other collegiate environments,” said Jordan Madison, computer science (CS) major and HackGT’s director of communications.

In 2017, less than one percent of students in Georgia public schools took the Advanced Placement Computer Science exam. Only two schools in the Atlanta Public School system offered the course. This lack of access motivated the organizers to keep the event completely free, from registration to swag, allowing students from any background the opportunity to participate in Catalyst.

Sponsored by Amazon and Facebook, with in-kind donations from Disney and Pixar Animation Studios, Catalyst offers four tracks for participants to choose from: software, hardware, gaming, and design.

Catalyst welcomes participants with no prior STEAM experience, and each track offers a workshop to help participants develop the basic foundations and skills that are needed to complete the track’s tasks. Participants will create technology pieces in the workshop that they can take home.

Within each track, students are divided into small groups and mentored by college student. The mentors provide hands-on support to help students better grasp concepts. The students will also hear from industry professionals about pursuing an education or career in STEAM during panel discussions scheduled for the event.

“Underserved students success in computer science or other STEAM-related fields is mainly linked to their lack of access to resources and opportunities. They have plenty of talent, but no idea about the options that are waiting for them. Events like Catalyst are crucial for exposing more kids to STEAM who might not otherwise have the opportunity to do so,” said PK Graff, a fellow at the Constellations Center for Equity in Computing at Georgia Tech who teaches computer science at schools in Atlanta Public Schools and serves as an advisory member for Catalyst.

Registration closes April 5 at https://catalyst.hack.gt/#registration

This year is no different as six GT Computing individuals have been awarded fellowships with four different companies, including J.P. Morgan, IBM, Snap, and Facebook. Only those who accepted their awards are listed below.

J.P. Morgan Chase & Co.

J.P. Morgan Chase & Co. awarded Charles David Byrd (Research Scientist and Ph.D. student advised by Professor Tucker Balch) and Assistant Professor Xu Chu for efforts in artificial intelligence research. It is the company’s first AI Research Awards, which are aimed at studying the use of AI and machine learning in areas including investment advice, risk management, digital assistants, and trading behavior. Only 47 fellowships were awarded by the company.

Byrd’s work, along with Balch, is focused on machine learning for financial applications, investigating mutual fund portfolio inference, intraday equity market forecasting, stock market simulation, and machine learning approaches to the evaluation of market efficiency. Byrd has been recognized in the past as the 2018 Graduate Student Instructor of the Year Award in the School of Interactive Computing.

Chu’s research interests revolve around two themes: using data management technologies to make machine learning more usable and using machine learning to tackle hard data management problems like data integration. Chu also earned the Microsoft Research Ph.D. Fellowship in 2015.

IBM

Ph.D. student Stacey Truex of the School of Computer Science was named a 2019 IBM Ph.D. Fellow. The Fellowship, which has been around since the 1950s, recognizes and supports outstanding graduate students who are focused on solving problems that are fundamental to innovation. This includes pioneering work in areas like cognitive computing and augmented intelligence, quantum computing, blockchain, data-centric systems, advanced analytics, security, radical cloud innovation, and more. This highly-competitive award was given to only 16 Ph.D. students worldwide.

Truex (advised by Professor Ling Liu) focuses on research from two complementary perspectives: 1) privacy, security, and trust in machine learning models and algorithmic decision making, and 2) secure, privacy-preserving artificial intelligence systems, services, and applications.

Snap, Inc.

Snap, Inc., recognized Ph.D. student Harsh Agrawal of the School of Interactive Computing with the 2019 Snap Research Fellowship and Scholarship. This fellowship recognizes students carrying out research in areas of computer science relevant to the company, including computer graphics, computer vision, machine learning, data mining, computational imaging, human-computer interaction, and other related fields. Each awardee will receive a $10,000 award and an offer for a full-time paid internship with the company.

Agrawal (advised by Assistant Professor Dhruv Batra) does research at the intersection of computer vision and natural language processing. Prior to joining Georgia Tech, he spent time as a research engineer at Snap Research, where he was responsible for building large-scale infrastructure for visual recognition, search and developed algorithms for low-shot instance detection.

Facebook

Facebook Research announced the selection of 21 Fellows and seven Emerging Scholars this year out of more than 900 submitted applications from Ph.D. students all over the world. Among the awardees were Abhishek Das with the Facebook Fellowship and Vanessa Oguamanam with the Emerging Scholar Award. The Facebook Fellowship program, now in its eighth year, is designed to encourage and support doctoral students engaged in innovative research in computer science and engineering.

Das (advised by Dhruv Batra) does research in deep learning and its applications in building agents that can see, think, talk, and act. His research has been supported by fellowships from Facebook, Adobe, and Snap, Inc., over the years. Oguamanam, who is in the School of Interactive Computing, pursues research in educational technology, human-computer interaction for development, diversity in STEM, and entrepreneurship. She is co-advised by Associate Professor Betsy DiSalvo and Assistant Professor Neha Kumar.

But researchers in the Georgia Tech IMAGINE Lab didn’t have to wait for brick and steel to start being laid or watch a “construction cam” on a website to envision the possibilities for the new building. They were able to use their expertise in digital imaging, 3D modeling, and augmented reality technologies to create Tech Square in a digital model that included Coda in its earliest concept.

In 2015, the IMAGINE Lab, part of the Center for Spatial Planning Analytics and Visualization at Georgia Tech, was tasked by stakeholders at the institute to create a pilot project for a quick visual tool for planning the future Coda building.

“The main goal of the digital application was to quickly visualize a few possible options with building concepts that included 20, 30 and 40 stories, and allow people to interact with the models and see how the cityscape in midtown would be altered,” said Miro Malesevic, digital designer at the IMAGINE Lab.

In essence, the researchers gave decision makers a virtual time machine to the future that brought the building to life and showed how it might be situated in Tech Square and impact the area.

The visualization tool came in the form of an augmented reality app on mobile devices that allowed users to point the screens at a 2D physical map of Tech Square and watch a 3D model of the space come to life on the screen. Users could tap the screen to start with a 20-story building and tap twice more to end up with a structure twice the height (Coda eventually ended up with 21 levels).

Users could also understand how the length of shadows cast by the building or the structure itself might occlude views at the street level or other buildings. The digital AR application even provided a glimpse of the possibility for traffic simulations.

“Use of the 3D AR application has an advantage over traditional 2D blueprints as it provides an individual user with 3D perspective of the design, interaction with the environment, and the ability to use simulations to help in decision-making,” said Malesevic, who worked on the project.

The powerful tool was built within a week, thanks to the IMAGINE Lab’s 3D modeling library, compiled over a 20-year period.

Over the years the IMAGINE Lab has produced numerous architectural visualizations for Georgia Tech, non-profit, and local private organizations supporting economic development efforts at the city and state level.

The third phase of Tech Square was announced in September. It includes preliminary plans for a two-tower complex at the northwest corner of West Peachtree and Fifth streets and possibly a retail plaza as well as an underground parking deck.

The design team in the IMAGINE Lab is already building this next version of Tech Square inside their digital world. The rest of us will have to wait and see how it turns out sometime in 2022 or later.

Story: Joshua Preston

Video: Noah Posner

Video Editing: Terence Rushin

Devi Parikh received tenure and was elevated to the position of associate professor. Parikh joined IC in 2016 and also currently works as a research scientist at Facebook AI Research (FAIR). She earned her Ph.D. from Carnegie Mellon University in 2009. Her research focus is in artificial intelligence (AI) at the intersection of machine learning and computer vision, and she has been recently recognized as one of the top women in AI in publications like Vogue and Forbes.

Dhruv Batra, who joined IC in 2016, received tenure and an elevation to associate professor. His machine learning and computer vision research has been featured in the Boston Globe, Newsweek, and other media outlets. Batra earned his Ph.D. from Carnegie Mellon in 2010 and is also a FAIR research scientist.

Karen Liu, who already has tenure, was promoted to the position of full professor. Liu received her Ph.D. from the University of Washington in 2005. Her research focus is in computer graphics and robotics, including physics-based animation, character animation, optimal control, reinforcement learning, and computational biomechanics. Along with her students, she founded the physics simulator DART, which won the Grand Prize of Open Source Software World Challenge in 2016. Additional research has been featured in places like NBC News, among others.

All three faculty are also members of Georgia Tech's Center for Machine Learning, GVU Center, and Institute for Robotics and Intelligent Machines.

“Each of these faculty members has done tremendous work in Georgia Tech’s School of Interactive Computing,” IC Chair Ayanna Howard said. “They are recognized not only for their incredible research but also for their teaching and leadership service in the community. This is a well-deserved recognition of their hard work hard work and accomplishments.”

In this collaborative environment, IC student researchers are impacting domains including artificial intelligence, robotics, health care, social computing, data visualization and analytics, and more. As a result, their backgrounds are as varied as their research areas.

In this series of Q&As, we’d like you to meet some of our talented graduate students. Today, meet Matthew Guzdial, a machine learning and creativity researcher under advisor Mark Riedl who has already gained plenty of attention for his work on artificial intelligence in video games.

Advisor: IC Associate Professor Mark Riedl

Research Focus Areas: Creativity and machine learning

Hometown: Atlanta, Ga.

High school: Lakeside High School

Undergraduate Degree: B.S. in Computational Media at Georgia Tech

Current Degree Program: Ph.D. in Computer Science

Tell us a little bit about your research:

In creativity and machine learning, I’m interested both in how we get machine learning approaches to work in creative domains, as well as how we take cognitive models of creativity to improve machine learning approaches.

Given the focus on creativity, is there something in your research you have created or would like to create?

In terms of a “thing” I am building, I’ve been collaborating with a team of undergrads and a master’s student on an intelligent level editor for the past few years (video and paper). What I’d really like, though, is to combine this with our work on automated game generation to create a new, intelligent automated game generation tool that would allow anyone to make a 2-D game in, hopefully, an intuitive way.

Do you have a favorite place to hang out on campus or in the city?

On campus, I mostly hang out at my desk or on an armchair in my lab. In the city, I’m a fan of Bookhouse Pub and the restaurant One Eared Stag.

It sounds like you spend a lot of time with research. Do you have any other hobbies you like to do in your spare time?

I’m an avid runner, so I squeeze in a minimum of one 3-plus-mile run each week and aim for 2-3. I’m also hugely into podcasts, with my favorites currently being My Brother, My Brother and Me, Punch Up the Jam, and Friends at the Table.

What is your favorite memory from your years at Georgia Tech?

I was part of this kind of wild production of After the Quake at DramaTech, where we built a visualization for one of the actors that he could “conduct” with his hands through a Kinect, Processing, and a Projector. That was hugely fun to go see.

What is your proudest accomplishment?

Ask me in, like, six months, and I’ll say this Ph.D.

Best, who holds appointments in the Sam Nunn School of International Affairs and the School of Interactive Computing, will offer closing remarks on the report’s launch during the 63^rd session of the Commission on the Status of Women, the principal global intergovernmental body dedicated to the promotion of gender equality and empowerment of women.

The report, titled Taking Stock: Data and Evidence on Gender Equality in Digital Access, Skills and Leadership, highlights the impacts of technology on women in various contexts like jobs and wages, security and privacy, cyber threats, and new technologies such as artificial intelligence.

Among the report’s results are a few key findings:

• While gender gaps are observable in most aspects of information and communications technology (ICT) access, skills, and leadership, the picture is still complex with large regional variations.
• There is no one final strategy for eliminating gender digital inequalities.
• The dominant approaches to gender equality in ICT access, skills and leadership mostly frame issues in binary (male/female) terms, thereby masking the relevance of other pertinent identities.
• To ensure privacy and safety as well as full participation in the digital economy, women and girls should have equal opportunities to develop adequate basic and advanced digital skills.
• Developments in digital technologies open new pathways to gender diversity and inclusion; however, lack of attention to gender dynamics hampers the potential for true progress.

“This report offers a comprehensive look at the issues affecting women and girls equality in a digital age,” Best said. “While surfacing many current challenges, it also sketches pathways forward towards achieving digital gender equality.”

Best joined Georgia Tech’s faculty in 2004 and has directed the United Nations University Institute on Computing and Society (UNU-CS) in China since 2015. He co-founded the EQUALS Global Partnership and the EQUALS Research Group, the latter of which is responsible for the report.

EQUALS works to reverse the increasing digital gender divide, aiming to close the gap by 2030 and supporting the U.N. Sustainable Development Goals by empowering women through their use of information and communication technologies.

“From its inception, the EQUALS Global Partnership has focused on evidence and data to illuminate the intersections of gender with ICTs,” Best said. “We have positioned the EQUALS Research Group at the forefront of this investigation, shining a light onto the realities and possibilities for women and girls in a digital age.”

The report will be released on March 15 at the United Nations Headquarters in New York.