As part of the Cognitive Empowerment Program (CEP), Georgia Tech researchers first explored the People Power IoT System in the technology-equipped Aware Home, a 5,000 square foot home designed to facilitate research. The CEP’s technology team focuses on designing interventions and applications, as well as collecting data in the homes of study participants including sleep patterns, bathroom and kitchen habits, and more.

The team will use the data to empower CEP members and their care partners, through mobile apps and in-home interventions supporting independence. They'll also share key information with the CEP therapeutic team to identify where members may need further training on strategies to compensate for cognitive impairment.

“Changes in daily habits provide key indicators for informing lifestyle interventions that combat a decline in cognitive function. Through this collaboration, we have the unique opportunity to collect critical data to inform care and empower people with MCI and their care partners,” said Elizabeth Mynatt, executive director of the Institute for People and Technology and co-director of the CEP technology core.

In the Aware Home Research Initiative (AHRI), the People Power IoT System will provide device and data management and a framework for creating machine learning microservices, allowing researchers to more effectively conduct long-term studies and collect and view data, according to Brian Jones, director of the Aware Home initiative.

“What People Power is providing is the opportunity to hit the ground running on this project,” said Jones. “We can have different studies running, view all of the homes in each study, and through a web browser see the status of in-home devices that provide the system with data.”

Although the joint effort between IPaT and People Power currently focuses on adults with MCI, Jones envisions expanding it to other data science research at Georgia Tech.

People Power is a leading provider of consumer and institutional services for senior care, energy management and home security. The company’s recent introduction of a comprehensive senior care solution that brings together agencies, caregivers, clients and their families for improved senior care reconfirms their mission of helping with life’s important challenges with easy-to-use technology.

“Collaborating with Georgia Institute of Technology’s Aware Home Research Initiative is a real honor for our company,” said Gene Wang, CEO and co-founder of People Power Co. “As a technology provider for senior care solutions that address technical, design and social challenges of aging, we are delighted to assist in this important research initiative with one of the top research universities in the country.”

The company also recently partnered with the University of California Berkeley to design senior care technology for people with dementia. Learn more about People Power on their website.

Georgia Tech researchers are using augmented reality (AR) tools to help older adults with disabilities to choose smart home technology for their individual needs that will work in their homes.

The project is now in its second year. During the first year, researchers wanted to know what older adults with disabilities knew about smart home tech – Do they use it? If not, why? Study participants provided feedback about which products they're most interested in and tried out the voice-activated Amazon Echo, a device that’s easier to use for someone with a mobility impairment.

Researchers found, “There’s a lot of misunderstanding or incorrect expectations around what the devices do or how they work,” said Maribeth Gandy Coleman, the project’s principal investigator and director of Georgia Tech’s Interactive Media Technology Center.

Some study participants were hesitant to use the products because they felt overwhelmed or assumed that they wouldn’t be able to use them. 

Currently, Coleman, along with Research Scientist Laura Levy and Research Technician Ben Thompson, are taking what they learned in the first year and have built a prototype that uses AR as a guided home tour. Study participants, with assistance, use a smartphone and AR-tracking product information cards to test different smart home technologies. Is the wi-fi strong enough in a particular corner of their home? Can they reach the device? Is the electrical outlet close enough?

“By situating virtual information for the person in the real world, we can help them better figure out what smart home tech would be best for them and their environment. It would help them be able to preview what the user experience would be like in their home,” Coleman explained.

There’s also an interactive, printed floor plan. “You’re able to see a bird’s eye view of the whole house.”

Potential consumers can develop she calls “a more robust mental model” of how smart home tech works, which could help them choose what to buy, where to install it, and how to fix any technical problems.

During the study, Coleman and her team discovered accessibility barriers with commercial augmented reality products. Microsoft’s HoloLens interface was difficult to use for people with certain mobility impairments because it involves pointing and pinching motions. Participants also had difficulty putting on and taking off the AR headset.

In addition to supporting consumers in finding the right smart home tech, Gandy said the AR experience could also help technology designers to create more accessible AR and VR products for older people as well as those with cognitive and physical disabilities, which impacts how they use these products at home and work.

"This could be critical if future job training is done through virtual reality," she said. "You could inadvertently be excluding someone from a job, not because they couldn't do the job but because the training was inaccessible. Better technology doesn't necessarily mean more accessible; it can mean less accessible."

This project is supported by TechSAge, a Rehabilitation Engineering Research Center (RERC) to support aging-in-place for people with long-term disabilities. The center is a collaboration between Georgia Tech and the University of Illinois at Urbana-Champaign. A grant from the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR) funds the center. 

The Institute for People and Technology (IPaT) at the Georgia Institute of Technology is working to help people take their power back through an array of research initiatives, centers, and partnerships, including the new Cognitive Empowerment Program in collaboration with the Emory Brain Health Center at Emory University.

“With our work in aging, we want to help people achieve the priorities and goals they have for their third stage of life,” said Beth Mynatt, executive director of IPaT and Regents Professor in the College of Computing at Georgia Tech. “We think technologies and different healthcare approaches, like help at home and a focus on social connections and community, can help people do this.”

Improving Health Outcomes

Tele Tai Chi

Tracy Mitzner, a research scientist in the School of Psychology, is working on an evidence-based program with world-renowned tai chi instructor Paul Lam on a project designed to help older people with mobility impairments increase both their activity level and their social connections. 

"A strong social connection has very significant health outcomes,” said Mitzner, who’s also co-director of TechSAge, a collaborative research center with the University of Illinois at Urbana-Champaign, which supports aging-in-place for people with long-term disabilities. “And a lack of social connection increases your mortality risk. So, it's not just about feeling good; it's directly tied to your health.”

The Tele Tai Chi program brings together a group of older adults through OneClick.chat, a web-based video conferencing platform. They will not only learn tai chi through the system, but they’ll also participate in social time, much like one does in a traditional class.

Measuring gait speed

Known as an extremely accurate, fast, and simple health indicator, doctors use walking speed and changes to predict significant health issues like physical and cognitive decline, quality of life, hospitalizations, and even mortality. 

Brian Jones, director of the Aware Home Research Initiative, is collaborating with Ted Johnson, director of the Center for Health in Aging at Emory, to test a system they placed in doctor’s offices and medical centers in Atlanta and Oregon. It measures a person’s gait speed as they walk from the waiting room to the exam room, and is designed to be a quick and unobtrusive method for detecting issues that otherwise may go unnoticed.

Enhancing the Quality of Life

Smart Bathroom

Jones has also just completed a trial testing a bathroom that helps people get on and off the toilet. It uses off-the-shelf technology to observe the person’s gait, posture, and their speed of movement. The Smart Bathroom uses that information to raise or lower the toilet and grab bars to assist with the mount or dismount. 

“Our next trial of the Smart Bathroom will involve testing it with people who use a wheelchair and people who use a walker,” said Jones. 

HomeLab

To learn if people can and will use a piece of assistive technology in the home, it pays to test it in their home. “There’s only so much you can do in the lab,” said Brad Fain, director of HomeLab and principal research scientist at Georgia Tech Research Institute. “People get tired, and older adults get tired even quicker.”

HomeLab deploys products in the home to observe how people interact with them over time so the lab can test assumptions or make improvements. With a network of more than 600 homes of people aged 50 and older, HomeLab has assessed systems such as a wearable device that reminds users when its time for their medication, an in-home medication dispenser, and a smart mirror.  

“It's real-world, and it gives people a lot more experience with the product or service we’re testing,” added Fain.

 A recent trial involved testing commercial activity trackers (FitBit, Nike+ Fuel Band, Striiv, and MyFitnessPal) with older adults to see if knowing their actual level of daily activity would inspire them to change their behavior. 

“We saw real change with people trying to increase their activity level by adding just a bit more to what they’re already doing, like taking the long way when they went for a walk or took the dog for a stroll,” said Fain. 

Everyday Memory Intervention

One of the more frightening aspects of aging is a dwindling memory. “It’s something other people may be aware of before you are,” said Christopher Hertzog, a professor in Georgia Tech’s School of Psychology. 

Scientists typically measure memories with diaries people use at the end of each day. "But they forget what happened during the day, and often once they’ve forgotten it, then it’s gone,” said Hertzog. 

One of his current projects, Everyday Memory Intervention, gets around that end-of-the-day problem by using a smartphone for people to record the events as they happen. It’s an attempt to overcome the difficulties of measuring memory in the real world. One of the problems with current mnemonics training is that while people can learn to memorize the information, they have more difficulty transferring that training into something useful in real life. 

“What we’re doing,” he said, “is turning that on its head and focusing on helping people remember the target behavior in their everyday life,” basically cutting out the middleman.

Cognitive Empowerment Program

In January 2020, IPaT, in partnership with the Emory Brain Health Center and the SimTigrate Design Lab in Georgia Tech’s College of Design, will open the Cognitive Empowerment Program (CEP) at Emory's Brain Health Center. The program is made possible by $23.7 million in grants from the James M. Cox Foundation and Cox Enterprises. CEP has four major pieces: the therapeutics core, built environment core, technology core, and the innovation accelerator. Together they will provide a holistic approach to managing mild cognitive impairment (MCI), a neurological condition that occurs between the typical cognitive decline of aging and dementia.

Memory problems, as well as difficulties with language, judgment, and general thinking to a degree that is greater than typically experienced in healthy aging, all characterize MCI. People with MCI may stay at that stage, or the condition may be a precursor to a dementia disorder such as Alzheimer’s disease, vascular dementia, or another neurological condition. It rarely reverses. The American Academy of Neurology estimates that about 14.9 percent of Americans over the age of 65 develop mild cognitive impairment.

“There’s a use it or lose it type of aspect about human cognitive capabilities,” said Mynatt. "So, if we can delay, stop, or even reverse the trends associated with cognitive decline, we can help people improve their quality of life.”

This year the World Health Organization released a report on reducing the risks of cognitive decline and dementia. Chief among its recommendations is for people to increase their physical activity. 

“The idea behind the empowerment program was really to put a structure to the recommendations given by a physician for how patients can manage their diagnosis,” said Amy Rodriquez, director of therapeutic programs at CEP and assistant professor in Emory School of Medicine’s Department of Neurology. “We’re saying we can support these patients in making changes that could impact the course of their condition.”

A person with MCI may find that the instrumental activities of daily living are more difficult than before, said Rodriguez. Things like managing finances, getting to places on time, navigating, even dressing, become challenging. 

Emory’s cognitive neurology clinic will refer older adults to the program where they can engage in a host of activities known to stimulate their cognition. They’ll also have the opportunity to join groups to learn strategies for supporting memory, exercise, including strength training and yoga, take part in a book club or art class, and cook in the demo kitchen. These are all activities meant to empower people with MCI, boost their social engagement, and support their continued independence. 

Craig Zimring, director of the built environment core at CEP and director of the SimTigrate Design Lab, is overseeing space design. Zimring’s team will concentrate on how the center’s environment impacts people’s behavior. Made with the principles of universal design, it will be easy to navigate no matter what physical ability a person possesses. It’s also a visually open space with generous natural lighting, a layout intended to encourage interaction and co-design between program participants and researchers. The team will survey how people use the center; are they being social, reaching out to others? They'll analyze the environment, including noise and temperature.

“I want people to be able to look in at the research we're doing and say, 'That looks neat. What are you creating? Lemme tell you my thoughts about it,” said Jennifer DuBose, director of the innovation accelerator core at CEP and associate director of the SimTigrate Design Lab.

The innovation accelerator will offer seed grants to support research aimed at developing, testing, and disseminating evidence-based interventions for MCI. DuBose is also getting students in on the action. They will be busy developing solutions related to some of the difficulties people with MCI experience in their daily lives. The most promising solutions will be picked up and worked on by researchers at CEP, who will test some in the real world.

The technology core, directed by Mynatt, will design interventions and apps, as well as sense and gather data, connecting the program to participant’s homes and daily activities. One of her projects involves creating a Siri-like social assistant for people with MCI and their care partners. Key goals driving work in participant homes include facilitating important activities such as physical exercise and healthy nutrition while detecting and reversing harmful situations such as poor sleep.

While they have funding for three years, Rodriguez said, they’re working to create ways to sustain the center that empowers patients, families, and staff as co-designers.

“First and foremost, it’s about empowerment; how can we systematically increase people’s sense of empowerment?” said Mynatt. “You can take scans of people’s brains all day long, but we want to empower the people.”

Writer: David Terraso

QuickSip, which won first place in the competition’s Health on the Move category, aims to simplify the ordering process and reduce wait times for service at busy bars. The application uses a messaging service for real-time drink ordering and status updates. 

Georgia Tech students Renner Brown, Thomas Horton, Whitney Nelson, and Cameron Pepe developed QuickSip. In addition to their CIC win, the team also earned a “golden ticket” to participate in this summer’s CREATE-X Startup Launch 2020 and further develop their application.

Eighteen teams participated in the Fall 2019 CIC live demo and judging finale. Here’s a look at the full list of winning projects in each category:

Players and Fans

1^st place - Handl is an app that uses QR codes to facilitate exchanging contact and social media information.

(Team: Yu-Lin Chung, Kenneth Huang, Wyndham Hudson, Phalguna Rupanagudi, Pragati Singh)

2^nd place - LetzChill uses notifications to arranges spontaneous meetups between friends. (Team: Brooke Brennan, Faiyaz Chowdhury, Reagan Hanna, Aditya Vadhavkar)

Health on the Move

1^st place - QuickSip uses a messaging service to simplify the drink ordering process at bars.

(Team: Renner Brown, Thomas Horton, Whitney Nelson, Cameron Pope)

2^nd place - LDR focuses on providing a sense of online intimacy to long-distance couples.

(Team: Rahul Brahmal, Dennis Eddington, Samuel Huang, Songming Liu, Richard Zhang)

Climate Solutions

1^st place - Stumble connects travelers to great local experiences and tips.

(Team: Younmin Bae, Devon Bhavsar, Luis Gonzalez, Joshua Reno, Rahul Zhade)

2^nd place - CryptoPass is a green password manager that uses decentralization to save energy and is always available.

(Team: Nathan Hazelton, Zach Sanchez, Jacob Watson, Jacob Zipper)

Students participating in the CIC create technology products, applications, and experiences in categories determined by campus, industry, and community partners. Each fall and spring semester, they work individually or on teams and can submit class projects or projects unrelated to a class. Current Georgia Tech graduate and undergraduate students, as well as those enrolled in the Institute the previous semester, are all eligible to participate in the competition.

RNOC also provides access to the App Lab in the Technology Square Research Building, where students can get help with their projects and check out mobile devices for testing.

Learn more about the CIC, including sponsorship opportunities, at cic.gatech.edu.

Students participating in the competition, also known as the CIC, create technology products, applications, and experiences in categories determined by campus, industry, and community partners. Each fall and spring semester, they work individually or on teams and can submit class projects or projects unrelated to a class. Current Georgia Tech graduate and undergraduate students, as well as those enrolled in the Institute the previous semester, are all eligible to participate.

Here are the categories for the Fall 2019 Convergence Innovation Competition:

Climate Solutions, in partnership with the Georgia Tech Global Change Program, is focused on developing practical solutions related to climate change. Solutions may contribute to limiting carbon emissions, coping with environmental, social, or business changes, or promoting a greater understanding of how climate change impacts wellbeing and prosperity.

Health on the Move promotes the vision that all individuals — regardless of age, socioeconomic status, or health — fully engage in life, sustaining relationships, independence, and quality of life while continuing to learn, grow, and contribute to society. Innovation in this category will support health and wellbeing in any phase of life.

Players and Fans encompasses innovations that benefit athletes, artists, entertainers, esports gamers, fans, team owners, or professionals, whether at the amateur, youth, or professional level. Solutions could focus on improving performance, expanding access and interest in the craft, lowering financial or other barriers to participation, skills acquisition, injury prevention, and enhancing the fan experience.

Competition organizers are reminding students that the solutions outlined in these categories are suggestions to spark ideas. If you have a question about whether your idea fits, send an email to RNOC. 

RNOC also provides access to the App Lab in the Technology Square Research Building, where students can get help with their projects and check out mobile devices for testing.

Students must submit competition entries on the project submission page by Sunday, November 10, at midnight. The invitation-only CIC demo and judging event is on Wednesday, November 13, from 4-7 pm at IPaT’s Centergy Building location.

Winning CIC entries include a working end-to-end prototype and a value proposition model. Previous winners have leveraged their CIC experience to participate in other competitions, find internship and job opportunities, and commercialize their projects.

Learn more about the CIC, including sponsorship opportunities, at cic.gatech.edu.

The initiative will focus on three components:

• Athletic performance (athlete health and human performance, wearables, and analytics)
• Fan engagement (in-game engagement, relationship management, segmentation, communications, sales, and customer marketing)
• Operations (ticketing, game-day operations, parking, security, concessions, ingress/egress, and venue management)

Doug Allvine is leading the effort as assistant athletics director for innovation, a role created in 2017 by Athletic Director Todd Stansbury within the Georgia Tech Athletic Association.

“The Georgia Institute of Technology is a leading research university,” said Stansbury. “Applying our research innovation to sports is a huge opportunity for faculty and students, and gives our teams on and off the field a competitive advantage. Sports innovation creates opportunities for research, grants, corporate partnerships, industry collaboration, and student advancement.”

IPaT will bring together Georgia Tech’s research community to support and amplify the work of faculty in this area, including the emerging esports industry, the use of technology to engage sports fans, and the physical and mental health of athletes.     

“Georgia Tech researchers have examined the critical role of technology through the lens of sports for many years,” said IPaT Executive Director Beth Mynatt. “IPaT and our partners are excited to convene multidisciplinary research teams engaged in pioneering work that improves human performance and enhances the fan experience.”

Faculty and staff from the Georgia Tech Research Institute (GTRI), the Office of Industry Collaboration (OIC), VentureLab, the Exercise Physiology Laboratory, the School of Biological Sciences, the Wallace H. Coulter Department of Biomedical Engineering, the Sports, Society, and Technology program, and CreateX are also partnering in the initiative.

To learn more about SPRINT, including research, news, events, and student and partnership opportunities, visit the SPRINT website.

Technology for Social Good

Two of the winning CIC projects use the power of technology to create social change. Food for Thought is an app that connects businesses, non-profits, schools and individuals that have excess edible food to those who need it. App users have the option to either share, transport or request food, and can add images to their posts. “Brownie points” gamify the app and create a feel-good factor around sharing.

Approximately 40-percent of food in the U.S. is never eaten, making this country the world’s largest food waster. “You assume that because this is a first-world country, there’s enough food,” said Meghna Natraj, Georgia Tech master’s student in Computer Science. “Food’s being wasted; it’s not being channeled to the right sources.”

The Food for Thought team aims to have a final version finished by the end of April.

Another winning project in the same category uses technology to address gentrification of the area surrounding the Atlanta Beltline. “Concerns about creating a discriminatory environment, rather than universally-available resources are growing,” explained the team in their submission video.

The Beltline Display project envisions interactive experiences along the Beltline to promote connected and walkable communities. Utilizing big data, the team proposes creating technology-focused art pieces along the trail to educate users about their surroundings, neighbors and history of the area. The team’s goals are to promote social change, foster curiosity, and connect Atlanta communities.

Real-Time Information for Transit and More

It’s been nearly a month since fire engulfed part of I-85 causing it to collapse, and the Georgia Department of Transportation says it could take at least another seven weeks to repair the damage. In the meantime, commuters are exploring alternative means of transportation. The developers of MARTAnow say their app is the perfect use case for solving transit problems.

Even before the I-85 collapse, the team behind MARTAnow has been researching why some commuters are reluctant to use MARTA. They’re also trying to solve a larger issue in public transit called the first-mile/last-mile problem – getting people from their location to the bus stop, and from the bus stop to their final location.

“A bridge collapses and all of a sudden, hundreds of thousands of people can’t get to where they need to go,” said Amit Garg, Georgia Tech master’s student in Human-Computer Interaction. “Using our app, people can visualize other modes of transportation to get to their final destination, whether that’s ridesharing, through renting a bike, or using MARTA trains and buses.”

MARTAnow combines real-time MARTA schedules, walkability scores, and ridesharing into one easy-to-use transit app. MARTA is now working with Garg and his teammates to integrate the app into its ecosystem.

While they may not be working to solve something as complicated as transit issues, the creators of NowWhat are working to answer a question important to students – where’s the party?

“All of us Tech students have had this problem at one time or another,” explained Ryan Brooks, Georgia Tech Computer Science student.

NowWhat is a map interface app that allows you to see events happening around you in real time, get rides to those events, and see friends’ current locations. Information from different event sources is gathered in one app for ease of use.

“We’ve never had that one central location, to see on a map how far away something is and being able to visualize it in that way,” said Brooks.

The team will soon work on developing the app for iOS and Android.
 

Enhancing Medical Intervention

As many as 40% of people with autism never speak or have difficulty communicating verbally. Rapid Prompting Method, or RPM, was developed by a mother who wanted to teach her non-verbal autistic son to express himself. A group of Georgia Tech students has now found a way to enhance RPM using real-time data.

The process of RPM is simple: a teacher gives a short lesson on a topic, asks a question and then elicits a response using verbal, auditory, or visual prompts. The child will answer based on the teacher’s question by spelling out the letters written on a letterboard. The CIC project Responsive Letterboard for Autism Spectrum Disorder reinvents the board. When the child presses a letter, their selection is transferred to a web user interface in real time. This method allows teachers and clinicians to view the data, track a child’s progress and make improvements.

The team was inspired by their professor, Gregory Abowd, who has an autistic son. “That community of parents and technicians are really positive and helpful and they’re engaged in our research,” said Fereshteh Shahmiri, Georgia Tech Ph.D. student in Design Computing.

April is Autism Awareness Month.

One of the greatest threats to patient safety during an operation is surgical site infection. According to the CDC, in 2011 approximately 157,500 acute care hospital patients were diagnosed with infections.

“Healthcare professionals try their best to prevent this type of infection, but the increasing mortality of surgical site infections has forced professionals to consider every possible way in which the surgical environment can be controlled,” said Luka Antolic-Soban, Georgia Tech Computer Science student.

ORCA, or Operating Room Computer Asepsis System, is designed to help operating room professionals detect and prevent contamination during operations. The system has two components. One is a belt that goes underneath surgical scrubs to alert operating room staff when they violate protocol. The other uses a camera and infrared sensor to collect information about the operating room environment, such as the distance between personnel.

The ORCA team is collaborating with Emory University School of Medicine and Children’s Hospital of Atlanta to test the system. They've also been invited to participate in Georgia Tech's Startup Launch (formerly Startup Summer), a faculty-led, student-focused program to help student teams launch their startups.

Following the competition, teams are leveraging the feedback received and contacts made during the course of working on their CIC projects and showcasing them at the live judging event. They are debugging their apps, incorporating enhancements, pursuing partnerships, and learning more about entrepreneurship. This semester’s CIC categories were aligned with IPaT’s research priorities; here’s a full list of winning projects and teams:

Lifelong Health and Wellbeing
1st place – Operating Room Computer Asepsis System (ORCA)
Team: Luka Antolic-Soban, Jianming Zeng

2nd place – Responsive Letterboard for Autism Spectrum Disorder
Team: Anisha Bhandari, Geunbae Lee, Fereshteh Shahmiri, Vedant Das Swain

Smart Cities and Healthy Communities
1st place – NowWhat
Team: Ryan Brooks, Joan Chen, Will Christian, Aaron Parry, Brandi Van de Houten

2nd place – MARTAnow
Team: Vishal Bhatnagar, Prasenjeet Biswal, Amit Garg, Samyukta Sherugar

Socio-Technical Systems and Human-Technology Frontier Innovation
1st place – Food for Thought
Team: Akshay Agarwal, Luka Antolic-Soban, Satyajeet Gawas, Meghna Natraj

2nd place – Beltline Display
Team: Aparna Iyer, Jayanth Krishna, Meghana Melkote, Caity Taylor, Eric Thompson

VIEW: More photos from the Spring 2017 Convergence Innovation Competition

The CIC is a semester-long event dedicated to helping students create innovative and viable products and experiences with the support of campus resources and industry guidance. The competition is open to Georgia Tech students from every class and discipline. To learn more, or to find about partnership opportunities, visit cic.gatech.edu.

IPaT Distinguished Lecture, Dolphin Communication: Cracking the Code

On October 20th, IPaT and the College of Computing will host the first IPaT Distinguished Lecture, Dolphin Communication: Cracking the Code. Scientists have struggled to understand dolphin vocalizations, but new computer tools to both track dolphins and decode their complex vocalizations are now emerging. Dr. Denise Herzing has been studying Atlantic spotted dolphins, Stenella frontalis, in the Bahamas for over three decades. Her video and acoustic database encompasses a myriad of complex vocalizations and dolphin behavior. Dr. Thad Starner works on mining this dataset and decoding dolphin sounds, and has created a wearable underwater computer, CHAT (Cetacean Hearing and Telemetry), to help establish a bridge for communication between humans and dolphins. Starner and Herzing will present this cutting-edge work and recent results, including perspectives on the challenges of studying this aquatic society, and decoding their communication signals using the latest technology.

RSVP for the lecture  |  Directions and parking

The next day on October 21st, two more events related to wearable computing and future technology – The Georgia Tech Wearable Computing Center Forum, and the GVU Fall Research Showcase.
 

Georgia Tech Wearable Computing Center Forum

Following last year’s success, the Wearable Computing Center Forum is now an annual endeavor. The WCC Forum is an opportunity to hear from pioneers in wearable technology about the state of the art in the field. Experts in important application areas will also speak to how wearable technology might help solve problems unique to these areas; healthcare or wearables in the workplace for example.

The forum keynote speaker is Brian Ballard, CEO & Co-Founder of APX Labs where he is working on the next industrial revolution - using smart glasses to create powerful ways for the workforce to interact with the digital and physical world. Brian has been on the forefront of developing software solutions that enable revolutionary new scenarios for smart glasses in the enterprise.

Talented and multi-strumentalist Rhò will perform at the forum. He studied music extensively when he was growing up thanks to a kindly uncle who saved him from playing football every night with the other kids. Born Rocco Centrella, the young maverick has a unique relationship with sound. Balance is key to the work of Rhò, whose voice travels to some unusual and very Italian harmonic vistas, mixing folk and electronic.
 

Register for the Wearable Computing Center Forum

GVU Fall Research Showcase

What will be the next great innovation in wearable computing, mobile apps, and digital design? Explore and interact with future technology at the GVU Fall Research Showcase. Experience a hands-on vision of the ideas and applications that you may carry in your pocket or use in the workspace in the future. Test drive many of the 80+ demos including: the latest social media research, artificial intelligence that crowdsources plots for interactive stories, and computational remixing of hip hop.
 

More info about the GVU Fall Research Showcase

“We’re serving two audiences with this exhibition,” said Gregory Zinman, co-curator of MORE LIGHT and assistant professor in the School of Literature, Media, and Communication. “One is the art community of Atlanta; we wanted to do an exhibition that we hadn’t quite seen in Atlanta, which was to focus solely on media arts and ideas of different forms of the moving image. We also want to show off the Atlanta community to this other audience, which is all of these visiting folks coming in for the conference who may be less familiar with what’s going on in Atlanta.”

Visitors of the exhibition will see works using 16 mm film, augmented reality, video games, and even larger than life projections from the rooftop of the gallery.

“We’re used to looking at images and movies on our phones, tablets, laptops, or when we go to the cinema. But there are so many other manifestations of the moving image that we can think about, and the way technology shapes those images,” said Zinman. “So if you decide to use a VCR, how is it different than a digital projector? We wanted to get people to think about that by putting all of these different methods and materials in conversation with one another.”

Georgia Tech-related works include:

MOSAIC: Mobile Cinematics by Georgia Tech professor Jay Bolter and Maria Engberg, associate professor of media technology at Malmö University in Sweden, which uses audio interviews, Twitter streams, and Flickr images to create a mosaic that probes people’s use of these mediums.

Nightmare by Georgia Tech Digital Media master’s student Kristan Woolford, an experimental observational documentary that merges archival video and footage of the protest of Troy Davis’ execution at The Georgia Diagnostic and Classification State Prison in 2011. Woolford uses a first-person perspective to immerse the audience into the protest experience.

Pioneering video artist Dara Birnbaum will also be contributing newly-digitized documentation of her now-lost Rio VideoWall (1989), commissioned for the Rio Shopping and Entertainment Complex, once located at the corner of Piedmont and North Avenue. The VideoWallwas the first public artwork of its kind in the United States, combining an unedited live-stream from CNN glimpsed through the moving silhouettes of mall patrons with a video record of the site’s natural landscape from the time before the mall’s construction.

Eyedrum Art & Music Gallery is hosting the exhibition, which will span all of the gallery’s spaces, including the music room, gallery, and rooftop. Eyedrum has been the center of the Atlanta arts community for over 20 years and is part of an emerging arts scene in downtown Atlanta.

“The Southeast has been long underrepresented in the national conversation around the arts. It’s underrepresented at major museums; it’s certainly underrepresented as soon as you get outside of the United States,” said Zinman. “I don’t mind beating the drum a little bit as a point of pride to say, there’s a lot of great work being done here, take a look.”

MORE LIGHT runs from March 31st – April 2nd at Eyedrum, 88 Forsyth St, Atlanta GA, 30303. It’s free for SCMS members and $5 for the general public. For more information, visit: morelight.lmc.gatech.edu

How has your experience been at Georgia Tech?

It’s been really cool being exposed to all sorts of interesting projects here at Georgia Tech. Everybody’s brilliant and I’ve been able to have all sorts of opportunities with some of the leading researchers in the field. It’s just been really amazing.

What are your plans after graduation?

I’ve considered academia, but I’m definitely leaning more towards industry. On the one hand I do enjoy teaching and tutoring and I’ve done a lot of that in the past, so I could see myself doing some part time teaching. But I’ll probably go into industry first and perhaps eventually become a consultant and do something more entrepreneurial. I've also become increasingly interested in alternate (post-scarcity) economic systems in the last several years, so another thing I will definitely want to explore after I graduate is how we can use technology to introduce and facilitate new ways of living and working together as a society.


Photos by: Joshua Preston
Graphics by: Raul Perez

*  *  *
 

Student: Logan Sand, M.S. Digital Media
Advisor: Michael Nitsche, Associate Professor, School of Literature, Media and Communication, Digital World & Image Group
Project: Augmented Reality Comic Book

Sand is using comic book theorists Scott McCloud and Will Eisner’s ideas on graphic novel gutters, or the space between panels, to create an augmented reality comic book. He wants to divide the user’s attention as equally as possible between the physical artifact of the comic book and the companion app.

The goal of Sand’s project is to address the instability that is created in hybrid media when the user’s attention switches between the digital and physical artifacts. For example, apps that use QR codes as markers obscure the codes, removing them from view. Or for things that focus heavily on the physical object, the app is almost always optional or incidental.

Many game companies, both video and tabletop, have begun to research and develop for augmented reality. But according to Sand, their designs are flawed. “It becomes apparent that they are only just tiptoeing outside of their comfort zones in the creation of artifacts,” he said. “The companies that are used to designing purely digital games use the tangible creation as just a tool for image recognition without focusing too much on its design aside from pure utilitarianism, and the companies that are used to designing tabletop games use the augmented reality as minimal enhancement that’s usually optional for gameplay.”


Photos by: Joshua Preston
Graphics by: Raul Perez

The festival features performances, conversational talks, and nearly 50 interactive installations from 15 ACC schools. Georgia Tech will send four projects created by students, faculty, and researchers, including Creative Collisions. The project is a collaboration between Georgia Tech Research Scientists Laura Levy, Clint Zeagler, and Scott Gilliland and Los Angeles-based performance artist Katherine Helen Fisher. Last fall, the group facilitated a series of workshops to discuss how artists and technologists can work together; they also created an interactive garment. “Le Monstre” is a fun, “Fraggle Rock” type of creation that includes sensors and accelerometers and encourages audience participation. Fisher wore the garment in her performance piece Characters last March at the Ferst Center for the Arts.

The team behind Creative Collisions also recently won a design award at the International Symposium on Wearable Computers (ISWC), September 11-15 in Maui, Hawaii. Their paper, Le Monstre: An Interactive Participatory Performance Costume won the Best Design Exhibition Award in the Aesthetics category. ISWC, held in conjunction with the Ubicomp 2017 conference, brings together researchers, fashion designers, product vendors and other related professionals to share information and advances in wearable computing. This year, the conference accepted 11 Georgia Tech papers.

Partners on Creative Collisions include the Georgia Tech Wearable Computing Center, the Interactive Media Technology Center, and the Office of the Arts. The project is funded by IPaT as part of the Research and Engagement Seed Grants Program.

Learn more about Creative Collisions and the team’s participation in ACCelerate at the GVU Center Brown Bag, September 28 at 11:30 a.m. in the Technology Square Research Building.

“As we were processing this information, we also had to face the fact that we weren’t really working with teens,” said Joan Prittie, executive director of Project Safe. “Here we are in Athens with the University of Georgia, other universities, and several high schools. Apart from going in and doing a few presentations here and there, we weren’t hearing from teens and young adults.”

The text line, according to Prittie, is a way to assist teens through a familiar mode of communication. Trained text line interns are “near peers” – local college students studying subjects like criminal justice and social work. “It’s like talking to a really smart older brother or sister,” said Mary Haddon, Project Safe youth educator & advocate.

When Breaking Silence began, a group of interns had one smartphone, which they handed off to each other between shifts. “That worked for a while,” explained Prittie. “We were just starting out and kind of feeling our way.”

Connecting with Georgia Tech

As the program expanded it became difficult for interns to keep up with the surge of texts coming in, especially after staff members promoted the text line during school presentations. They also discovered an uneven distribution of texts. Some interns would be bombarded, while others wouldn’t receive any messages. That’s when Project Safe board member Margaret Wagner Dahl had an idea.

“I said to Joan, I think we should talk to Georgia Tech through the Institute for People and Technology. That’s an appropriate place to try and sort this out,” she said. Wagner Dahl, who is also Georgia Tech’s associate vice president of Health IT, connected Project Safe with Jeremy Johnson, a senior research scientist with IPaT and the Interactive Media Technology Center (IMTC).

In 2015, Johnson and a team of Georgia Tech undergraduate and graduate students began developing a web-based interface that addresses several concerns. Instead of sharing one phone, the software allows interns to receive texts on multiple, individual phones. Incoming messages are distributed in a round robin fashion, solving the problem of uneven distribution.

“I think we fail to appreciate just how useful text messaging alone can be in addressing the need for certain applications,” said Johnson. “People often think they need a flashy, cross-platform application. You can accomplish a lot for certain types of applications just using a purely text-based interface.”

Text line managers can also review transcripts of conversations, which helps with supervision and training. “The cool thing about the technology is if an intern has a text and they’re not sure how to respond, we can go in and look at the whole conversation from our phones and help them,” said Adrienne Shook, Project Safe text line manager. 

Since Project Safe started using the new system in August 2016, the text line has steadily grown from about 30 clients a month to around 50. Staff members are promoting Breaking Silence statewide, and can now receive text messages from across Georgia. Prittie predicts, “Over time, this is going to become more likely the way that hotline type services are offered in our movement.”

She hopes to continue collaborating with Georgia Tech and says the partnership has not only been beneficial for Project Safe, but also for the Georgia Tech students who worked on the project.

“I think it was very interesting [for students] to see how their skills could help solve human problems and make life safer and better for people through the use of technology.”

Johnson, who is the father of two young daughters, agrees. He volunteered some of his time in order to complete the project. “It’s a wonderful cause. What Project Safe is doing is tremendous and impacts the lives of so many young people who are having moments of crisis. I just really believed in their mission and what they’re trying to accomplish.”

February is teen dating violence awareness month. To learn more about Project Safe’s programs and services, visit their website.

Journalists and other communicators have used visualization for years to summarize sports statistics. Coaches and trainers use it, too, for training regimens, understanding injury patterns, and mapping the movement of players on the court and field.

With the growing popularity of fantasy sports leagues, fans are becoming more engaged with player stats and performance and getting in on the data vis action. We're also more visually literate—comfortable digesting bar graphs and scatter plots—according to School of Interactive Computing Associate Professor Rahul Basole.

“People are getting more data savvy and hungry, and the tools have gotten easier across the board,” said Basole. “We are visual creatures. We like to have [data] summarized visually and in an aesthetic and compelling fashion."

Basole also says there’s just simply more data out there on a wide range of sports, ready to be analyzed. He and fellow interactive computing Professor John Stasko are currently examining ShotLink data from the PGA Tour, which tracks every shot from every player across all courses. The organization has made the raw data available to academic researchers for novel analyses.

"Our goal is to visualize ShotLink data to make sense of individual player performance as well as their overall patterns across time and different courses,” said Basole.

He and Stasko plan to visualize the data in an interactive dashboard for a user-oriented perspective. "As with many domains, vis alone is not the sweet spot for sports. It's great for communicating and presenting; it's also great for sensemaking. But it's the combination with analytics that can truly provide insights."

* * *

From USB-enabled cup holders and massive LED scoreboards and displays to the fastest wi-fi at any sporting venue in North America, SunTrust Park is considered one of the most technologically advanced stadiums in the country.

“We view technology as an enabler," said Greg Mize, senior director of marketing & innovation, Atlanta Braves. "One of our sayings is, ‘Don’t pursue and integrate technology for the sake of technology.’" It has to provide value for fans in the form of entertainment, engagement, or information according to Mize. "Our goal is to create the best experience for fans as possible, and technology plays a massive part in that.”

SunTrust Park and the Battery Atlanta are using artificial intelligence to help visitors navigate shops and services, and augmented reality technology to educate visitors about historical artifacts on display at Monument Garden.

Georgia Tech is also partnering with the Atlanta Braves to enhance the fan experience at SunTrust Park. Last season, graduate students studied how long visitors waited in concession lines and how long it took to prepare their food. The Braves then used this research to streamline the process.

Georgia Tech's Interactive Media Technology Center (IMTC) is using augmented reality to give fans an in-depth look at SunTrust Park and Braves players. A high-resolution camera feeds a live view of the entire baseball field to a computer located in the Verizon Lounge. Using a 55-inch touchscreen, lounge visitors can explore the stadium and the live game by interacting with augmented reality (AR) graphics integrated into the camera video feed. Interactive AR graphics include showing the current lineup on the field and individual player statistics. They can also explore SunTrust Park and read little-known facts about the ballpark. Georgia Tech and the Atlanta Braves will launch the augmented reality experience later this spring.

“We’re so lucky to have one of the foremost research and technology universities in Georgia Tech right in our backyard," said Mize. "So it made perfect sense to align with Georgia Tech and form that partnership.”

Scott Robertson, research scientist and IMTC associate director, is envisioning the future of the project. “Right now this is a standalone system so all of the content is static, but the next version could include live information sources,” he said. “One thing we’re interested in doing is, when a player comes up to bat, rendering a transparent heat map on the field which shows where they’re statistically likely to hit the ball.”

Robertson and IMTC researchers are also exploring streaming AR content to mobile devices and installing more cameras for additional views around the ballpark.

* * *

Researchers are also making sports more engaging for a community that venues have historically overlooked: the deaf and hard of hearing.

Captions for Life is a real-time, personal captioning system developed by Leanne West, a principal research scientist at Georgia Tech who has a background in optical physics, and Ethan Adler, a former computer scientist at Georgia Tech. It works by collecting captions on a server in one of three ways—through voice, typing them in real-time, or pre-recorded text. The captions are then sent out via wi-fi or cellular service and displayed on a smartphone screen or on electronic eyewear like Google Glass.

Initially designed for movie theaters, Captions for Life is now meant for use in classrooms, churches, and sports venues. “The idea is that wherever you are at a sports venue, you can get the color commentary to whatever degree they're doing it at the stadium, and it goes with you. You are not tied to a seat or screen,” said West. Using multiple channels, venues can also provide captions in different languages.

West and Adler developed the system in 2000, and the following year received funding from the National Institute on Disability and Rehabilitation Research (NIDRR) through Georgia Tech’s Rehabilitation Engineering Research Center for Wireless Technologies (Wireless RERC). Back then there weren’t many captioning options for the deaf and hard of hearing, but according to West things have changed over the past 15 years. “Now, people are much more aware. Venues are starting to be more proactive, so we think that’s a good sign.”

Measuring Joint Sounds

As a former student-athlete, and now a researcher, Omer Inan has unique insight into the physical challenges that athletes face and how technology can address them. The All-American collegiate discus thrower joined Georgia Tech in 2013 as an assistant professor in the School of Electrical and Computer Engineering (ECE).

“I had this interest and passion for allowing people to get better information about their health and possibly their performance—from an athletic standpoint—outside of clinical and lab settings,” said Inan.

He had an idea to use his background in audio engineering to develop a knee brace that measures the popping and cracking sounds inside joints. Sensors are attached to the skin, while two types of miniature microphones detect sound. One catches soundwaves in the air, and the other senses vibrations on the skin’s surface in response to acoustic energy inside the body.

Thanks to funding from DARPA’S Warrior Web program to develop wearable technologies for soldier rehabilitation, and the Smart and Connected Health program from NSF and NIH, Inan along with collaborators in ECE and the College of Sciences are building better, less expensive microphones and sensors. They’re also examining what the sounds mean for determining if joints need further rehabilitation.

“For example, if we inject saline into the knee capsule to mimic swelling, how does that impact the sounds?” explained Inan. His team is conducting cadaver studies, including performing small meniscal tears, which simulate one of the most common knee injuries in athletes.

Inan recently expanded his research to study more complex joints—elbows and shoulders—and is working with the Georgia Tech baseball team to take measurements from pitchers and infielders throughout the season. His team is trying to determine markers of joint overuse that could be used to predict the risk of overuse injury.

More broadly, this research could also impact people living with arthritis or those who are just trying to stay active. “Maybe we're recreational tennis players, or maybe we go for a jog, and we want to understand if we should back off that day. I think that having in-depth physiological information about joint health with non-invasive sensors would be a huge win.”
 

Protecting the Brain

A bump or blow to the head that rattles the brain and disturbs normal function is known as traumatic brain injury, or TBI. It’s a complicated injury, with a wide range of symptoms from a headache to memory loss and even seizures. In athletics, from youth to professional, mild concussion is the most frequent type of traumatic brain injury.

At the college level, the NCAA and the U.S. Department of Defense have embarked on the largest-ever study of concussion in sports with the goal of better understanding and preventing these injuries. Preliminary data from the CARE Consortium study, released in January, indicates that attitudes about concussion among student-athletes, medical providers, and athletic programs have changed over the past 17 years. Campuses are emphasizing adequate recovery time, which could result in fewer repeat concussions.

In 2016, Georgia Tech President G.P. “Bud” Peterson became chair of the NCAA Board of Governors, the NCAA’s highest governing authority. Last April, the board increased its emphasis on the importance of safeguarding student-athlete health and safety by endorsing nine areas that the NCAA’s Sport Science Institute has identified as priorities, including concussion, cardiac health, and mental health.

“The NCAA has established the health and safety of its athletes as one of the Association’s top priorities,” said Georgia Tech President G.P. “Bud” Peterson. “We’re already learning a great deal from the concussion study launched by the NCAA and the U.S. Department of Defense in 2014. In addition, we’re addressing other areas that impact health, well-being, academic and athletic performance. The NCAA is taking a holistic approach. It reminds me of the way we bring together expertise in multiple disciplines at Georgia Tech. Our internal and external collaborations many times result in breakthroughs that wouldn’t have been discovered otherwise.” 

Dr. Peterson has invited the NCAA’s first chief medical officer, Dr. Brian Hainline, to visit Georgia Tech in the fall to explore collaborations between Georgia Tech, the Centers for Disease Control and Prevention, and Children’s Healthcare of Atlanta in the area of concussion research.

*  *  *
 

Michelle LaPlaca, associate professor in Georgia Tech’s and Emory's Department of Biomedical Engineering, has been researching TBI for more than 20 years. She says whether the injury is mild or severe, it’s difficult to figure out what's going on in the brain and how that contributes to function. Researchers study TBI in several ways: at the cellular level by mechanically injuring cells and observing how they break down; they also use computer-generated simulations and animal models. There’s always one thing missing, though: how to connect all these levels of research to human TBI.

“You can’t directly measure what happens in the brain of a human during a traumatic brain injury,” said LaPlaca. She calls TBI “a heterogeneous injury.” It manifests itself in different ways—memory problems in one person, a headache in another. Doctors consider underlying health issues, and gender, too. Then there’s the nature of the injury itself. Was the athlete hit from the front or the back? Were they wearing a helmet? LaPlaca examines all of these variables and how they contribute to the brain injury.

“Part of the motivation is to better understand those tolerances so that we can design better protective gear. Protective gear like helmets are designed to guard against skull fractures, but can't prevent movement of the brain within the skull.”

She also says there’s a need for more tools that can quickly and objectively assess concussion. LaPlaca and David Wright of Emory University have developed DETECT or Display Enhanced Testing for Cognitive Impairment and Mild Traumatic Brain Injury. After a suspected concussion, an athlete puts on a head-mounted display and earphones, which reduces outside distraction and creates a more immersive test-taking experience. Using a controller, they take a shortened neuropsychological test measuring memory and reaction time, as well as balance and eye movement. By covering multiple neurological domains, DETECT can better guide clinical management of the concussion. The assessment results also take into account off-the-field factors like learning disabilities and medications.

Georgia Tech researchers have tested DETECT with high school and college football players and find that it is accurate in distinguishing concussed players from those without a head injury. LaPlaca emphasizes, though, that DETECT is meant to enhance the knowledge and expertise of athletic trainers, not replace it.

“[Athletic trainers] have a lot of experience, and we don't want to dismiss that,” she said. “We want to figure out a way to integrate the human experience into our algorithms for decision making.”

She says although academics and athletics don't typically work together, “there’s a lot of willingness on both sides” to work together to advance innovative ideas that benefit athletes on and off the field.

“It’s not just a return to the field. It’s also a return to life, and a return to the classroom.”


Graphics by: Raul Perez

“We forget that they’re cognitively different, they’re physically different, their mental model of how technology works or how the world works is really different,” said Gandy. “They didn’t understand things that seem obvious to an adult - like why the train didn’t go the same distance every time no matter how much stuff you put in the bed of the train - because they don’t have an intuition about physics the way that older kids or adults do.”

Another hurdle is the handheld aspect. Researchers used a tablet for testing, but discovered that some of the younger kids struggled to hold it because they have small hands.

Gandy and her team will now take the data they collected, make tweaks and eventually make the app available to the public. They’ll also create physical and virtual “exploration kits” for students in Verizon’s Innovative Learning Schools and other schools across the country. Augmentations will be personalized to each student, with grade-appropriate content.

“The opportunity for today’s students to better understand complex STEM concepts is so important,” said Kristin Townsend, Manager - CSR Program Development, Verizon Foundation.

“Leveraging technology to help students not only understand but see everyday examples of STEM principles in their lives not only facilitates learning, but it inspires students to consider STEM fields that tend to be very abstract.”

• • Annual Open House - The consortium will hold an annual open house every November. Events will include workshops, hands-on tutorials for developers, and a special wearable industry day.

• • Technical Consulting - Members of the consortium have the ability to access an interdisciplinary group of research scientists who will provide technical consulting to help guide a company's internal wearable projects. A portion of the membership fee is used as a retainer to cover researcher’s time. (WATCH: Georgia Tech Researcher Rob Solomon demonstrates how the Google Glass app, Captioning on Glass, works.)

• • Collaborative Projects - The consortium may operate as a vehicle for collaborative projects among members.

• • Impact Industry Standards - The consortium will develop standards for wearable devices. Interoperability and protocol standards will be necessary to achieve interoperable wearable devices.

• • Networking Opportunities - Quarterly meetings and the annual open house will provide networking opportunities to consortium members.
  • User Testing - Members may engage consortium researchers to perform user testing of their internally developed devices.

"The program helps companies understand the potential of wearable computing by keeping them abreast of the past, present, and future of wearable technology, policy, and applications," said Gandy. "We will work with them to identify how wearable technologies can be applied to their business and help their organization gain the expertise needed to make these ideas a reality."

Support levels for the Industry Partners Program are designed to attract both small and large companies. Contact Maribeth Gandy at maribeth@imtc.gatech.edu or Peter Presti at peter.presti@imtc.gatech.edu for more information.

“My hope is a new approach to data-driven medicine,” said Zutty. “All of this data is out there and we can learn and take action from what the data tells us. If you have different patient descriptors you can look at how different courses of treatment affected similar patients. This enables a physician to make much better, more confident treatment plans.”

AGL President and CEO Marcel Sarzen, who is also a Georgia Tech alumnus, called the animal behavior algorithms the company’s “special sauce.”

“I always had this vision that I could go down to Georgia Tech, and with all of the great capabilities that they had down there, license some sort of intellectual property and bring it into one of my ventures that I was going into. And that became true here with AGL,” he said.

AGL Vetrax is expected to become more widely available to veterinary clinics later this year.

“The awards ceremony honored the top tier of the computing field, including the two Turing award winners. It was an amazing event and I’m deeply honored to be recognized as an ACM Fellow,” said Mynatt.

Mynatt established her Everyday Computing Lab at Georgia Tech in 1999, which investigates emerging interaction techniques, research methods, and applications that are compelling and effective in a world where computing technologies are ubiquitously available yet integrated into the social fabric of everyday life. Mynatt’s focus on everyday computing requires understanding people in the context of their everyday lives, so she often assembles multidisciplinary teams with psychologists, designers, and healthcare workers.

Learn more about Mynatt’s work at Georgia Tech and with other nationally and internationally-recognized organizations

Mynatt showed off a wearable tech wrap at the awards banquet that was designed and built by Ceara Byrne, Jessica Pater, and Clint Zeagler of the Georgia Tech Wearable Computing Center. Mynatt asked to wear the garment in order to highlight Georgia Tech's work in human-centered and wearable computing.

Playfully dubbed "Professor on Fire," the wrap includes a detachable modular structure made up of aluminum rods. The structure's honeycomb shape is a nod to the Georgia Tech Yellow Jackets. Zeagler designed and created the structure that housed LED lights and a microphone. Byrne programmed the lights to provide a personal spotlight and react to sound captured by the built-in microphone, while Pater created the textile component, knitting a shawl out of black raw silk.

"Combining the shawl with a wearable tech device beautifully blends together traditional and modern handicrafts," said Pater. 

ACM recognized 42 of its members for their contributions to the development and application of computing in areas from data management and spoken-language processing to robotics and cryptography. Additional information about the 2015 ACM Fellows is available on the ACM website.

They found study participants were consistent in their responses to each type of assessment, which shows that game mechanics didn’t interfere with a player’s answers. They also discovered that responses in the game corresponded with specific personality traits. For example, game players who collided with the most jellyfish were found to be more extraverted and less agreeable.

A second smaller study by Georgia Tech and ACT used a video game emphasizing multitasking to examine player action and behavior. Researchers then compared the results to three traditional assessments, including ACT Engage, which measures ten factors that have been shown to accurately predict academic performance and persistence in college students. Once again they found connections between game play behaviors and scores from the traditional assessments, such as the relationship between the time students spent reviewing their scores in the game and low self-confidence.

Researchers can use the results of these studies as a roadmap for developing future game assessments.

"Engagement is only one of the potential benefits of game-based assessment," said Maribeth Gandy, co-author of both studies and director of the Interactive Media Technology Center and the Wearable Computing Center. "Yes, playing a game can be much more fun and motivating than taking a traditional stress-inducing standardized test. But we believe the benefits can go far beyond that. A game environment allows us to create much more sophisticated scenarios where the test-taker 'answers' via rich interactions and complex decisions rather than picking an answer from a multiple-choice list. We hypothesize that by leveraging game design along with the complex science of test design, we can create more valid assessments and that we can assess many more dimensions of human ability which are increasingly relevant to the school and career success in our modern world."

Gandy said another possible benefit of gamification is the creation of an online resource that students can return to as they evolve. “What ACT is interested in for the future is, rather than a gatekeeper to get to the next level of education, the assessment is about self-enrichment and empowerment.”

This work is supported through grants from the National Science Foundation and ACT, Inc.

CITATION:
Maribeth Gandy, et al., Grouches, Extraverts, and Jellyfish: Assessment validity and game mechanics in a gamified assessment

Maribeth Gandy, et al., Actions Speak Louder Than Words: An exploration of game play behavior and results from traditional assessments of individual differences

“The garment Katherine and I have worked on together will aid in audience interaction," said Zeagler. "Specifically, the audience will physically touch it during her performance, creating an engaging experience. The garment is called Le Monstre and it's a jumping, frolicking, happy thing that lights up, and helps control the stage experience through touch.”

Characters debuts at the Ferst Center for the Arts on March 5th at 5:00pm.

Partners on Creative Collisions include the Georgia Tech Wearable Computing Center, the Interactive Media Technology Center, and the Office of the Arts. The project is funded by IPaT as part of our Research and Engagement Grants Program. Each year IPaT and the GVU Center support research initiatives committed to building on our success in interdisciplinary research and innovation in the human experience of computing. Research Grants provide seed funding to conduct interdisciplinary research, while Engagement Grants are designed to foster new sorts of engagements and collaboration, whether internally or externally.

Purchase tickets for Characters

Beyond Gaming

Medical error is the eighth most common cause of death the U.S., with up to 98-thousand patients dying each year from preventable mistakes. While teamwork and communication skills are crucial in reducing these errors, interdisciplinary teams of healthcare staff that work together in hospitals are often trained separately. Another problem: training focuses on power differences between healthcare workers.

While augmented reality has been used in medical technical skills training, it hasn’t seen widespread use in nontechnical and team training. Georgia Tech researchers are working with faculty, residents, and students from Emory School of Medicine’s Department of Anesthesiology to develop an AR scenario that puts operating room trainees in each other’s shoes.

“A lot of the bad outcomes in a surgery are because of communication breakdown,” said Maribeth Gandy, director of Georgia Tech’s Interactive Media Technology Center and Wearable Computing Center. “What we want is an environment where you can become the surgeon and see the surgery from their point of view.”

Participants in the simulation acted out emergency situations in a real-life operating room where they were able to see virtual co-workers and hear their inner thoughts. It’s all about empathy according to Gandy. “That guy looked like a jerk, but when I’m in his mind I’m realizing that he’s freaked out because the patient has been under anesthesia for too long.”

After completing the scenario, participants said they saw the potential of augmented reality training for practicing new medical skills, crisis and code simulation training, and improving reaction time in critical situations. They also believed that compared to traditional training, AR could better simulate the emotional nature of clinical scenarios. Researchers are now working toward full mission simulation—realistically simulating an entire medical intervention with a full healthcare team in the operating room.

                                                                 *  *  *

Gandy and IMTC are also working with Georgia State to use augmented reality in social psychology research. They’re trying to figure out what leads people to engage in either peaceful protests or violent attacks, specifically, terrorist acts.

For years, Georgia State has used online text-based scenarios to collect data from study participants, but they wanted to administer more realistic experiments. So researchers collaborated with a screenwriter and filmed scenes with professional actors to create an AR experience. Through video montages and first-person narration, participants took on the persona of a member of a fictional oppressed minority group. They also listened to dialogue from two virtual actors who try to persuade them to join a peaceful, student-led protest, or join a violent resistance movement.

“You can imagine this AR experience being applicable to all kinds of scenarios,” said Gandy. “It’s really relevant in our current world where we’re having a lot of cultural conflict.”

GSU and Georgia Tech will soon start a study using the augmented reality system, and will compare the results to the online version.

Creating Augmented Reality Experiences

Using AR to present educational content has been around for years. It’s shown to be effective and compelling, and at least one school in the Atlanta area has adopted the technology in its kindergarten class. However, designing successful educational content for AR is another story. It requires input from a diverse and essential group of people—teachers, students, technologists and artists.

Three years ago, Georgia Tech and the Verizon Foundation began a partnership to explore the elements of good AR design for kids in school. Using Georgia curriculum guidelines, researchers developed a STEM-related augmented reality application, and built and tested six different tablet-based AR prototypes based on physics concepts. Some of the prototypes are very hands-on—launching a catapult or manipulating a cube—while others have more virtual content. Other design choices include group vs. single user, and game elements vs. no game mechanics.

After testing the prototypes with students and teachers, researchers realized there were some factors that they missed—how AR fits into the classroom environment, the possible overstimulation of kids, and their physical limitations. During the testing sessions it became clear that young students, with their small hands, couldn’t hold the tablet for long periods of time.

Researchers will use the knowledge they gained to create better educational AR experiences. It’s also knowledge that can be shared. “The impact of this kind of project is this body of knowledge that can inform game designers who want to build AR for kids, or teachers wanting to understand how these things might be integrated into the classroom,” said Gandy.

                                                                 *  *  *

Georgia Tech is also making it possible for people to create augmented reality experiences using web technologies. Argon, an augmented reality web browser, is on its recently-released fourth iteration. The goal of the project is to make AR accessible to the widest possible group of developers and users through a web-centric platform.

Blair MacIntyre, professor in the School of Interactive Computing and director of the Augmented Environments Lab for over a decade, started working on Argon in 2009.

“We started thinking about—from a research perspective—what are the things that both makes the web a unique and powerful platform for doing AR?” he said. “With the web ecosystem, the fact that you don’t have to build and deploy apps is a unique capability that makes it easier to produce augmented reality experiences.”

Using the web can also be more secure and cost-effective than building an app, and easier for people who don’t have a lot of development experience.

MacIntyre and Gheric Speiginer, Georgia Tech PhD student in human centered computing, are also exploring how Argon can support users running multiple independently-authored AR applications at the same time, and across different platforms such as mobile phones or head-worn displays. “In the future, augmented reality is going to only really be useful—especially for consumers—if you can see virtual content from a lot of applications or services at once,” said MacIntyre.

The team is working toward making AR an add-on feature for webpages, similar to adding video or a social media button. This opens up more possibilities for developers and consumers, like taking an augmented reality tour of Georgia Tech even if you’re not physically on campus.

The Future of Augmented Reality

Scientists will soon be able to use augmented reality in an unexpected way - to support their research. MacIntyre, Speiginer and Georgia Tech College of Engineering Professor Magnus Egerstedt are developing a tool that would allow scientists to debug their robots.

“We didn’t set out to use augmented reality as a debugging tool,” said Egerstedt. “But now, in a very concrete way, we can use it to see what the robots are thinking and why they’re not acting right. This has been extremely powerful and a little surprising.”

Using a smartphone with the AR debugging interface, scientists can see things like a robot’s sensor measurements, power levels and communication functions. The interface is currently only available to Georgia Tech researchers, but Egerstedt expects to make it more widely available this year.

The interface is a tool in the Robotarium, a one-of-a-kind lab that allows roboticists from around the world to remotely conduct experiments on over 100 ground and aerial robots located at Georgia Tech. They upload their own programming code, watch the robots in real-time via livestream and receive data of the results. To date, over 200 researchers, including those from universities in the United States, Japan, Canada and England have used the lab.

                                                                 *  *  *

Phones and tablets have been the entry point for consumers using augmented reality, and there are already a multitude of available AR apps, from games and tours to virtual car maintenance. Head mounts like Microsoft’s HoloLens could be the next wave for consumers, but MacIntyre says developers first have to ask, what are the things that we would want access to all the time? He believes vertical market applications will come first, perhaps with hospital workers or delivery drivers using head mounts.

“It’s easy to imagine wearing a HoloLens for a few hours here and there as part of your job if it’s giving you some very specific benefits. And then once people get used to that, we’ll start seeing the add-ons.”

Vu Ha agrees. Ha, a software engineer and Georgia Tech graduate in Computational Science and Computational Media, has created augmented reality experiences for independent game studio Funomena. He thinks that the next AR milestone will be replacing mobile phones and monitor-based laptops and desktops with virtual displays.

“Eventually we won’t have a giant machine that we do most of our work on. Instead, we’ll have virtual displays that are all around us any time we want. And this affects every part of software, from games to enterprise to education – anything that’s currently digital,” said Ha.

Removing the hardware could also create more socialization in computing, says Ha, an advantage that augmented reality has over virtual reality.

“Virtual reality technology is very singular; the person in the headset is really the only person enjoying the experience. While ideally, augmented reality allows you to interact with other people, see their faces and communicate directly.”

Privacy and security may soon be a big concern with augmented reality. Not only do AR devices have cameras, they also build 3D models of your space. Also, what are app makers doing with the data collected from your device’s camera? “Over time, this is going to be a much bigger issue,” said MacIntyre. “Not a huge nightmare issue, just something we have to be aware of and help consumers understand, just like other security issues on the web.”


Photos by: Joshua Preston, Rob Felt
Videos by: Ashton Pellom, IMTC
Graphics by: Raul Perez