The path to college and a career can be a complicated journey for young students. 17-year old high school senior Jordan Harris has grappled with questions of whether to join the military after graduation, or go to college and study aerospace engineering.

“Through JROTC I was exposed to a lot of opportunities dealing with the military, and recruiters make it enticing to want to join,” he said. “But I love the idea of being able to create and invent, and most of my family has gone to college so that’s really the only route I know.”

Multiple choice assessments measuring personality, interests and abilities have helped students like Harris narrow down their choices, but tests can be stressful. “I get extremely nervous taking tests,” he explained. “If it’s a big test, I usually can’t focus on anything else. It’s a relief when it’s all over.”

Studies show a nervous test taker can produce responses that aren’t a true reflection of their knowledge or traits. Some people might deliberately choose answers that indicate more desirable personality qualities, and traditional testing formats could inaccurately measure characteristics like creativity.

Researchers at Georgia Tech and ACT, Inc. are now exploring video games as an alternative to conventional assessments. In a study of more than 200 people presented at the recent DiGRA-FDG Conference in Dundee, Scotland, researchers looked at the differences in results between a traditional personality assessment and a custom, gamified version of the same assessment.

They found study participants were consistent in their responses to each type of assessment, which shows that game mechanics didn’t interfere with a player’s answers. They also discovered that responses in the game corresponded with specific personality traits. For example, game players who collided with the most jellyfish were found to be more extraverted and less agreeable.

A second smaller study by Georgia Tech and ACT used a video game emphasizing multitasking to examine player action and behavior. Researchers then compared the results to three traditional assessments, including ACT Engage, which measures ten factors that have been shown to accurately predict academic performance and persistence in college students. Once again they found connections between game play behaviors and scores from the traditional assessments, such as the relationship between the time students spent reviewing their scores in the game and low self-confidence.

Researchers can use the results of these studies as a roadmap for developing future game assessments.

"Engagement is only one of the potential benefits of game-based assessment," said Maribeth Gandy, co-author of both studies and director of the Interactive Media Technology Center and the Wearable Computing Center. "Yes, playing a game can be much more fun and motivating than taking a traditional stress-inducing standardized test. But we believe the benefits can go far beyond that. A game environment allows us to create much more sophisticated scenarios where the test-taker 'answers' via rich interactions and complex decisions rather than picking an answer from a multiple-choice list. We hypothesize that by leveraging game design along with the complex science of test design, we can create more valid assessments and that we can assess many more dimensions of human ability which are increasingly relevant to the school and career success in our modern world."

Gandy said another possible benefit of gamification is the creation of an online resource that students can return to as they evolve. “What ACT is interested in for the future is, rather than a gatekeeper to get to the next level of education, the assessment is about self-enrichment and empowerment.”

This work is supported through grants from the National Science Foundation and ACT, Inc.

CITATION:
Maribeth Gandy, et al., Grouches, Extraverts, and Jellyfish: Assessment validity and game mechanics in a gamified assessment

Maribeth Gandy, et al., Actions Speak Louder Than Words: An exploration of game play behavior and results from traditional assessments of individual differences