Review: Digital Games, Design, and Learning A Systematic Review and Meta-Analysis

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital Games, Design, and Learning: A Systematic Review and Meta-Analysis. Review of Educational Research, 86(1), 79–122.

Abstract: In this meta-analysis, we systematically reviewed research on digital games and learning for K–16 students. We synthesized comparisons of game versus nongame conditions (i.e., media comparisons) and comparisons of augmented games versus standard game designs (i.e., value-added comparisons). We used random-effects meta-regression models with robust variance estimates to summarize overall effects and explore potential moderator effects. Results from media comparisons indicated that digital games significantly enhanced student learning relative to nongame conditions (g⎯⎯g¯ = 0.33, 95% confidence interval [0.19, 0.48], k = 57, n = 209). Results from value-added comparisons indicated significant learning benefits associated with augmented game designs (g⎯⎯g¯ = 0.34, 95% confidence interval [0.17, 0.51], k = 20, n = 40). Moderator analyses demonstrated that effects varied across various game mechanics characteristics, visual and narrative characteristics, and research quality characteristics. Taken together, the results highlight the affordances of games for learning as well as the key role of design beyond medium.

Study included 69 studies in 70 reports from 68 journals with a total of 6,868 participants. 

Type of Study: Meta-Analysis

Terms highlighted in the article: 

Computer games are interactive, based on rules and constraints, directed toward a clear goal often set off by challenges. They also provide constant feedback, so players can monitor their feedback.

Serious Computer Games are those same games, but the objective is not to entertain (which is an added bonus), but to use as training, education, health, public policy, and strategic communication tools. 

Story Relevance is how close the narrative is to the actually work or gameplay. (more below)

Story Depth refers to the complexity of the story. (more below)

This meta-analysis focused on examining certain game moderators to see if they enhanced learning. They looked at Play Duration, Additional Instruction, Player Configuration, Sophistication of Mechanics, Variety of Player Actions, Intrinsic Integration, Scaffolding, Visual Realism, Anthropomorphism, Perspective, Story Relevance, Story Depth, and Contextualization. 

For my purposes at this time, I highlighted the narrative components of this meta-analysis.

The authors looked at both story relevance (SR) and story depth (SD). SR refers to how close the narrative is to the actually work. For instance, a math game where the student has to shoot zombies in between answering questions would have very little story relevance. However, a science game where students have to complete science related objectives in a certain period of time to score points may be considered more relevant. Surprisingly, they found no different in learning results between the games with irrelevant storylines and those with closely tied narratives. 

SD refers to the complexity of the story. Thin Depth stories have only setting, scenery, and context while Thick Depth stories have a rich evolving story that lasts for the entire game. Again, surprisingly, games with thick SD did not demonstrate significant effects on learning vs. games with no SD or games with less SD.

As a gamer this is surprising and not surprising. I, too, enjoy a great narrative. I've been playing Persona 5 on the PS3. I return to this game over and over because A) I want to see what new trouble I can get into B) the game not only features a narrative, but also choice. I can choose how to proceed in many different ways, which all influence how the narrative unfolds, and C) I am not personally invested in the characters; I want to see how they evolve. On the other hand, I also enjoy games like Pac-Man Championship Edition (No SD) and Cuphead (Thin SD); sometimes I don't need or want a story. In fact, sometimes when I am in the mood to just play and am playing a game with a narrative, I will grow impatient and just skip those parts - especially if the narrative is lame. Take Resident Evil 6 for instance. The story is, to put it simply, really stupid. I could care less, so I skip the cut scenes to get to the zombie hackin' and slashin'. 

Which brings me to my point: Are storylines in educational games any good anyway? These students have played games with some amazing narratives (e.g. Fallout 4, Legend of Zelda, Persona 5, Uncharted, Final Fantasy), so how can a math game storyline even compete? 

The authors do address this later on in the article. 

Research and theory would tell you that rich narratives are a good thing; however, sometimes a rich narrative and visual complexity distracts from the learning objectives. For instance, during my years as a classroom teacher, I have witnessed students playing educational games, but getting caught up in the parts of the game that are not part of the learning objectives. My kids, too, are sent home with information about online math games where they answer some math questions to earn time playing arcade-type games, so they spend 10 minutes doing math and then spend 20 minutes playing the game. Not so good. Also, if a teacher wants to use a popular off-the-shelf game like Assassin's Creed, it better be closely tied to the learning objectives and the students better understand what you want them to do or else they may just trail off to climb all the buildings or instigate fights between animals and characters. 

Also, in the article, the authors note that learning was measured during post tests focusing on lower order outcomes. This goes against what we know about situated learning. We want students to develop a deep understanding that they can use across contexts, which is not reflected in a post test. A student who played through a game with Thick SD should have his or her learning objectives tied to that story somehow or what is the point of including it? So, the problem may not be with the depth or relevance of the story, but how the students were assessed.