Remember the motion control fad? Not long ago Wii Sports became one of the most popular games of all time thanks to how it integrated motion controls into games like bowling, tennis, and other sports games. This led to many developers to think that motion controls would enhance any game, so gamers were expected to flail our arms around like idiots to throw grenades and access menus by pretending to wave at someone crouching on top of our TV. It didn’t exactly stick. Microsoft and Sony have all but abandoned this capability in their consoles and even Nintendo seems to be downplaying it.

Why is that? Why do we hate some motion controls and tolerate or even embrace others? Does this have anything to do with why racing wheels and flight sticks are so great in certain games and why it felt so awesome to rock out with fake plastic guitars in Rock Band? Researchers have been working on these questions under the heading of “natural controller mapping” so let’s take a look.

This kind of thing was NEVER as much fun as advertised.

In 2010, for example, Paul Skalski and his colleagues argued that controllers are “natural” to the extent that they closely match the actions that you perform in the game and allow you to focus on the game instead of what’s in your hand. Swinging a Nintendo Wii controller like a tennis racket, for example, is a very natural control scheme, as is using a driving wheel in a racing game.

The researchers thought that using a keyboard or gamepad to control movement in a game, however, is not nearly as natural. They suspected that more natural controllers were more likely to create a better experience, so they put it to the test.1 They had one group of volunteers play Tiger Woods PGA Tour 07 with either a Wii controller, which required them to swing it like a real golf club. Another group used a gamepad with thumbsticks and buttons. In another experiment they had subjects play the racing game Need for Speed Underground 2 with either a keyboard, a gamepad, a joystick, or a steering wheel. Across all these experiments, Skalski and his pals found that players enjoyed the games more when playing with a more natural controller.

Why? The researchers argue that using more natural controllers made it easier for players to retrieve mental models of how the real world worked, since they were familiar with the devices (a golf club or a car) that those controllers represented. Using button-based controllers or keyboards requires us to do a little mental work to imagine the golf swing or the steering correction. And the less work that our minds have to do in constructing the mental model, the better.

So, that’s great, but I think everyone would agree that the idea that slapping motion controls on every game or asking someone to play Mortal Kombat with a racing wheel is a bit silly. This seems to be a point that was somehow lost on many game developers and publishers during the height of the motion control craze. When the controller and the mental skills one needs to use it to play a given game do NOT fit, gnashing of teeth and muffled screaming result.

Awww yeah! I’m totally shooting some Nazis! *SICK GUITAR SOLO*

So fast forward to 2015 and a paper by Ryan Rogers, Nicholas Bowman, and Mary Oliver.2 All of the subjects in this study got to play a Call of Duty game. Some played Call of Duty 2: The Big Read One for the Nintendo GameCube (with GameCube controller) while the other played Call of Duty 3 for the Nintendo Wii (with the Wii remote). The GameCube game had no motion controls, but the Wii game did. 3 In the Wii game, players had to aim at on-screen targets by pointing the controller at the screen, then press the trigger to zoom in on the target, then press the A button to fire.

“Our primary focus was to help explain that the notion of ‘natural mapping’ has to be considered in relation to what is being done,” one of the authors, Nicholas Bowman, told me via e-mail. “Developers (especially in the early Wii development phases) seemed very quick to assume that natural mapping meant motion controllers, and that motion controllers meant better games.”

The idea was that while this kind of controller mapping works fine when it’s very natural and the skills one needs to use the controller mesh well with the game (e.g., a racing wheel or a ping-pong paddle). But it falls flat when the skills needed to work the controller don’t map to what’s needed to excel at the game. To test this, Rogers, Bowman, and Oliver proposed that the degree to which a controller was “naturally mapped” to a first person shooter could be quantified like this:

Taken from Rogers, Bowman, and Oliver (2015).

“We used the six cognitive skills in this study as they had been found to be related to first-person shooter in my past research on Quake,” says Bowman.4 “This is what we found interesting in that the ‘natural mapped’ Wii controlled didn’t seem to tap the same cognitive skills as the traditional game controllers” That is, a controller for a game like Call of Duty is naturally mapped to the extent that performance is affected by the cognitive skills you see on the left. Performance then leads to frustration with the controls and enjoyment of the game (which is also affected by frustration). Taken together, the relationships between all these things describes how natural a controller feels to a given person and how much they like it.

To the extent that players’ success with a game using a given controller requires 3D rotation, hand/eye coordination, etc., the correlations5 between these abilities and performance define how well the controller is mapped. “The skills we measured were really related to how one interfaces with a controller to make something happen onscreen,” Bowman points out. “For example in a first person shooter, the user has to have the ability to envision the on-screen content as 3D and then use their controller to navigate that 3D space –this is an example of the 3D mental rotation skill.” And indeed, the researchers here found that the group who played with the Wii motion controls experienced worse natural mapping, worse performance, and more frustration than did those playing with the traditional controller.

I have no idea what’s going on here, but I feel mildly threatened.

But, as mentioned above, they might have found different results if they had compared racing game players using a racing wheel with those using mouse/keyboard. And there might be additional cognitive skills that are important for that kind of game. What this study suggests to me is that user experience researchers should…

1. Create a conceptual model of what cognitive abilities are important for their game
2. Measure those abilities in a group of playtesters
3. Do A/B testing with different control schemes or technologies and see how well they predict performance and frustration.

Easier said than done I know –that’s why I have the easy job. But this kind of approach could have let us entirely avoid a lot of really weird motion control schemes in the recent past.