Overmatching, Undermatching, and Change-Over Delays

At its discovery, matching was hailed as a great leap forward, an example of a relatively complex human behavior described by a mathematical equation, akin to physics equations describing the behavior of elementary particles. However, it was quickly discovered that humans and animals often deviated from the nice straight line described by the Matching Law. In some situations, participants overmatched, giving more weight to the richer option and less to the leaner option than the equation would predict. In others, the participants undermatched, treating the various contingencies as more equal than they actually were.

Neither of these tendencies is especially bad for game design, in small quantities. As long as the players are exploring different options and aren't bored, we don't usually care how much time they spend on each. Extreme undermatching implies the player isn't really paying attention to the merits of each option. Overmatching can mean that the player has chosen an option for reasons other than merit, such as enjoyment of the graphics.

Fortunately for behavioral psychology, these deviations could be predicted and controlled. One important factor in determining how closely participants match is the amount of time and/or effort required to change between options. The farther apart the options are or the more work is required to switch between them, the more players will tend towards overmatching. For example, imagine a typical first person shooter game, in the vein of Quake or Unreal. If switching from their current gun to a different one has a delay of 20 seconds during which they can't fire, they'll switch from one to another less often than they would otherwise. Even if the current gun isn't perfect for the current situation, the changeover cost might keep the player from switching. If the delay is long enough, switching can become non-existent as the costs outweigh any possible benefits.

At the other end of the spectrum is the case where changeover is instantaneous. Consider a massively multiplayer game where monsters spawn periodically in various locations. Switching between multiple spawning sites normally takes time, but suppose a player could teleport instantly from one to another with no cost. The best strategy would be to jump continuously back and forth, minimizing the time between the appearance of a monster and the kill. That makes sure the player gets as many points as possible in a given period of time.

Obviously, neither of these extremes is really desirable for game designers. Ideally we want to be able to adjust the time/difficulty/expense of changing strategies to strike just the right balance exploration and exploitation. What that balance is has to be an individual choice, the concept of a change-over delay is just a tool for achieving that balance.

Another important factor players consider in choosing between alternatives is risk. Game theory says that players should weigh the options such that they'll maximize overall reward in the long term. For each alternative, they should multiply the possible reward by the odds of receiving that reward and choose the best option.

However, this article is concerned with what players actually do, not what they mathematically should do. Psychologists generally use two terms to describe how subjects react to risky situations. Subjects are risk-prone when they prefer the more uncertain alternative and risk-averse when they tend towards safer options. In one experiment, pigeons were offered a choice between two keys to peck. The left provided 8 pieces of food every time, the right provided 16 half the time and no food half the time. The pigeons consistently preferred the more reliable schedule, and were therefore risk-averse. In a later study, the left key produced 3 bits of food every time while the right key produced 15 one-third of the time. In this study, the pigeons preferred the riskier alternative.

So far, this is perfectly in accord with game theory, with subjects taking risks when those risks offer an overall greater chance of reward. But what about the example mentioned earlier in this article, where subjects preferred a variable interval schedule to a fixed interval schedule? Even when the two options provided equal rates of overall reward, subjects preferred the probabilistic option. The difference lies in the expected outcome of each individual response. In the pigeon experiment we just described, each choice was discreet. A peck, an outcome, and the subject was presented with a fresh choice. Each choice contained the totality of possible outcomes, so the subjects' behavior reflected the total contingency.

In the fixed-interval / variable-interval experiment, one could respond any number of times on the fixed interval option but would not receive the reward until the interval had elapsed. On the variable interval schedule, every single response had a small chance of being rewarded. Therefore, there was always a reason to try the variable schedule, but only occasionally a reason to respond on the fixed schedule. The subjects were responding to the proximate outcomes, rather than the overall outcomes. This is an example of how subtle changes in the schedule can cause drastic changes in behavior. Whenever we provide players with rewards, we're creating a schedule of reinforcement that will influence them to behave in particular ways. Because we can't avoid these effects, we have to understand them so that they can be made to work for us, rather than against us.

Odysseus' Choice

One factor we haven't addressed yet is when the decisions are made. Many of the choices we make in games don't have immediate effects, only helping or harming the player minutes or hours down the line. A character might have to choose whether to take a potion that gives them extra strength now or save it for later play. A player in a tank combat game might choose a fast, lightly armored tank rather than a slower, better protected one. Not all choices are followed by immediate consequences, and this delay often distorts the player's perception of their options.

Take the situation where a person has two possible options, each with a different level of reward. For example, a person might choose between receiving one piece of candy or two pieces of candy. If the delays are equal, the person would naturally choose the one with the larger reward. However, as the delay to the lesser reward decreases, the relative value of that reward starts to rise. If someone is offered one piece of candy right now compared to two pieces next year, most people would probably choose the more immediate reward.

Because he wanted to hear the Sirens but also make it home alive, Odysseus ordered his crew to tie him to the mast and to plug their ears.

A practical question arising from this research is under what circumstances do people tend to make more accurate decisions. One of the answers that psychologists have discovered has a parallel in an ancient Greek myth, Odysseus and the Sirens. Odysseus knew his boat was about to sail near the place where the Sirens were singing and that anyone who heard them would throw themselves into the sea in a vain attempt to reach them. Because he wanted to hear the Sirens but also make it home alive, he ordered his crew to tie him to the mast and to plug their ears with beeswax so they would not hear the call. In this way, his ship sailed safely past, his crew unhearing of both the Sirens and his pleas to be untied.

Because he made the decisions at a long delay from both outcomes, his choice was a good one. If he'd waited until the Sirens were right there and had to choose, his decision would have maximized the short term happiness of listening to their song over the longer term reward of making it home alive.

More generally, the more distant all of the outcomes are, the more people's choices tend to maximize long-term success. Of course, you may not want players doing deep long-term thinking. It's up to the designer what's best for his or her game, whether to skew the players towards one option or another, towards one strategy or another. Delays between action and outcome are just one of the tools available to influence how players choose.

To explain every choice a real human being makes would take a model as complex as the human mind. Psychology cannot offer use that yet, but it can give us rules of thumb and general patterns of choice that can describe a generous portion of what we do when presented with multiple options. Every game offers its players a sequence of choices, each with attendant consequences for choosing wisely or poorly. By understanding some portion of the rules that govern how human beings react to those choices, we can design games that elicit the kinds of choices that make the game a more enjoyable experience for the player.

||||     By Jacob Habgood [Author's Bio] Gamasutra August 7, 2006 Compulsory Game Development for Everyone Introduction Game Literacy Benefits   Change Login/Pwd Post A Job Post A Project Post Resume Post An Event Post A Contractor Post A Product Write An Article Get In Art Gallery Submit News   Logged in as: Ralph Noble No resume hosted. [Upload Resume]   Latest Letters to the Editor: RE: Persuasive Games: How I Stopped Worrying About Gamers And Started Loving People Who Play Games by Steven An [08.03.2007] Latin American Game Development by Patrick Dugan [07.26.2007] June NPD numbers - PSP software by Carl Chavez [07.24.2007] [Submit Letter] [View All...]    Features Compulsory Game Development for Everyone Children I believe the games industry has a lot to lose from failing to train the game developers of the future. Many of us from the present generation learned our craft from a very early age, on the home computer systems of the 80’s. We were able to do this because these computers came with their own novice programming languages and editable example games to help get us started. Unfortunately this is no longer the case, so future generations of developers are actually in danger of having a worse grounding than our own. Hopefully tools like Game Maker can fill this gap and make sure that there is still a vibrant hobbyist community to nurture young talent. Nonetheless, if we want to harness the best potential talent from future generations, then children need to be given a chance to try game development for themselves in clubs and schools. I’m married to a teacher, so I’m well aware that the last thing they need is another compulsory topic to add to their workload (the games industry may have a reputation for long hours, but I can assure you that many teachers put in just as many!). However, game development is such a multi-disciplinary field that it is can often provide a great way of meeting existing curriculum requirements in a fun and engaging way. For example, here in the UK there is a “Command and Control” ICT curriculum for 7-11 year olds, which is often taught through activities involving computer-controlled traffic lights. My wife has recently developed her own ‘scheme of work’ for teaching the same objectives by making computer games with Stagecast Creator (see the resources page of Gamelearning.net). Other projects are trying to bring game development into the classroom as a way of inspiring storytelling and traditional literacy skills. There are also numerous clubs and holiday camps around the world that are already successfully teaching game development to children outside of the school curriculum. “Nile The Quartz Crystal”: If you want an original idea – ask a seven year old.