Performance Metrics

Is a game something that players either win or lose? Or can it be something more intricately constructed?

In 21^st Century Game Design, we defined a game as a toy with some degree of performance. ‘Performance’ is an intentionally ambiguous term; chiefly it refers to a measure of success, but there is a second meaning more applicable to tabletop role-playing games in which it implies theatricality. Ever since writing the book, this idea has been percolating in my head. If there are different ways in which the performance aspect can be measured, there must exist different performance metrics. But what effect does the choice of performance metric have on players?

Let us begin by looking at the most basic, the most well-worn, the (arguably) most tedious performance metric: win or lose. This can be considered a digital metric. Many people consider the definition of a game to be tied up in this digital metric; I personally find that approach too constraining. A tabletop role-playing game certainly seems like a game to me, yet it has no win or lose state. (Some might argue that dying is losing – and yet, the death of a character might be the most narratively satisfying experience for the player).

Let’s consider digital metrics in terms of the chance of success (for a given player):

High chance of success

With a high chance of success, players usually succeed in any given segment of play. Because they mostly succeed, when they lose it can be frustrating. However, if a sense of reward is attached to the win, this can still be satisfying. Solitaire games demonstrate this – players who enjoy these games (often corresponding to Lazzaro’s Altered States key, or Wanderer by DGD1) might not be enjoying the win versus the loss, but rather the process. Short play segments seem to be key to making this work, however.

50-50

In the middle ground, what is the feel of play if the chance of success is largely even? If the segments of play are short, and no progress is loss (that is, ratcheting is in place) it will be similar to the high chance of success, above. If the play segments are longer, and progress is lost, it will be similar to the low chance of success, below.

Low chance of success

Facing little chance of success, the nature of play becomes a struggle. This is fiero country (Lazzaro’s Hard Fun, or Conqueror by DGD1) – the emotional payoff of triumph over adversity. While some players become discouraged facing poor odds, the fiero-seeker appears to thrive on being denied their success. It doesn’t matter that they must tackle the same challenge repeatedly, because all of the frustration experienced on the way to victory increases the depth of adversity, and therefore heightens the emotional payoff of fiero when success is finally achieved.

---

At the opposite end of the scale are what we can consider analogue metrics. In the general case, an analogue metric is a continuum of performance, the most common of which in games is a score. An analogue metric could be one of any number of different measures, including a ratcheted score (such as experience points), a letter grade or similar categorised score, or a completion percentage (of a collection, or of map coverage), but for our purposes we will consider the general case, and mostly pass over the nature of the specifics.

Most games featuring an analogue metric are actually built upon an independent digital metric substructure. That is, the score is not directly linked to success or failure. For example, in Pac-Man, the player’s score does not determine when a level is cleared, only the eating of all the dots matters. However, for the purposes of exploring performance metrics let us imagine that these two elements are intrinsically interrelated. For example, in Pac-Man, the analogue performance metric of ‘percentage of dots eaten’ is intrinsically connected to the transient ‘win’ state of clearing each level.

We can now look at different types of analogue metric by considering how the digital and analogue metrics interrelate:

Win and lose, with a score in between

The presence of a score in this case varies the way players can approach the central play. There is more diversity here than can be explored in so short a space, but two cases in particular are worth examining.

If the win state is distant and hard to achieve, the score may become a player’s chief measure of success, as happened with arcade games before the advent of pump-and-play (which allowed players to complete any game by continuing to pay money). This is not very popular in modern game design.

If the win state is relatively easy to reach, but the scoring metric increases by order of magnitudes in relation to player competence, then the win state becomes the draw for continued play while the player is learning the game, then the score takes over as the dominant metric at a later date.

Bombjack had this essential nature, counting the clearing of a level as a win state. Players who were learning the game could complete some 20-odd levels, but scored little. Expert players scored vastly higher scores completing just 2-3 levels perfectly, because of the huge bonus for collecting the bombs in sequence.

We also see this in NiGHTS: Into Dreams. Completing the game is not a particularly difficult goal to achieve, but scoring big combos then comes to the fore as the player’s chief focus. Players who connected with this challenge came to fall in love with the gameplay because the nature of the analogue metric was vastly more engaging than the simple digital metric of whether or not a level was cleared.

Lose, with a score

Games with no win states have become quite rare, but once upon a time, the arcades where packed full of this style of game. In essence, play was a continuous (and repetitive) experience, and players would eventually fail (die) either because the constant increase in difficulty made this outcome inevitable, or because of human error. In these games, the score is the only metric of interest to the player. This approach lacks appeal, because failure is inevitable, and there is little to draw the player to continue to play unless they happen to hook into the scoring mechanic.

However, there is a particular class of scoring mechanic which is so compelling that it supports tremendous player loyalty. Computer role-playing games are almost universally built upon a leveling structure in which the player earns score towards character progress. Although they tend to have an ultimate goal state, and therefore are not strictly devoid of a ‘win’, the character of the play is such that the win state can be largely irrelevant to the gameplay (especially because the win is usually narrative closure, and therefore separate from the game mechanics). We can therefore imagine these games as instances where the win state is kept from the player, while the lose state (dying) is always a threat.

The nature of play in these games is terribly addictive to the players that enjoy them. Almost everything the player does contributes to the progress of their character or characters, and so the scoring mechanic creates the illusion of constant progression. The deferral of the goal state (the end of the game) creates greater focus on the scoring metrics, and thus locks the player into a near-continuous cycle of progress.

Disgaea is an example of this sort of approach. Players who strongly connect with the play of this game are not generally hooked into the story, and therefore the goal state (the end of the story) is irrelevant to their enjoyment of the game. Instead, they become totally absorbed in the multi-dimensional analogue metrics. The game design is packed with them – even the healing mechanics level up. They have to decide for themselves when to stop playing, because there is nothing that really feels like a definitive end condition. However, the play of a game like Disgaea is abstractly mathematical, and so the appeal is quite low. Nonetheless, players who enjoy the game love it dearly.

Win, with a score

The absence of losing states is quite rare because most game designers do not consider such an arrangement to be viable. The only class of games in which winning without losing has been common was the point and click adventure, which mostly (but not always) included no lose states, allowing players to plug at them until victory was achieved. However, since progress was mediated by puzzles which were frequently arcane and difficult, their appeal had a relatively strict ceiling (hence their decline once development budgets rose above the likely returns for such a game). Even in the case of point and clicks, score was generally absent.

Games organised in this way presumably lack appeal to fiero-seekers, because the sense of struggle is lost. Victory is inevitable. There is therefore little hope of achieving triumph over adversity. Given the fiero-focus of most game development at the moment, this probably explains the lack of games built on this pattern. Nonetheless, there is absolutely no reason that games cannot be built this way.

The games that get closest to this are 3D platformers. Ever since the game development community finally realised that having limited lives was completely irrelevant to the play of these games (it just created frustration without providing any benefit) these games have been built in such a way that the player generally does not have to worry about failing, and just works towards victory with a number of scoring mechanics (often collections of objects) as the backdrop of play.

Sadly, the game design of 3D platformers is now rarely of a high quality. Many of these games contain highly hetrogenous play, such that players are constantly forced to learn new things instead of being allowed to play with expressive mechanics. Variety of mini-games and so forth is considered de rigeur without any thought as to whether building a game out of lots of separate smaller games is a good idea. (Although there are benefits in variety, it is hard for a player to really enjoy a game which waters down its identity through lack of coherent central play). What has really killed this genre however is the loss of interest of the Hardcore players. Lacking evangelists to push these games into the hands of the Casual players who might enjoy them, the genre has floundered, and the more high profile titles (Jak and Daxter, Ratchet and Clank) uniformly have moved towards violent play to re-court the Hardcore, thus losing the Casual audience they once had.

This approach will doubtless reappear. The capacity to win without fear of loss is of great value to a wider audience unconcerned with fiero. The problem is how to get these games to the audiences that enjoy them when the Hardcore evangelists responsible for spreading awareness of games have lost interest. New ideas will doubtless appear to regenerate the 3D platformer at some point.

Score with neither win or lose

It is perfectly possible to have an analogue metric without win or lose states. Games such as Animal Crossing, The Sims are perfect examples: the player is not working towards a goal state, and they are not working to avoid a failure state. They are just playing. In Animal Crossing, the score is the player’s mortgage, then their savings. In The Sims, money once again is the chief metric.

Is it possible to have this form without the scoring metric being money? Probably. But the advantage of money as a central mechanic is that this is an analogue metric that people can enjoy for something more than the simple pride of achieving a high score. Money allows players to purchase things inside the game world, which is rewarding in and of itself.

I suspect this form will become more and more popular as the games industry gradually comes to recognise the value of play in a wider context than just the win and lose mentality which we have inherited from an antiquated conception of what constitutes a game. Sadly, at the moment it seems that publishers don’t actually understand this side of play, and so it is hard to get games of this kind funded, despite the vast commercial success of The Sims.

---

This post has looked in a broad ranging and quite abstract way at the way performance metrics serve to define the play of a game. Before closing, we should look at the best case ways to apply this model to game design. As a crude generalisation, the more performance metrics a game employs, the more choice the player has as to what challenge they are actually undertaking. However, it is possible for a game to have too many performance metrics – complexity is only a draw to a minority of players. We should, for the most part, stick to using metrics that are immediately understandable, except when making cRPGs, strategy games and strat RPGs for a niche market that enjoys complexity.

In the best case, a game wishing to cater for the needs of a very wide audience should be rooted on a performance metric with no lose condition. For instance, in a snowboarding game, the player should always be able to reach the bottom of the mountain. Success of the most basic kind is therefore guaranteed, but there will still be a wide variety of different degrees of performance in the journey down. If you are racing other people, for instance, there is still the challenge of beating the other racers independent of the base-level metric.

Fiero can be achieved without the need for a low chance of success and a win-lose mentality as the underlying framework: in the case of the snowboarding example, it is perfectly possible to present difficult challenges within a framework with no overall failure. Indeed, even if a snowboarding game always allows the player to reach the base of the mountain (a win of the most basic kind) the player will gradually increase their expectations of what they can achieve. Losing a race to another snowboarder will feel like a loss to a player hooked into the challenge of beating them.

By widening our view of what win, lose and score mean, and by layering these elements in hierarchies to support simple freedom to play without failure at the base level, and to support the quest for fiero at the higher level, we can build games with a much greater audience appeal. That’s the theory, at least. In practice, the games industry and the Hardcore community are so addicted to fiero, we may struggle to make this transition, even though without it we will increasingly labour to make back the development costs of games whose budgets are already in excess of their likely returns.

The opening image is Bayu Utomo's Win or Lose, from the Guandong Museum of Art. I was struck by this piece  as the arms held aloft is a universal symbol of fiero. As ever, no copyright infringement is intended, and I will take the image down if asked.

Comments

[Patrick]

For interactive storytelling (which I'm considering calling "drama games" as its five syllables fewer and much more semantically clear to people in the industry) where you've got no lose state but a dynamic win state (which distinguishes it formally from adventure games) I'm thinking of metrics more in terms of recognizing interesting dynamics. For instance, the most obvious dynamic, you can play your protagonist as good, evil, or in between, and recognize the places along this spectrum where the conflict is most interesting. For instance, totally back-stabbing and malicious is more interesting than a guy who kicks his cat once or twice, since its more dramatically consistent, and a guy who is zen ambivilant is more interesting than someone who displays a more practical ambivilance that doesn't catalyze as much conflict. Think of it as a wave along an X axis, you try to set-up the metrics so the player enganges a feedback loop and finds themselves in the most interesting regions of state space. Of course, things get really fascinating when you introduce multiple dynamics that have confluences with each other, like an generally evil guy who is highly devoted to his little sister. This encourages the player to express themselves through the character, carving their own contradictions and quirks, and finding an agency on a whole other level than Mario getting the 3-up with Yoshi on the castle roof after finding all the stars.

Then theres the issue of player metrics as feedback mechanisms in general, well illustrated by Craig Perko's Pattern Adaptation Control theory. He was working on a simple 2D shooter where difficulty, gun preferences and narrative affinities (towards certain characters or ideologies) would affect dynamically created content tailored by those metrics. Sadly he lost code on it and started on another project, but you get the idea.

I discuss the benifits of metrics in a post of mine about a month ago, where I append the MDA framework (Mechanics, Dynamics, Aesthetics) with an M for metrics, making it MDMA. This sort of meta-system can be drafted during initial prototyping and helps give a sense of the dynamics, enabling better content production.

[Chris]

I read the post you allude to here; there's a lot of good stuff fermenting at your blog. I look forward to seeing it all take fruit in due time.

'Drama games' - yes, it seems so obvious! I'm keen to adopt this term (or at least give it a test drive and see if it sticks). We could give up the long and arduous debate to agree on common definitions for 'interactive storytelling' (and/or 'interactive narrative') and instead coin a new genre term.

[Patrick]

I have a tendancy to repeat a lot of things that can't possibly be useful for at least a year, so heres an idea for the near term:
I've been thinking of ways for a Fireball sequel to have not only performance metrics (quickly the player handles verticle platforming, how good the player is at horizontal platforming, combo averages, fuse timings) but also preference metrics (anti-gravity/abstracted fields versus concrete archetechtures, stage preferences, puzzle vs. challenge vs. fun) and have this system feedback into a dynamic content creation algorithm that takes content primitives (such as objects) and assembles them into cohesive designs as they player goes. So the tutorial aspect and the path aspect will be one, the player's response to the learning curve would be able to parametize the macro content within the first twenty minutes of play, or less. We can discuss this in greater detail next week.

[Chris]

I have wondered about dynamic content generation for Fireball; it's a non-trivial problem, and the expense of developing such a system is not necessarily warranted for a project of this scale. Frankly, I'm not sure we should be banking upon a sequel for Fireball; we might get it, but it's by no means certain. It's a treacherous business, making games. :)