Noun Games
In the Balance

DGD2: Measuring Play

How does one measure play? Are there instruments we can construct that detect different kinds of play? Nicole Lazzaro employed Ekman's instrument of emotional cues to build an observational model, but what other tools are available that we might use to build a working model of play, and of play styles?

It has been some time since I've posted on the journey towards the DGD2 audience model. To a considerable degree, this has been a result of being incredibly busy - working on Fireball, helping to set up a company in India, working on the Game Writing book and so forth. However, the other problem I'm facing is that I have set the bar rather too high for a next step.

I'm very keen to pursue the game tests as a means towards DGD2, but what I need to reach a new model in this way is a rather large collection of tests - and in truth, defining even one of these tests is a difficult challenge. I am therefore faced with the decision of either breaking down the task into smaller steps, or choosing an alternative course of action.

Science is the process of measuring. In order to do this, it uses instruments, which in some cases are broadly reliable devices - such as rulers, voltmeters and radio telescopes - and in some cases are subjectively agreed forms - such as biological taxonomies, social experiments and personality inventories. The value placed in these instruments is largely a product of the dominant scientific paradigms, and therefore of prevailing belief systems among scientists. Even apparently reliable instruments often depend upon more subjective instrumentation to proceed to conclusions - for example, see the telltale signs of cognitive dissonance in the cosmologist when you ask them what it would mean if the speed of light were not constant...

If DGD2 is to proceed to a next stage via game tests, then what I am facing is actually an instrumentation problem. I need to define instruments. And realistically, I cannot hope to be providing all this instrumentation in a single step, as each instrument is in effect a seperate experiment, and hence a seperate measurement. It may behoove me, therefore, to narrow my focus, and consider what instruments can be designed to measure play.

The hypothesis I am working on for DGD2 at the moment is something like this:

Different people enjoy different forms of play. Temperament Theory suggests that there is an applicable model of skills in four related clusters: Logistical, Tactical, Strategic and Diplomatic, expressed to different degrees by all people. These skill sets can be applied to play...

That, however, is the point that the paragraph runs out. What exactly is the hypothesis guiding DGD2? Always bearing in mind that my hope is not to prove the hypothesis correct - I'm perfectly happy to prove it wrong (we did with DGD!). The goal is to produce a new model based on experimental observation - measurement, if you will. The hypothesis just determines the direction we head out in, not what we will ultimately find. But this hypothesis is currently incomplete.

Having now identified that coming up with a suite of instruments for DGD2 research is too large a problem to be solved in a single step, it follows, therefore, that I should either choose one skill set and attempt to produce an instrument to "detect" it (or to detect enjoyment in people employing it), or I should choose some element of play, determine an instrument to "detect" it, and then produce a hypothesis as to the expected results in the context of the model I already have.

And this is roughly as far as I have managed to come. I need tools to measure play, and I believe it may be possible by constructing mini-games or micro-games (a la Warioware). But I still need a more solid framework before I can design these games, or find partners (academic or commercial) to realise the instrumentation.

Perhaps part of my problem is the attempt to produce these games in isolation, and in fact I would do better to choose existing games and to produce instruments that detect different play inside the same game (which could be done with observational metrics). Perhaps my problem is continuing to work with Temperament Theory, and I would do better to give up the model I have (despite it's apparent utility) and try another approach. However, since I find the subjective instruments of Temperament Theory to be useful, and they are validated to some degree by the capacity for other people using the same model to communicate meaningfully with me, it seems a shame to give up the map that I have, before having a means to build a new map.

Whatever the reason, I am currently stalled, and can find no obvious way to proceed. I continue to think about the problems at great length, but the brutal truth is I may be stuck like apocryphal Newton under the tree, waiting for a clue to hit me over the head.


Feed You can follow this conversation by subscribing to the comment feed for this post.

Have you read any of Ben Cousin's essays? Perhaps his work is what you descibe as "Observational metrics", but one way or another he is certainly measuring play. I personally find it interesting that his gameplay "Atoms" are equivelent to a game's fundemental verbs.

His site is here:


"Science is the process of measuring"

Sorry, not convinced. Measurement is not useful in isolation; it is, instead, useful for testing the results of hypotheses, as you found with DGD1. The difference is significant.

I subscribe to Karl Popper's view that science is the repeated process of coming up with hypotheses (conjectures) that are amenable to disproof (refutation), subjecting them to the most stringent tests, and using the results to derive new and hopefully more soundly based hypotheses. If you haven't already done so, you may wish to dig out a copy of Popper's "Conjectures and Refutations" and read at least the first essay in it. Popper's style is very clear but dense, and the result is remarkably readable. There are updates to the theory, but Popper's is still by far the best introduction that I've found.

I think this may help you with the design process, as you might be able to devise some conjectures for which it is relatively cheap to construct experiments that might refute them. Rather than waste time and effort constructing a number of games, is there a way of dipping one's toe into the water?

- Peter

Brad: I'll check our Ben Cousin's stuff. The name rings a bell; I think I've come across this somewhere else.

Peter: sorry, Kuhn trumps Popper in my game. :) Much as I enjoyed reading Popper, Kuhn's The Structure of Scientific Revolutions goes further in my estimation. Kuhn found little evidence of scientists following a falsification methodology, and I find Kuhn's position to be more valuable. I still use Popper, though, but as a boundary condition for science. If you can't falsify it, it must be outside science. :)

I remember Ben, now; he had an article in Develop a while back. It's interesting what he's doing, but it's too low level for my research, and much too reductionist for my tastes. It's focussing on measurements at the game world level; I need measurements of the player. There's no way to marry the two approaches.

Thanks for the tip, though! I appreciate all leads!

PS: The reason I refer to science as a process of measuring and not (as is more traditionally the case) as the process of acquiring knowledge, is that 'knowledge' is just as empty a term as 'Truth' - which is probably why there are no great epistemologists. :) Kuhn suggested, and I agree, that the body of "scientific knowledge" was best seen as an adapted set of instruments for modelling the world.

But of course, this isn't the only possible model one can have - it just happens to be the one I have preferred to adopt after examining the options. :)

Heres an exercise for you, play Flow and read Jenova's material on his MFA thesis site, try to apply temperment theory to Flow, and then try and apply it to Fireball.

Then read my latest blog post and play Warning Forever. But not nessecarily in that order.

I think I'm hobbled here by never really managing to get Flow to work for me. It was never clear what I was playing; indeed, only at one point in the entire experience was I in any way confident that I had an avatar. The rest of the time, it was just a swirly mass of dots. Thus, I found it primarily confusing, and hence frustrating.

I may have been suffering from a bug, as I noticed (trying it again) that my avatar seemed to appear and disappear at random.

I'm guessing this is a Tactical game, but I'd like to see other people play to know more, as it may be that the core play is Logistical (depending how much it revolves around optimisation of action). I suspect this game only appeals to a very narrow Rational/Idealist audience, though; it's way too abstract for the person on the street.

Fireball is supposed to lend itself to Tactical, Logistical and Strategic play, but it's not yet at a sufficient state to check that it can be enjoyed Tactically. It may be solely a Logistical (repeat-and-beat) and Strategic (solve the puzzle) game.

Warning Forever is a classic 2D shmup (may this form live forever!) Once upon a time these were often Logistical games that you played until you learned the patterns; these days (as bullet mazes predominate), they are much more Tactical, requiring situational skills instead of learned responses.

Regarding the theory of Flow, my current assumption is that players enjoy using different skills at different degrees of difficulty, therefore each player reaches Flow states in different games under wildly varying conditions, according to how it relates to their skills. But I also suspect (as I've mentioned before) that different players enjoy being at different places in the Flow channel, which complicates things somewhat.

I seem to have lost my coherence... I should get something to eat... :)

"Kuhn suggested, and I agree, that the body of "scientific knowledge" was best seen as an adapted set of instruments for modelling the world."

From my own (incomplete) understanding, Popper's view is very similar *in that respect*. It's certainly a statement I would agree with. Presumably (I haven't read to this depth) when examined critically, the models have varying degrees of fidelity, with models constructed later typically having higher fidelity than those constructed earlier; at least when evaluated by someone whose own views are themselves shaped by later models in that same succession.

As a programmer by trade, I happen to find Popper's approach of more use. If I try to model something in a particular way, how could I then attack that model? This tends to produce a finished artifact that is relatively robust.

Incidentally, if 'knowledge' is an empty term, what term would you use to describe 'the highest-fidelity and best-tested models of the world we have to date'? (Yes, I'm aware that I am ranking elements by a function that takes two arguments here - just call me Bentham)

I think Popper's model appeals more to pragmatists and Kuhn to those with a taste for the abstract, to some degree. Since it's all models, we might as well take the ones that work for us and use them. :)

Perhaps I was coming down too hard on 'knowledge', since your definition seems a pretty stellar description. :) I'll retreat my claim to knowledge being unrelated to big-t Truth, rather than being entirely empty. After all, to come back to Wittgenstein (as I tediously always do), the meaning of a word is how it is used, and your definition has plenty of utility.

Also, when I said knowledge is the process of measurement, I did leave out the aspect of this process which is involved in devising measurements, and models to interpret them (the role of the hypothesis and the theory). Perhaps I should have said 'a set of processes concerned with measurement?' Or perhaps, I was just being concise. :)

Incidentally, I just picked up a book by Paul Feyerabend (who died recently). He was right on the bow wave of philosophy of science, and seems to have been advocating that science should give up its roots in a 'mythological' method and instead adopt a more anarchic structure. I'm very much looking forward to reading what he has to say! If it lends itself to being summarised, I'll post about it here (although not all books do).

Verify your Comment

Previewing your Comment

This is only a preview. Your comment has not yet been posted.

Your comment could not be posted. Error type:
Your comment has been posted. Post another comment

The letters and numbers you entered did not match the image. Please try again.

As a final step before posting your comment, enter the letters and numbers you see in the image below. This prevents automated programs from posting comments.

Having trouble reading this image? View an alternate.


Post a comment

Your Information

(Name is required. Email address will not be displayed with the comment.)