The Empty Rituals of Peer Review

Is peer review a valuable contributor to the objectivity of the sciences? Or an absurd obfuscator of the actual practises and views of scientists?

The Jewish people have a word, ‘kibitzing’, which means to offer unwelcome or even unhelpful advise in a back seat driver sense. It is often associated with games – for me, the archetype of this is playing solitaire and hearing over your shoulder “put the red ten on the black jack”. In academia, however, kibitzing has been made central to the progression mechanics of the game, and dignified with the title ‘peer review’. I have given and received many peer reviews of papers for conferences and the like, as well as reading feedback other people have had from the peer review process and am forced to conclude that the system, while having many strengths, also serves as a means to strangle innovation and contrary perspectives, and to uphold and enforce every half-baked dogmatic philosophical prejudice at large in the research community.

I have written, in The Mythology of Evolution and elsewhere, about the way ground-breaking new ideas struggle to emerge in scientific fields because peer review tends to uphold orthodoxy and therefore suppress alternative points of view. The peer review process is far more effective than the central authority of something like the Vatican in this regard, as both responsibility and blame is anonymously distributed, ensuring the status quo is always upheld. This is one side of the problems inherent to peer review, but since innovations that upset the applecart are rare, this cannot be the worst excess of peer review. No, this occurs in the almost mindless way that empty ritualism becomes substituted for good judgement in the exchange of opinions dressed up in artificial authority.

When asked to be a peer reviewer, I have always taken the view that I am doing a favour both for conference organisers and authors of papers: my feedback, I believe, should help clarify the strengths and weaknesses of a paper, and provide suggestions for additional citations that might be helpful (sometimes supplemented with apologies for the fact that there won't be enough time to consult these extra sources). Despite my efforts, some of my responses were probably badly received – the peer review process is doomed to hurt feelings, no matter how hard a reviewer tries to be civil. I view this as unfortunate but inevitable, and still preferable to the bland uselessness of the publishing review system which issues stock rejection letters of zero use to anyone.

However, some peer review feedback I have read is not helpful, nor polite, nor necessarily well-reasoned. Under the cloak of anonymity, it seems some peer reviewers vent their frustrations and prejudices in the sure and certain knowledge that this will never come back to them. Papers are subject to review; peer reviewers are generally immunised from assessment – no matter what their feedback might be like. One cannot be censured as a peer reviewer, although a bad incident might have someone removed from the invitation list for a single journal or conference.

The worst empty ritualism, however, occurs in the sciences, which persist in the farcical belief that facts speak for themselves, that everyday language is less scientific than jargon (especially jargon derived from the classical languages) and that the scientist is strangely irrelevant to scientific practice, being a mere observer – like Cassandra's pronouncements on behalf of the gods. I am always irritated when peer review asks for me to remove first person references, something that only happens in the sciences and which has no methodological advantage: the sole benefit gained by excising individuals from accounts of their research is the maintenance of a form of writing that is less accessible to a wider audience, thus preserving the mystique of the expert.

This complaint concerning inaccessibility is even more true of the substitution of comprehensible terms for bland neologisms based on Latin and Greek. The Victorian scientists did much to invoke the aura of objectivity around dead languages – a process amusingly reminiscent of the old priestly requirement to speak Latin. Just as the priest used to be viewed as apart from – or even sometimes above – the “common man”, so too the scientist’s resort to dead languages creates an unnecessary mystification that allows them to preserve the sense of separation from the ordinary, a communion with the imaginary objective world they believe they are investigating that somehow elevates their work from being merely the practises of mortal scientists to the perfect exposition of immortal Science.

The peer review process in the sciences serves to maintain the mythology of Science by providing specific rituals of objectivity that do not and cannot make the work more objective. An experiment’s most vital component is the scientist running the protocol – eliding their involvement or perspective is to obfuscate what actually happened in the lab, or to censure the view of the only people who witnessed the relevant events. This preserves the mythic status of those facts which are claimed to speak for themselves, a claim that ironically is advanced by people whose speech is absolutely required before any notion of fact could conceivably be brought to bear. Furthermore, muzzling personal opinions in scientific papers on grounds of objectivity is to confuse esoteric ritualism for transcendent principles of knowledge, and to favour the maintenance of a community of priest-like experts over the integration of the scientist into their communities.

I do not see the peer review system changing, especially while belief in its empty rituals serves as a prop for the objectivity of Science – that grand abstraction that substitutes (often in highly suspicious circumstances) for the actions, beliefs, and discourse of scientists. But neither can I go on pretending that these vapid traditions are anything but arbitrary components of a professional rituality, like the wigs worn in British courts. Objective they are not, nor could they ever be so. If we wish to be objective about the sciences, and I’m uncertain we should, perhaps we could begin by letting scientists talk about their work, instead of pretending that the work speaks for itself.

The opening image is by James Yang. As ever, no copyright infringement is intended and I will take the image down if asked.

Comments

[Ttocb.blogspot.com]

I'm reminded of Churchill's comment: "Democracy is the worst form of government except for all the others."

Seems peer review might be the same.

[Chris]

Chairman: Aye, I'm aware of the silliness of railing against better-than-nothing systems. But sometimes, you just gotta wail. :D

[translucy]

Quite entertaining rant ;-)
If you havent done so you may be interested to check the book "understanding knowledge as commons" by Ostrom and Hess.
As a matter of fact the emerging theory of common-pool resources may actually suit your taste ;-)

[Chris]

translucy: Cheers for the suggestion - it looks right up my street, but will have to wait until after I get through writing "Chaos Ethics" - the reading list for it has held me up all year! *waves*