Ethics of Science

How can a scientist square their research decisions with the applications they are put to? Do scientists bear some obligation in respect of the research they conduct? Or is all scientific practice essentially blameless? What are the issues in the ethics of science?

When interviewed in 1965 about the Manhattan Project, the creation of the first atomic bomb, the physicist J. Robert Oppenheimer noted his conflicted feelings about his success by making reference to a line in the Hindu scripture the Bhagavad Gita which came to him in connection with this achievement, albeit misremembered. Oppenheimer’s misquote became famous: “Now I am become Death, the destroyer of worlds.”

As it happens, humanity has thus far managed to avoid eradicating itself in a sudden nuclear oblivion – a fear that abounded in the short-lived “Atomic age”. Yet there can be no doubt that scientists worked together to create nuclear weapons, the most terrible and destructive force on the planet. Was it ethical for them to do so? All ethics, we now realise, are relative. So to ask if the Manhattan Project was an ethical research project is to ask about the ethics of scientists.

Science, as with most traditions, has its own inherent values. In the case of science in general and scientists in particular, the central value is truth, therefore, from an agent-focussed perspective, the central virtue of ethics is honesty. From a rights-focussed perspective, this commitment to honesty manifests as the unwritten rules of scientific practice that render fabrication of results and plagiarism forbidden and highly disreputable. From an outcome-focussed perspective, scientists are usually not held accountable to the consequences of their research; as long as they have been suitably honest in the conduct and reporting of their research they have done more or less all that is required of them by conventional scientific ethics.

The trouble is, this enshrinement of truth as the central value of science leads to a naive attitude as to the consequences of research. If we are honest about scientific experiments, we must acknowledge a tremendous realm of future events about which we cannot adequately anticipate. Oppenheimer could not have known that forty years after the Manhattan Project, a nuclear power station in Ukraine would explode, contaminating the city of Chernobyl and the surrounding region, and having effects throughout the world. This was a consequence of the early research into nuclear weapons, which were a step on the road to nuclear power. Was Oppenheimer in any way responsible for Chernobyl?

To hold Oppenheimer (and the other scientists in the development of nuclear power) entirely blameless, we must say that knowledge and application are separate. The creation of knowledge is thus rendered inculpable, and only its application can be “evil”. But by this approach, a scientist can research into a biological weapon capable of killing all life on the planet and claim no responsibility when it is eventually employed. This separation of duties into a required commitment to truth and an optional commitment to society or humanity is ludicrous.

But equally ludicrous is to blame Oppenheimer for Chernobyl, which after all he was not directly involved in at all. His contribution to the knowledge that lead to the building of the Chernobyl nuclear power station connects him tangentially, at best, to the disaster that occurred there. One may also argue: if it hadn’t been Oppenheimer, it would have been someone else.

Yet what kind of ethics are we suggesting if the justification for taking an action with the potential for tremendously negative ultimate effects is if I don’t do it, someone else will? Is it really acceptable for someone to conduct research which has as its natural consequence a fatal weapon simply on the grounds that if they didn’t someone else would? This sounds rather akin to the excuse, so often touted in the context of war crimes, that one was only following orders.

The basis of ethics is individual responsibility. If scientists can’t take responsibility for what they choose to research, the natural response from society would be to deny them access to the funding that they need to conduct that research. That this doesn’t happen is a sure indicator of just how addicted to technological progress we have become.

Of course, it is a simplification of the facts to claim that the only aspect of ethics of science that applies is a commitment to honesty. After all, there are ethical guidelines for the treatment of people (and, to a lesser extent, animals) in experiments – such as the requirement that test subjects provide informed consent. Yet science as a field is a long way shy of the kind of commitments we expect from our doctors – informally captured in the phrase “first, do no harm”. It may once have been the case that the potential for abuse was greater from doctors than from scientists, but in our modern world we see disaster scenarios behind every scientific frontier, from genetic engineering to high energy particle physics.

Many of the ways in which the public respond to these issues are reactionary – disproportionate to the matter at hand. But the reason that public outcries against genetic engineering of food and so forth happen is because it is seems to the general public that scientists will not take into account the consequences of their research as a factor in whether or not to pursue it. This inevitably leads to excessive backlash in those cases where the possible problems have the potential to be extremely severe. The scientific community will not and cannot make a commitment to ‘do no harm’ – because no scientist has any greater capacity to predict the future than any non-scientist. Humans are just naturally poor at predicting the future. Just as Oppenheimer could not foresee Chernobyl, no scientist can truly foresee the negative consequences of their research.

But of course, that Oppenheimer could not foresee Chernobyl does not bear on the question of Hiroshima. Oppenheimer could certainly see that what they were making was a bomb, and that this bomb would be used on human targets. It seems that Oppenheimer justified this in his own mind in the belief that the creation of the atomic bomb would necessitate an end to war – in his own words “The atomic bomb made the prospect of future war unendurable.” He hoped that the threat of nuclear war would drive humanity into global co-operation. Was this hope enough to make it ethical to pursue the research for such a weapon in the first place?

Ultimately, Oppenheimer acknowledged that there was no ethical decision at the root of the Manhattan Project. He said of it: “When you see something that is technically sweet, you go ahead and do it and you argue about what to do about it only after you have had your technical success. That is the way it was with the atomic bomb.” Here, we have the honest assessment of a scientist: I must conduct my research; what it is used for is a problem only thought about after the fact.

Scientists are reluctant to admit the dangers inherent to unrestricted research. There is a tacit assumption with many scientists that the scientific community – as intelligent, educated men and women – would not instigate research with severe negative consequences, despite the many historical instances to the contrary.  An idealised view of science persists in the mind's of many scientists, one which sees science as not only harmless but presumptively beneficial, while other traditions are publicly berated for their hypothetical dangers.

I believe we can stave off the worst disasters that might come out of the scientific process. But we would have a much better chance if we acknowledged that science is not a blameless process, uncovering truth independently of its consequences, but the force feeding the rapacious technological progress that has lead us to a world in which we consume our natural resources in a manner apparently leading to a global disaster of our own creation. What is the cause of the rape of the environment if not the technologies born of our science, and the people that use them?

Now the most urgent voice in this apparent calamity is concerned with global warming, which has become a highly politicised topic. This is a contentious issue, and it goes to the heart of science. Basically: climate scientists mostly agree that we are the cause of a currently small but potentially disastrous greenhouse effect that is warming the planet. Some climate scientists, albeit a minority, disagree. The public is more divided. Who do you believe? There are scientists on both sides, so do you trust the majority or the minority? Do you look at the evidence and make up your own mind? Well, some do the former and some do the latter, but either way there's no way to resolve this issue except by exercising your own freedom of belief.

It's up to you to interpret science. It's up to you to interpret everything.

Here are the facts, as far as I can ascertain. The greenhouse effect is a genuine theory, first reported by French mathematician and physicist Joseph Fourier, and subsequently verified by the Swiss chemist Svante Arrhenius. We have an excellent example of what it can do in the form of the planet Venus, which is a roiling hell world thanks to the positive feedback of a greenhouse effect caused by the carbon dioxide in its atmosphere. Venus is believed to have got that way because there was no life there to recycle the carbon dioxide – what keeps our planet so delightfully habitable is in part its diverse ecology which moderates the gas cycle. We're killing off that ecology in a variety of ways, and in a manner far more direct than the global warming we're worrying about.

On the other hand, the evidence for global warming is very slight – the mean temperatures have risen, but the claim that humanity is responsible for this change is quite tenuous and difficult to prove. Crucially, the existence of a majority of climate scientists on one side of the debate doesn't change the fact that climate predictions are very difficult to make. It would be premature to conclude the matter of human-influenced global warming one way or the other, and the threat of disaster is apparently insufficient by itself to force decisive action at this time.

What it comes down to is this: do you trust the climate scientists? In the case of global warming there seems to be no reliable way of establishing the extent of the crisis or the degree of urgency. But the climate scientists aren't the only people warning of the onset of an environmental catastrophe. According to a 1998 survey, 70% of biologists believe we are at the beginning of a mass extinction event, not unlike the ones that killed the dinosaurs and many other earlier forms of life, but this time driven by human activity. We have had at least 36 extinctions in the last century,  not counting the species that went extinct before scientists could identify them, and in geological terms that's precipitously rapid. The technological society we have built with the assistance of science is killing off all life on the planet –  slowly from our perspective, in the blink of an eye from the viewpoint of the planet itself.

Having said all this, science also carries the potential to rescue us from the very environmental disaster it has given birth to. After all, it is the scientists who have made us aware of the collapse of biodiversity, and there are scientists committed to research into sustainable energy solutions. With co-operation from society, we may yet be able to rescue much of our the ecology of our planet which, I might add, is essential to our continued existence – even the extinction of a small collection of species such as the worms could lead ultimately to our own extinction by collapsing the food web. There is some hope that it won’t come to this, and if not, we will have in part the action of committed scientists to thank, although it will not be the value they have placed upon truth that will have lead us from the brink of catastrophe.

How long can we continue with this situation whereby scientists value truth and only truth, especially in the wake of Kuhn’s realisation that science is not so much a journey towards truth, but rather a process of adaptation and specialisation? What is it that we want science to adapt itself to? Do we even know? Perhaps we need to rethink the ethics of science so that we can position this tradition, with its potential for tremendous benefits, into a better relationship with society... perhaps it is up to society to create the necessary pressures to ensure that science is its servant, and not its master.

Beyond the current catastrophe of environment, and the future crisis that genetic engineering will place squarely at our doorstep, there are further disasters and crises that science will bring about in time, presuming we survive long enough for it to do so. (Not to mention the many other distant threats – such as asteroid impacts – that science could help protect us from if we were to consider such things important.) Given this, we should seriously consider how we wish to proceed with the scientific process.  To do that, we must first be willing to talk about the role of science in our societies and decide what kind of future we will try to make.

For translucy, who may have different views on the subject.

Comments

[Patrick]

I'm curious as to how you're apply this frame to the creation of transhuman artificial intelligences.

[Chris]

I don't see transhuman artificial intelligence coming about in my lifetime, whilst I see issues pertaining to environment, cloning and potable water shortages as being more immediate.

At the moment, we can make AI that behaves like an insect. So, just another 400 million years of evolution to recap. :)

Of course, that doesn't preclude talking about the ethics of a-life. I tell you what, let me sort out Animal Rights first (although this is a while off yet...), then I might add something on ethics of a-life to compliment it.

Best wishes!

[zenBen]

I was going to say, this is like an aetheist blaming the artists for the preponderence of religious art in the Rennaissance, but it's not even like that.
Science and engineering are two seperate disciplines, and everything that you've blamed on science here is in fact the result of excessive engineering of a scientific solution. Sure, they couldn't have done it without the scientists arriving at the solution, but once the solution is there, the scientists move on and the people who paid for the science take over (after all scientists aren't very interested in what they already know).
Of course all the scientists could independently ethically evaluate every line of research they have the opportunity to undertake, and I'm sure many do. And if they come up with an negative evaluation, they could all say 'No, we refuse to undertake this'. And the people who were going to pay for it would say 'Well where do we turn to get the job done?' and the scientists could only reply 'Educate a new generation, we're off!' What then would be their relationship with society? Ostracised and impotent, most like.
If ethics are relative, then they are not independently absolute, since social relativity demands compromise and acknowledgement of the positions of others - the social trumps the absolute. The furthest you can get from this is within the capitalist system where you might have scientist/entrepeneurs like Venter, but how many similar examples exist? If the scientific ethical agenda is necessarily one of compromise with the larger society, then it is in fact the society's ethical agenda. If proportional democracy is the best case society we've seen, then scientists form only a tiny minority of the electoral register.
You want the scientists to hold back the tide, it seems, and then mop up if they let it through.

[Chris]

zenBen: "You want the scientists to hold back the tide, it seems, and then mop up if they let it through."

I want nothing more than scientists to admit that the practice of science is not risk free. This seems a small admission. I'm willing to make this concession: are you?

"If the scientific ethical agenda is necessarily one of compromise with the larger society, then it is in fact the society's ethical agenda."

Perhaps ideally, yes. But this isn't what we see at this time.

"What then would be their relationship with society? Ostracised and impotent, most like."

You presume that the people who pay for research are society as a whole. Do you believe this to be the case?

Thanks for biting back on this one - I was hoping for some discussion, which is why I came out so polarised, but it didn't happen.

Best wishes!

[zenBen]

I think I can make that admission, yes, and I believe most other scientists do the same - but almost always on behalf of competing research, not their own :D The politics of funding and prestige.

I don't presume that society pays for research, quite the opposite - but it wouldn't take much recalcitrance on the part of scientists, before 'society' would begin to question their already imperilled and tenuous independence, if not the need for them at all.
I believe that unless those who have the money publicise it, the money to do science is usually far less publicly visible than are the scientists. So the scientists have a more delicate path to walk in relation to public opinion. Perhaps this is only right, but it disempowers scientists to a great degree.

If you decouple the research aims from the begging bowl, you give a vast amount of power directly to the scientists. But you take it away from, for want of a better phrase, the shadowy power-brokers. So maybe in that hypothetical case, with the visibility of the science community matched by their power and third parties removed, the ethics of the whole system would shape up?

[Chris]

zenBen: I find this a fascinating proposition. Could we really separate capitalism and science? I just don't know, but it's an interesting thought to ponder...

[Peter Crowther]

Given that I'm now involved with various organisations trying to do science (if not Science), I thought I'd just drop in the comment that most scientists are perennially searching for resources to further their own research aims. Grants and specific projects are generally inconveniences - you do the minimum to satisfy the grant giver, and then continue with your preferred research with whatever resource is left over.

Perhaps if scientists were more open about their research agendas, and were less involved in bidding wars as to who could do the most research on the topic specified in the call, people would trust them more. However in order to do that, a 30-year-old (or more) trend towards the enslavement of the scientist by the accountant has to be reversed...

[zenBen]

I think both of the above comments speak to an even larger issue, which scientific research is caught up in. Namely, the selfish or selfless motivation of the deployment of human resources. Global warming is the big current question here. If it were even more immanent, would that change our attitude? What would we do if threatened with an avoidable mass extinction event? Would we avoid it by deploying humanity's resources to good effect? On the evidence, I'd have to say no, because those questions hinge grammatically on the 'we', and I don't see any 'we'ness to the human attitude. It's all 'us and them'.

Think we've seen this kind of idea in the comments before.

[Chris]

I've just finished reading Hannah Arendt's 'Between Past and Future' which contains further thoughts in this regard. I was particularly struck by this thought:

"Seen from a sufficient distance, the cars in which we travel and which we know we built ourselves will look as though they were, as Heisenberg once put it, 'as inescapable a part of ourselves as the snail's shell is to its occupant.' All our pride in what we can do will disappear into some kind of mutation of the human race; the whole of technology, seen from this point, in fact no longer appears 'as the result of a conscious human effort to extend man's material powers, but rather as a large-scale biological process.'"

Or to put it another way: if we as a species were in control of science, what an incredible tool that would be - but it seems rather as if the scientist's commitment to science, instead of to society, has led us down a particular path without us ever actually choosing it.

Something has to change in the relationship between science and society - but as intimated here, finding a way to break already established patterns will be difficult.

Best wishes!

[zenBen]

"Or to put it another way: if we as a species were in control of science" [italics mine].
Didn't that Arendt quote specify technology, not science?
I think you should step back and sufficiently delineate the areas about which we are discussing - science and technology are, as you seem to be saying in your latest comment over on the Mythology of Science post, quite different. In one sense, they can be seen as a pipeline, and in general there are dependencies and cyclical relationships inherent, but they are not homogenous.

In reply to your comment on Arendt, I'd say that technology can seem like an extension of a biological process, and this process is to facilitate individuals in doing less physical work. Like a crane, the things we can achieve through physical work grow on top of existing technology (like cars, only possible after one has advanced metallurgy) and then are subsumed into new technology in their turn (like robotics on car production lines).
The comparison with science is tenuous - the advancement (I hesitate to say progress anymore!) of science follows the impulse to do more work, this time mental. And it entails more work too, since the sum total of knowledge needed to be conversant in an area is always growing (a good reason to work in a brand new research area).

"Something has to change in the relationship between science and society - but as intimated here, finding a way to break already established patterns will be difficult."
If we can agree that science often tries to beat its own path, but is ultimately paid for by society and thus directed to a large degree, then it seems that maybe the problem is a kind of cyclical abrogation of responsibility. And, obviously, if we wish to break the cycle we have send all the arrows of the graph to one destination. In other words, responsibility must be wholly owned by one party. The question then is, do you want responsbility for the ethics of science to lie in the hands of an elite few (with associated problems like how are they chosen); or do you want it to lie with the uninformed masses (with problems like how do you aportion blame and punishment to so many? Democracy? Does it work for the world's biggest science spender, the U.S?).

Or there may be a third way. Doesn't spring to mind though.

[Chris]

The point about the Arendt quote is that we like to think that we are in charge of the scientific process (which feeds the technological process), yet there is little if any evidence to show that we really are in control. Rather, it is as if the technology has appeared as if from a purely biological process over which we have no control. Consider also this quote from the same essay:

"The simple fact that physicists split the atom without any hesitations the very moment they knew how to do it, although they realized full well the enormous destructive potentialities of their operation, demonstrates that the scientist qua scientist does not even care about the survival of the human race on earth or, for that matters, about the survival of the planet itself."

Note that 'scientist qua scientists' means 'scientist in the role of scientist'; it does not mean that such people do not have concerns in other roles - in their role as citizen, for instance.

I don't think it is necessary for responsibility to end up in one place for us to seize control of the process; the anarchy of research is actually an asset of a kind.

What perhaps is missing is a public space for discussion of science: the companies and universities that are primarily responsible for science/technology are still economically tied to the societies in which they are embedded, such that market forces have the main influence. If we can manifest the public space for discussion, the issue of how to organise and direct the process can be addressed. It does not need to be solved in advance, I suspect.

But it is manifesting this public space that we have failed to do, or even fully consider, until now.

[translucy]

in terms of epistemology science and technology are distinct. but at least if a scientist uses outcome focused ethics she will turn into a technologist: to worry about outcome means to worry about application of knowledge.