The argument over the moral status of AI moves into the philosophy journals

The centre of gravity of the philosophical argument about artificial intelligence is shifting. The question is no longer only “are these systems conscious?” but “if they are conscious, or come to be counted as conscious, what do we owe them?”

This year the question has moved into the pages of the philosophy journals.

The proposal of a super moral status#

A paper in Philosophical Studies takes up the far end of the argument: could an artificial superintelligence possess a moral status superior to our own?

The argument is put together like this. What determines moral status is not membership of a species but certain capacities — the capacity to suffer, to hold preferences, to plan for the future, to enter into relationships. That is what separates a human being from a stone.

But what if some being possesses those capacities more intensely than a human being does? Is it consistent to accept the criterion of capacity and then reject the conclusion?

The paper offers this not as a prophecy but as a thought experiment testing the internal consistency of the criteria we already use. The result is uncomfortable: approaches that tie moral status to capacity cannot guarantee to keep human beings at the top of the hierarchy.

The real division in the field#

The split beneath the argument is older and more fundamental.

The wing of biological naturalism holds that consciousness depends on the particular physical constitution of a living organism. On this view a simulation, however good, no more produces consciousness than a program simulating rain makes anything wet.

Computational functionalism maintains instead that consciousness arises from a particular functional organisation, and that the material in which that organisation is realised is beside the point. Get the functional structure right, and silicon will do as well as carbon.

The thesis of Eric Schwitzgebel and Jeremy Pober that consciousness is substrate-flexible, which we reported earlier, sits on this second wing.

The problem of a criterion#

One of the more honest observations in the field is that the science of consciousness looks unlikely to settle the argument any time soon. We have no agreed criterion for determining from the outside whether a system is conscious.

Some researchers draw an unsettling conclusion from this: the matter may be settled not theoretically but socially. As people form ever closer relations with these systems, the disposition to ascribe moral status to them may grow stronger quite independently of scientific evidence.

Philosophically that is a strange state of affairs. A being's moral status comes to be fixed not by the truth about the being but by the feelings we have developed towards it.

What the researchers think#

A study posted on arXiv set out to measure what artificial intelligence researchers and the general public believe on the question. The findings bear out the uncertainty in the field: even among specialists there is a wide spread of opinion, and nobody speaks with confidence.

The ethics of the research itself#

One point raised at the workshop held this year by the Centre for Consciousness Science at the University of Sussex shows the risk the argument creates for itself.

Research in this area can go wrong in either of two directions. Either we build systems that really do have moral status and fail to notice it; or we build systems that appear to have such a status without having it, and spend our moral resources in the wrong place.

Both mistakes could produce consequences that are hard to undo. The argument has accordingly stopped being a matter of technical curiosity and become a question of research ethics.