"The Metaethics of Joy, Suffering, and Artificial Intelligence" with Brian Tomasik and David Pearce

Or: mind-meld thought experiments show the foundations of ethics are ontologically subjective

“Even if all the technical issues surrounding the creation of an artificial general intelligence or super intelligence are solved, we will still face deeply challenging ethical questions that will have tremendous consequences for earth-originating intelligent life. This is what is meant when it is said that we must do philosophy or ethics on a deadline.” The above quote was spoken seven years ago, a seeming eon in today's AI landscape, to introduce the podcast episode The Metaethics of Joy, Suffering, and Artificial Intelligence with Brian Tomasik and David Pearce. Since the deadline of superintelligence has not yet arrived, time still remains to delve into these philosophical issues and further clarify foundational ethical questions. The conversation between Tomasik and Pearce ranges widely, but this review will focus on the core metaethical debate and their discussion of a mind-meld thought experiment which, with some added analysis, helped me refine my own metathetical position.

What is metaethics exactly? Metaethics refers to the study of the foundations of ethics itself, in contrast to normative ethics and applied ethics. Normative ethics refers to the study of ethical principles and theories such as consequentialism (acts are judged based on their consequences), deontological ethics (acts are judged based on whether they conform to our duties/obligations) and virtue ethics (acts are judged by what type of person they cultivate). Applied ethics refers to taking these theories and applying them to concrete situations to determine how one should behave. For example, is it okay to kill animals for food? Different normative ethical theories will produce different answers when applied to that question. In contrast, metaethics seeks to understand ethics itself, inquiring into questions such as whether or not ethical questions have objective answers. This is one of the key disagreements between Pearce and Tomasik and it is a question with huge implications. If ethics turns out to be objective, then perhaps we can expect an artificial superintelligence (ASI) to figure it out on its own and act accordingly. If it is subjective, perhaps the best anyone can hope for is creating an ASI with values similar to their own.

They agree on a suffering focused ethics, but is it objective?

Pearce and Tomasik begin by agreeing that ethics should be focused on alleviating suffering:

David Pearce: My focus has always been on the problem of suffering, a very ancient problem. Buddhism and countless other traditions are preoccupied by the problem of suffering. I'm also a transhumanist and what transhumanism brings to the problem of suffering is the idea that it's possible to use technology, in particular biotechnology, to phase out suffering, not just in humans, throughout the living world and ideally replace them by gradients of intelligent wellbeing....
...
Brian Tomasik: I've been interested in utilitarianism since I was 18 and I discovered the word. I immediately looked it up and was interested to see that the philosophy mirrored some of the things that I had been thinking about up to that point. I became interested in animal ethics and the far future. A year after that, I actually discovered David's writings of the Hedonistic Imperative, along with other factors. His writings helped to inspire me to care more about suffering relative to the creation of happiness. Since then, I've been what you might call suffering-focused, which means I think that the reduction of suffering has more moral priority than other values...

After this initial agreement, the podcast host asked the pair to dive into the foundations for their moral views. They still seem to be in total agreement at this point:

Brian Tomasik: At bottom, the reason that I placed foremost priority on suffering is emotion. Basically, the emotional experience of having suffered myself intensely from time to time and having empathy when I see others suffering intensely. That experience of either feeling it yourself or seeing others in extreme pain carries just a moral valence to me, or a spiritual sensation you might call it, that seems different from the sensation that I feel from anything else. It seems just obvious at an emotional level that, say, torture or being eaten alive by a predatory animal, or things of that nature, have more moral urgency than anything else...
...
David Pearce: I would very much like to echo what Brian was saying there. I mean there is something about the nature of intense suffering. One can't communicate it to someone who hasn't suffered, someone who is for example born with congenital anesthesia or insensitivity to pain, but there is something that is self-intimatingly nasty and disvaluable about suffering. However, evolution hasn't engineered us, of course, to care impartially about the suffering of all sentient beings. My suffering and those of my genetic kin tends to matter far more to me than anything else. So far as we aspire to become transhuman and posthuman, we should be aspiring to this godlike perspective that takes into account the suffering of all sentient beings

The agreement between Pearce and Tomasik on suffering oriented ethics resides in the domain of normative ethics. This agreement sharpens their disagreement in the domain of metaethics, namely, whether or not morality is objective ^[1]. With Pearce taking an objective stance and Brian a subjective one, this is where the core of the debate starts taking shape. I quote here at length from the discussion (skipping parts I deemed nonessential) since it both cuts to the heart of their disagreement and sounds rather confused. I then attempt to clarify the disagreement with some alternative concepts to those used in the discussion.

David Pearce: I just say it's just built into the nature of let's say agony that agony is disvaluable. Now, you might say that there is nothing in the equations of physics and science that says anything over and above the experience itself, something like redness. Yeah, redness is subjective. It's mind-dependent. Yet, unless one thinks minds don't exist in the physical universe, redness is an objective feature of the natural physical world. I would say that for reasons we simply don’t understand, the pleasure-pain axis discloses the world's inbuilt metric of value and disvalue. It's not an open question whether something like agony is disvaluable to the victim. Now, of course, someone might say, "Well, yes. Agony is disvaluable to you but it's not disvaluable to me." I would say that this reflects an epistemological limitation and that, insofar as you can access what it is like to be me and I'm in agony, then you will appreciate why agony is objectively disvaluable...

Lucas (interviewer): I guess here, you would describe yourself, given these views, as a moral realist or an objectivist.

David Pearce: Yes, yes.

Brian Tomasik: Just to jump in before we get to me. Couldn’t you say that your view is still based on mind-dependence because, at least based on the thing about if somebody else were hooked up to you, that person would appreciate the badness of suffering. That's still just dependent on that other mind’s judgment or even if you have somebody who could mind meld with the whole universe and experience all suffering at once. That would still be the dependence of that mind. That mind is judging it to be a bad thing. Isn't it still mind-depending ultimately?

David: Mind-dependent but I would say that minds are features of the physical world and so, obviously one can argue for some kind of dualism but I'm a monistic physicalist, at least that's my working assumption.

Brian Tomasik: I think objective moral value usually—the definition is usually that it's not mind-dependent. Although, maybe it just depends what definition we're using.

Lucas (interviewer): Just to echo what Brian was saying. The traditional objectivist or more realist view is that the way in which science is the project of interrogating third person facts like what is simply true about the person regardless of what we think about it–in some ways, I think that traditionally the moral realist view is that if morality deals with objective facts, then, these facts are third person objectively true and can be discovered through the methods and tools of ethics. In the same way that someone who might be a mathematical realist would say that one does not invent certain geometric objects rather one discovers them through the application of mathematical reasoning and logic.

David Pearce: Yes. I think it's very tempting to think of first person facts as having some kind of second rate ontological status but as far as I'm concerned, first person facts are real. If someone is in agony or experiencing redness, these are objective facts about the physical world...

...

Brian Tomasik: ...My metaphysical position is anti-realism. I think that moral statements are mind-dependent. They reflect ultimately our own preferences even if they are maybe very spiritual and, like, deep fundamental preferences. I think Occam’s Razor favors this view because it would add complexity to the world for there to be independent truths. I'm not even sure what that would mean. Based on a similar reason, I reject mathematical truths and anything non-physicalist. I think moral truths, mathematical truths and so on can all be thought of as fictional constructions that we make. We can reason within these fictional universes of ethics and mathematics that we construct using physical thought processes. That's my basic metaphysical stance....I think very much of myself as an emotivist. It's very clear to me that what I'm doing when I do ethics is what the emotivist says people are doing. Yes, since I don't believe in moral truth, it would not make sense for me to be gesturing at moral truths. Except maybe in so far as my low level brain wiring intuitively thinks in those terms.

David Pearce: Just to add to this and that, although it is possible to imagine something like spectrum inversion, color inversion, some people who like ice cream and some people who hate ice cream, one thing it isn't possible to do is imagine a civilization in which [the beings have] an inverted pleasure-pain axis ^[2]. It seems to just be a basic fact about the world that unbearable, agony and despair is experienced as disvaluable and even cases that might appear to contradict this–like, let’s say, the masochist–in fact merely confirm a claim because. Yeah, I mean the masochist enjoys the intensity, [the] rewarding release of endogenous opioids when the masochist undergoes activities that might otherwise be humiliating or painful.

Lucas (interviewer): Right. David, it seems you're making a claim about there being a perfect convergence in the space of all possible minds among the pleasure-pain axis having the same sort of function. I guess I'm potentially just missing the gap or pointing out the gap between that and I guess your cognitivist objectivism?

David Pearce: It seems to be built into the nature of let’s say agony or despair itself that it is disvaluable. It's not “I’m in agony. Is this valuable or not?” It's not an open question...it's built in the nature of the experience itself.

This is the core of the debate and a lot of terms have been thrown around: objective, subjective, realist, anti-realist. I think these terms are being used in confusing ways but that some different concepts can cut through the confusion.

The key 2x2: Objective/Subjective x Ontological/Epistemological

Pearce used the term “objective” in a confusing way that seemed to simply mean “exists” when he said: “Redness is subjective, it’s mind dependent, and yet unless one thinks minds don’t exist in the physical universe, redness is an objective feature of the natural physical world.” A few terms introduced by John Searle [3] can help clear up the confusion. He makes two distinctions between the objective and subjective. The first is ontological, having to do with what exists:

• Ontologically objective phenomena exist independent of minds. Chairs, water, atoms, that sort of thing [4].
• Ontologically subjective phenomena exist only because a subject experiences them. For example, pain is real and it exists by being experienced.

Ontologically objective and subjective features of the universe are both real but they are meaningfully different. Redness, as a mind-dependent experience, is ontologically subjective. Put another way, it's “appearance and existence are identical.” That is, the appearance of redness in consciousness is what makes redness exist. That doesn't make it less real than ontologically objective entities, but there is a meaningful distinction between them.

The second distinction Searle makes is epistemological, having to do with the nature of knowledge:

• Epistemologically subjective knowledge comes from first-person experience. For example, I know that I prefer chocolate ice cream to vanilla from directly experiencing it.
• Epistemologically objective knowledge can be gained independent of any particular mind. That is, with the right training, any sufficiently intelligent mind can come to know it. This is the realm of science and rational argument. Note, that it is possible to objectively study ontologically subjective entities. For example, we can objectively study what causes pain, what kinds of brain activity correlate with it etc. The whole field of anesthesiology depends on the epistemologically objective study of the ontologically subjective phenomenon of pain. If you don’t believe this, then you shouldn’t trust anesthesiologists to prevent you from feeling agonizing pain if you ever need surgery. That said, this doesn’t mean that science can tell us anything about the intrinsic nature of the ontologically subjective.

When Pearce says that “it seems to be built into the nature of agony that it is disvaluable,” he is making an ontological claim that suffering is inherently bad. How does he know this? From direct experience. So, it is an epistemologically subjective claim. He (and presumably any conscious being reading this) knows from direct experience that agony is disvaluable. If you changed the experience of agony to be valuable, it wouldn’t be agony anymore. However, that is only a statement about one’s own agony. Ethics relates to the agony of others. Pearce addressed this when he said:

Now, of course, someone might say, “Well, yes. Agony is disvaluable to you but it's not disvaluable to me.” I would say that this reflects an epistemological limitation and that in so far as you can access what it is like to be me and I'm in agony, then you will appreciate why agony is objectively disvaluable.

One person’s inability to directly experience what another person experiences does reflect an epistemological limitation, but that does not tell us whether ethics is epistemologically objective or not. A small child who doesn't know how to read will not be able to use a tape-measure to find out the length of a table. That doesn't mean that the length of the table is not knowable in an epistemologically objective way. It just means the child currently has an epistemological limitation that prevents her from measuring the length of the table. Is the epistemological limitation regarding the experience of another’s suffering fundamentally the same as not knowing how to use a tape measure? This is where some mind-meld thought experiments can help clarify, but first one further distinction will be helpful.

Mind-contingent vs sentience-intrinsic

Later in the discussion, it becomes clearer that what Pearce means by “objective” is something akin to “non-contingent”. He brings up the example of being a fan of the soccer team Manchester United. He wouldn’t take a pill that changed him to be a fan of a different team, but he recognizes that his preference for Manchester United is arbitrary. A slight change to his life circumstances and he would have been a Liverpool fan instead. In contrast, it is impossible to imagine agony being good. It doesn’t appear to be arbitrary. Agony being bad is just the intrinsic nature of the experience. No change in life circumstances or upbringing would make agony intrinsically good.

A very simple model of world and experience can help clarify this distinction (phrased in the second person for clarity): (1) There is some state of affairs in the external world and in your body, (2) Your brain receives data through various sensory organs (both internal and external) from both the world and your body and, in combination with prior knowledge, constructs a model of the current state of affairs, (3) There are qualia (intrinsic experiences) associated with parts of this model, and a subset of the qualia are on the pleasure/pain axis. Schematically: