The Case Against Reality by Don Hoffman
Don Hoffman’s book "The Case Against Reality” makes the argument that reality as you know it does not exist.
For obvious reasons, this is a difficult case to make. Hoffman does a better job than I could’ve expected.
But throughout the book, my mind was racing with doubts. In between the most convincing details of his argument, he’s shunted entire chapters on barely-related research he’s done, and name-drops his conversations with Francis Crick so frequently it’s tough not to believe Don (or Don’s editor) wants you to remember Don as much as he wants you to remember that reality is a sham.
Here I want to distill Don’s argument, throw in a few thoughts of my own, and generally try to steelman the case against reality enough so that, by the end, it feels obvious that nothing you perceive actually exists.
In the second part, I’ll ramble about what I found unconvincing in his arguments. And then I’ll talk about why this book seems so important and relevant, even if huge chunks of it turns out to be false.
I.
Consider, if you will, color.
If you think about it, color is really weird.
I remember wondering as a kid what color atoms were. This was before Google, so I ultimately just wondered and wondered until one day I saw it in a science textbook: a fuzzy, gray blur on a slightly darker gray background.
As was obvious in that moment, this image of an atom was not exactly a photograph. A photograph implies you have so many photons bouncing off of something and hitting the cells on the insides of the camera’s sensor that you can actually see a consistent image. But for atoms, that can be a pretty tall order. Photons can get absorbed by an atom’s electrons, or knock the electron out of orbit, or fly out at random times. And when I realized this – years later – I thought, “Wow! So atoms are smaller than color!” This sounded profound to me, since color obviously isn’t a certain size.
But that’s a bit of a figure-ground inversion. “Atoms” are not a special edge case where color breaks down. “Color” is a special edge case in which something reliably emits light 400-800 nanometers in wavelength at a human eye. A table or chair falls into the has-color category. A single atom does not. Of all the information reaching our bodies at any given point, it is only that which is coded in a narrow band of electromagnetic radiation that we happen to perceive as hues.
Now I’m getting ahead of myself. I said humans can perceive light in the 400-800 nm spectrum, but other animals can perceive light outside of that band. To speak a bit loosely, they can see colors we cannot see. The birds and the bees can see UV (I know there’s a pun in there, but I’m unable to find it). Certain cold-blooded animals can see infrared light. The electromagnetic spectrum technically has no upper or lower bound, so we essentially see a small sliver of a reality in which infinitely broad swaths are hidden from our eyes.
And, to be clear, it’s not that we actually “see reality”. It’s more that sight in an interface with a small sliver of reality. But the better part of reality bounces along in what we could call darkness, since we have no interface allowing us to grasp any of it.
So, pop quiz. What’s wrong with this statement? "Reality itself, if you could perceive it directly, is colorless".
Answer: “Perceive it directly" is a contradiction in terms. We perceive everything through our senses. That’s what perceive means! We don’t have direct access to any property of reality. Only what our senses have given us. Saying “reality is colorless” implies that perhaps it's grayscale, like that grainy image of an atom I saw in high school. But we’re not talking about grays here – we’re talking about a realm in which the very idea color is so misguided as to be a distraction. Asking what color any object "really is" is like asking what a mathematical equation really tastes like. Mu.
And this applies to all of our senses. Reality isn’t just silent – it’s something in which the very notion of silence is a non-sequitur. To me, the fact that reality is not-even-silent feels a bit easier to grok than reality being not-even-colorless. And the other senses are even easier to recognize as pure interface. I don’t know that anyone, when presented with the beatific vision under the hood of the universe expected it to smell like anything.
Realizing that senses are interfaces to tiny slivers of reality is a massive figure-ground inversion. They seem so fundamental. “There’s no sound in space” feels wild the first time you hear it, but from our new vantage point, it should be obvious. There’s no sound basically anywhere except the tiny planet you and your extended family evolved on.
Which is a nice lead-in. Why do we sense some colors, but not others? Why do we hear?
Fortunately, basically every “why” question in biology has the same answer: selection. Creatures perceive what is useful to their survival and reproduction. Those that didn’t got selected by nature. And nature, it turns out, is a teacher who you really don’t want to call on you.
So for us humans, this means seeing light in the 400-800nm range let our ancestors (a) live to reproductive age and (b) have more babies. Seeing different wavelengths or more wavelengths didn’t have that same advantage – it was energy and complexity wasted on something that didn’t help with tasks A or B listed above.
(This doesn’t mean we have zero interaction with different wavelengths of light. UV light will sunburn us – and eventually kill us of skin cancer. But skin cancer strikes after we have kids, which means our ancestors never experienced much selection pressure to see UV – and therefore we also are blind to it. Were we able to see it, I like to imagine we’d perceive strong UV with a creepy, eerie-looking glow, in the same way that other health hazards are likewise noxious – e.g. the smell of rotting meat, or the appearance of molding fruit)
And all the crazy animal senses that we can’t comprehend – like sharks sensing electric fields and octopi sensing the polarization of light – those are all the same. A sliver of the properties of reality, morphed into something wildly different: a conscious sensation that gets us to act in evolutionarily fit ways.
Once you admit that we don’t sense reality, but instead, our senses create an interface to the useful slivers of reality we need to survive and make babies, a lot of physics starts to make more sense. Or rather, the confusing things feel more expected. For instance, neutrinos are particles that so rarely interact with other matter, it took us 70 years from when they were theorized to find all 3 types in experiment. At first it feels a bit weird to think there are things in the universe that are so astoundingly shy. Why? If they don’t interact with any known matter, what role can they possibly play? As I.I. Rabi said of the unexpected discovery of the muon, “who ordered that?”
But this line of thinking is like saying that once we’ve picked the low-hanging fruit (the part of the universe we can easily interact with), the top of the tree must be near. Again, invert figure and ground: the height of the lowest fruit is more a measure of us than the tree.
Why would we expect that everything in the universe readily interacts with us? What’s not to say the largest fraction of reality remains undetectable and undiscovered, simply because it was never of any survival value for us to know about?
(I actually wrote the above sentence before thinking of dark matter and dark energy, but it’s a perfect illustration of this idea. In the 1990s, we realized that literally 70% of the mass-energy of the universe was due to something outside of the realm of what we could see or interact with directly. Anyone would say that’s hugely surprising, but if we’ve really intuited the idea that our senses are interfaces into a small sliver of reality, then it’s no surprise at all. The height of the lowest-hanging fruit is not a measure of the tree)
Up to this point, we’ve only talked about sensory experiences. But this “interface to reality” principle applies to all our experiences. For instance, our experience of time, and our experience of 3D space.
Like our experience of vision, our experience of time feels pretty solid. At an intuitive level, time appears to march forward at an objective, steady rate. But Einstein shot down “objective” and “steady” a century ago, and whatever time is, it’s not those.
(Specifically – and bear with me if you’ve heard this – time could speed up or slow down depending on how fast you were moving, or how near you were to very massive objects. The center of the earth is literally 2.5 years younger than the crust – not because it was formed first, but because time moves slower down there. And for a photon, ever-moving at the universe's speed limit? The best we can posit is that it doesn’t actually experience time at all. Wherever it zips to and from, it’s always all of those places)
And this is just the well-established stuff! Physics is still in the process of figuring out exactly what time is. We can’t say with certainty. But the important thing to realize is that our sense of time (like any of our senses) is an interface with reality, and we were never promised our interface would capture the whole of underlying reality. In fact, the purpose of an interface is to not capture the whole. It’s to shunt away the extraneous details, and only communicate what’s useful.
In the absence of being able to describe what time actually is, I can only provide some guesses as to how different it might be from our experience of it. My favorite such guess comes from David Deutsch in his book The Fabric of Reality, who compares time to a puzzle. A puzzle is meant to be viewed as a 2-dimensional rectangular image. But, we could also look at a puzzle in a different way. For instance, we could go from top to bottom, removing one row of the puzzle at a time (the first row left-to-right, the second right-to-left, the third left-to-right again, etc), and gradually concatenate a single long row with all puzzle pieces. Each piece would fit into the ones next to it, you’d have a completely deterministic series of pieces, you could always reliably reassemble the whole, and yet it would be an entirely silly way of looking at the puzzle.
Likewise, we experience time as a single long series of snapshots – but perhaps the most explanatory way to assemble those snapshots is some other structure. Despite being arranged in a different structure though, each individual snapshot feels like one in a long series. Why? Because even in a single snapshot, our brains contain memories of snapshots that affected this one in a way such that we call it the past. The past can exist concurrently, or all snapshots can exist in not-even-time, but from the vantage point of any one snapshot, we feel like there was a past and we are now beyond it.
Again, this is only somewhat idle speculation as to how things might actually work. For right now, we need only recognize that (a) our interface with time, like any sensory interface, will hide vast swaths of what’s actually there, showing us only what’s useful for natural and sexual selection, and (b) since we already know time is far more complex that it feels like it is, we should maintain wariness of any of our intuitions about it.
The next “sense” on the chopping block is our sense of 3D space – that of living on a fabric on which everything else plays out. Einstein definitively hacked apart time a century ago, but the cracks are only more recently starting to show in 3D space.
Theoretical physicist Nima Arkhani-Hamed says of it:
“Almost all of us believe that spacetime doesn’t exist, that spacetime is doomed, and has to be replaced by some more primitive building blocks”
But while no one yet knows what those primitive building blocks are, we should take the same lesson that we learned from our other senses: what we experience of 3D space is merely an evolutionarily useful representation that’s misguidingly simple compared to the underlying reality.
A fun hunch as to what is more fundamental than space is from physicist Sean Carroll’s book Something Deeply Hidden. He notices that, of all the properties a particle might have, its position in space is the only one that allows it to interact with other particles. If two particles have the exact same velocity, cool – but indicative of nothing. If two particles have the same spin, wonderful – but again, it means nothing. If two particles have the same (or nearly the same) position, however – then they can magically influence each other’s other properties. “Bounce off each other”, in common parlance, but Sean would warn us not to read too much into the image of billiard balls ricocheting through space.
As Sean points out, there’s one particular elephant in the physics room that messes with this idea, and it’s quantum entanglement: the phenomenon by which two particles appear to influence each other’s properties instantly and regardless of distance. To see this clearly, Sean recommends inverting figure and ground. Perhaps entanglement is more fundamental than space. Since nearby particles are always entangled, perhaps distance is just our way of perceiving entanglement – and, in the rare cases where entangled particles are incredibly far away, it breaks our notion of distance. But of course it does – because distance is merely the emergent mirage of a network of entangled particles.
And if “a network of entangled particles” sounds hard to grok, perhaps that’s a sign it’s only closer to the underlying truth. We spend all our time using an interface to reality. Of course that interface feels intuitive. Reality itself, by this measure, should almost surely feel strange.
In the end, reality is likely to have about as much in common with what we see, hear, feel, and experience as your computer desktop has in common with the billions of wires and transistors inside it. The purpose of the interface is not to show the underlying reality, but to distort it usefully. What we perceive is more like a trash can icon on your desktop. Of course there is no trash can; it’s merely a useful interface for interacting with the billions of voltage levels your computer tracks as bits. The trash icon we see? Useful for sure, but not even remotely close to what lies underneath.
Put a spoon on your counter. When you look at it, it exists in your interface with reality – your sense of sight. But now look away. Is there still a spoon? If it has exited your interface, then only the underlying reality exists. A reality that we know has no color, no texture, no sense of weight, and doesn’t live in time or 3D space. Whatever exists there is best not called a spoon. We only use that abstraction for the appearance of the spoon on our interface.
So this all begs the question: what exactly then is real?
Great question. Um. Let’s change topics.
II.
I want to talk for a minute about the parts of Hoffman’s book I found unconvincing. Try as I might, there were a few things I had to omit from my steelman summary above, as I couldn’t do them justice.
“When I have an experience that I describe as a red tomato a meter away, the content of that experience is not that there is – in objective reality, even when no one looks – a red tomato a meter away."
Hoffman is fond of talking about the disappearing tomato – one that ceases to be a tomato when no one is sensing it. Now I can get behind the “redness" of the tomato being interface, not reality. Indeed, that was my opener. I can even imagine the “meter" being a little shortcut of our perception, not a low-level concept of reality. But dang it, it’s hard to imagine the tomato losing its "tomato-ness" just because no one’s looking.
Frankly, the only analogy I can think of that makes sense here is software. Does a simulation of a tomato cease to be a tomato when no one looks at it? Well, yes. There still might be some file tucked away with the shape and texture of the tomato, but that file, in the end, is ones and zeros, likely implemented as higher and lower voltages on a sea of wires and transistors. And perhaps the operating system has seen to split up that file to be stored in chunks, not even localized in one part of the disk. At some point, we’re talking about something enough layers of abstraction away from a tomato that the tomato-ness is just utterly gone.
And I suppose that’s Don’s implication? Perhaps we are akin to software beings (his video game example of choice is Grand Theft Auto, which I hope sets a precedent for all writing on the illusory nature of reality) realizing their experiences are merely an interface to an underlying reality, and even if we grok that “it’s all made of zeros and ones!” we’re still not really understanding what substrate we’re on. We’d still be pretty surprised when, I don’t know, a processor overheats and frames start dropping. You need to go deeper to understand that; even knowing "it’s all ones and zeros” won't explain it.
Hoffman only mentions the idea that we’re in a simulation a couple times in passing, and I’d pay to hear more of his thoughts on this. He is careful to avoid giving specific implementations of reality (entirely appropriately), but the lack of more in-depth metaphors for his thoughts can make it tough to understand exactly what he’s implying.
“[A]toms and subatomic particles are not visible without special equipment, but they’re still in space and time, and so they are still in the interface”
Or, to use his analogy:
“[This] claim is more like saying ‘I know that the icons on my desktop are not the true reality. But if I pull out my trusty magnifying glass and look really closely at the desktop, I see tiny pixels. And those tiny pixels, not the big icons, are the true nature of reality.’ Well, not really. Those pixels are still on the desktop, still in the interface. They may not be visible without a magnifying glass, but they’re part of the interface nonetheless."
Hoffman argues evolution shaped our perceptions – that is, our interface to reality – to be useful, not truthful. And that’s a fantastic insight. If we expect that our perceptions (and therefore, our intuitions) about things are there to keep us alive, it causes us a lot less grief when those intuitions go haywire at, say, the quantum level.
However, the converse seems to be that evolution wouldn’t shape our perception of things not useful for our own evolutionary fitness. Ignore atoms and electrons, which are undeniably critical for fitness. Let’s look at something I mention above, something that seems a little less debatable: neutrinos. They are mysteriously reticent particles, slow to interact with anything else we’ve ever discovered. Understandably, it took our species a really long time to find them. I’m not positive of the following statements, but I would put some money on (A) neutrinos were never at any point critical to human evolution, and (B) we have no way of sensing them in our evolutionary environment.
To me, this is an indicator that they’re not part of our interface with reality – they’re actually a step closer to what underlies our interface.
But if we can sense them at all, even with our fancy tools, Hoffman says they’re part of the interface, not reality. I’m not sure how he arrived here.
Another example: the spin of an electron. I like this in particular because it’s so inconceivable to our intuition. In college, we were simply told “Yeah, electrons have a property called ’spin’, it’s measured in units of angular momentum, it can be positive constant, negative constant, and nothing else, and by the way – it’s useless to try and imagine the electron as actually spinning”. Again, I don’t believe electron spin was ever fitness-enhancing to perceive. But we’ve built tools that can perceive it. And we’ve built math that can predict it, even if we can’t explain it per se. That some weird property from beyond the pale leaks into our environment seems like a knock on the door from the next level down, not simply “pixels viewed through a magnifying glass”. Hoffman hints as much when he talks about the holographic principle – the idea that the maximum amount of quantum information (never mind precisely what this is) storable in a volume is proportional to the surface area of the volume, not the… you know… volume of the volume. So what inconceivably weird aspects of our environment are hints to reality, and which are pure interface?
Until I hear otherwise, I like the converse of Hoffman’s own metric: if it wasn’t fitness-enhancing to sense, it’s not part of the interface.
"The network dynamic of conscious agents is compressed into dynamics within spacetime. For instance, perhaps a dynamical evolution of conscious agents toward small-world networks may appear in spacetime as the dynamics of gravity"
So ends the very last paragraph of the appendix, and Hoffman transitions directly into thanking such luminaries as Daniel Dennett, David Chalmers, Steven Pinker, V. S. Ramachandran, and… Deepak Chopra? I guess enough anti-reality talk gets you mixed up with an interesting crowd. But no hate. I’ll explain in a minute, but someone’s got to be thinking these thoughts.
Anyhow, the last sentence of this book was like a needle going directly into a tire that we’ve been inflating for over 200 pages. Hoffman has walked us through the illusory nature of all our perceptions, including those of space and time, decried even the objective existence of atoms, and finally, while we’re at this absolute trough of epistemological certainty, he sneaks in a few of his own suspicions about Conscious Realism – the idea that conscious agents are more fundamentally real than the supposedly objective world around us. And like I said above, fair enough. I want someone to be thinking through this stuff.
But then in the very last sentence, we get a glimpse into how he suspects conscious agents might work, and it’s… gravity is a side effect of conscious agents needing to network with each other?
I so badly wanted to know what he actually thought was going on down there in the basements of reality. And now I suddenly don’t.
But.
III.
I remember reading once a plea by a physicist: we really need to figure out what to teach the next generation of future physicists so that they can figure out the stumpers in quantum mechanics. I believe it was said with an eye towards making video games that have locality-breaking or cancelling-probability elements. But Don Hoffman’s philosophy, more than anything I’ve ever read, feels like the answer here.
What do we teach young scientists so they can see farther than us? Teach them their intuitions of reality owe them nothing.
A lot of questions in science now revolve around things that are very hard – or currently impossible – to interact with directly: dark matter, dark energy, the multiple worlds interpretation of quantum mechanics, consciousness, existence itself. When facing questions about a universe that we can only detect a small fraction of, it's tempting to sweep ideas away wholesale. “There’s no hard problem of consciousness”, or “MWI isn’t even science” are not uncommon opinions. But Don’s viewpoint is that of course the universe is filled with structures that are nigh-impossible to observe. Our senses were shaped by evolution to allow us to see what we interact with – so all the parts of the universe we can’t interact with will obviously be dark. But nothing suggests that they don’t exist; merely that we won’t be able to see them directly.
And where we can’t see and can’t interact, we won't have a good intuition for what’s there. So: best to expect our intuitions will be trampled roughshod.
I think the most important part of The Case Against Reality is this: before reading it, certain results would feel weird because they weren’t explained by current theories; after reading it, current theories feel weird because they can’t explain certain results. This book is not about consciousness, and it’s not about quantum gravity, and, well, OK, it is about whether our senses have any pertinence to the fundamental nature of reality, but even if it weren’t, it would've made me realize that the gaping holes in our explanations of our environment won’t be filled by normal-sounding theories. That ship has sailed. Whether that’s the hard problem of consciousness, or the unification of relativity and quantum mechanics, or what will replace spacetime, Hoffman has taken a look at those problems and realized our intuitions owe us no hints in truth-seeking. But why should they? That was never their job.
Here are a few things The Case Against Reality has changed my mind on – or, if not changed, then somewhat strengthened the direction I’ve been leaning:
Before: “The fact that anything exists at all feels crazy and almost unbelievable” After: “Oh shut up. You don’t know the half of it, or perhaps the hundredth of it, or perhaps the 10^-30th of it! Maybe literally everything exists. The only thing you know about the stuff you can’t see is that your intuitions aren't going to be much use!"
Before: “We can’t see (or meaningfully interact with) other universes in the multiverse. The Multiple World Interpretation of quantum mechanics is really tough to believe!” After: “Hey moron, wasn’t it evolution that gave you sight? If you can’t interact with something, it’s not affecting your fitness. You should have zero intuition about what’s beyond the realm of your senses. Zero. What on earth does not seeing something you can’t interact with count for its non-existence!? You’re talking in circles.”
Before: “Some particles move backwards in time!? What even is that!?” After: “You really don’t get it, do you? Why do you think your perception of time has any more relation to reality than your perception of color does to photon wavelengths? Time is a useful interface plastered over whatever it is that we’re living in, but don’t expect it to be too intuitive beyond the realm in which it’s useful to your evolved senses."
Before: “Atoms are smaller than color. How poetic and weird!” After: “You know is outside the realm of color? The literally 95% of the universe that we have no current means of detecting directly, and yet are sure exists. Color is the edge case, man. A tiny peep-hole into a little sliver of dark existence. Come on."
Before: “Consciousness is the hard problem! So hard! Everything else seems simple by comparison” After: “Dude, if you haven’t figured out the hard problem, don’t get too confident that you actually understand all the easy ones. Remember, from here on out, the going gets weird. We’ve picked all the low-hanging fruit from the tree of theories. We’re sailing well past our intuition at this point, and what we find, should we find it, might re-write everything we thought we already understood.”
The history of science feels like one slap in the face after another to anthropocentrism. We’re not the center of the universe. Not even the center of the solar system, let alone the galaxy – of which there are, by the way, billions. And your great-great-grandparents were one-celled organisms. And there may be infinitely many copies of you. You thought you were a special snowflake, and you were right – at least in the sense that there are billions of you!
And now as we run up against the edges of time and space and matter and consciousness, we’ll have to view our own perceptions – and the intuitions they foster – not as a reliable guide to what’s really there, but as such a nearsighted peering through such a dim, narrow keyhole that the shadows we see on the other side are practically distractions to understanding what it is that actually lies there.
In any case, good luck to us! Seems like we’ll need it.
The Case Against Reality by Don Hoffman