The Beginning of Infinity and The Fabric of Reality byDavid Deutsch

The incomplete and ongoing works of David Deutsch

I got hooked by a slogan. “Upgrade your brain.’  Upgrade my brain?  For $15?  Take my money!  As a long-time aspiring rationalist, isn’t that the whole point?  (Also, I had just re-read the AI FOOM article and a part of me was thinking, if I upgrade my brain, then maybe, just maybe, upgraded me can then…  The possibilities were endless!)  (I ended up being right about that last part, but for completely the wrong reasons…)

So I read The Beginning of Infinity.  I didn’t get it.  But: I had the definite feeling of reading something momentous.  So I read The Fabric of Reality, the prequel.  Twice.  A much better grounding, but still didn’t really get it.  Found Deutsch’s original paper.  Double didn’t get it (layperson).  The testability paper.  Sort of got it.  I read what other rationalists thought (I think they mostly didn’t get it either tbh).  Watched interviews, with people like Robin, Lulie, Sam Harris, lots of youtube videos.  Went back and reread.

And one day I looked up and realised.  I still don’t get it.  Not all of it anyway, but what little I do has turned me into a hardcore Deutsch/Popperian.  And I couldn’t be happier about it.

Deutsch’s claim to fame is quantum computing.  But his work there is inextricably tied up with his unique philosophical worldview.  And that worldview is exceptionally deep, coherent, powerful, optimistic and to my knowledge unrefuted.  Tim Urban once wrote, for any new topic he encountered, the arguments were like leaves on a tree with no trunk, fluttering and disconnected. Going back to basics, considering first principles, would give him a trunk, something to hang everything else off of.  Reading Deutsch, coming to grips with his ideas, and Popper’s ideas, is like that – not just for any topic, but for all topics.  Something to hang everything else on, a tree trunk for the very idea of sense-making at all. (I asked Tim what he thought about all this, btw, but didn’t get an answer swiftly. “I’m frightened, and I’ve spent all my money”, Tim said with scarcity and paucity.)

Deutsch’s worldview is based on four great theories, humanity's greatest and deepest pieces of knowledge, which woven together form a unifying Theory of Everything that acts as the filter through which he understands everything.  Popperian epistemology. Everettian quantum mechanics. Turing completeness. Darwinian/Dawkinsian evolution.

Of these, the first is the key, both to unlocking the others, and to understanding pretty much anything.  It offers the best account I have read of how knowledge is formed at its most basic level.  In short terms, that the process of knowledge formation is one of conjecture and criticism in response to perceived problems, with tentative adoption of those conjectures that survive the criticism.  Crucially, conjectures are never proven, belief is never justified or true, adoption is only ever tentative.  There is only error correction, a process of getting closer to, but never reaching “truth”.  Deutsch explains that what makes our best explanatory theories good explanations are that they explain their explicanda, they are hard to vary, in the sense of resistant to arbitrary change in their detail, and are generally consistent with other known good explanations.  Notwithstanding all of this, even our very best explanatory theories remain tentative conjectures, riddled with unknown errors, even as they contain useful knowledge, much as Newtonian mechanics is now known to be false in the face of general relativity, but remains useful in many domains.  This account of knowledge formation is refutable like everything else, but to my mind (so far!) is accurate.  Some of it seems so obvious as to not be worth mentioning.  And yet simultaneously it holds deep implications for knowledge formation that are not well understood and roundly rejected by many, indeed most, thinkers.

Interestingly, while the philosophical underpinnings are different, his writing covers much of the same ground as (Bayesian) rationality.  The injunction against taking a belief as justified or true is similar to adopting a P value that expresses appropriate uncertainty.  The process of criticism fills a similar role to updating one’s priors on new evidence (though Deutsch would say it is argument, rather than evidence, that is key).  Conjecture is the Popperian answer to the question of where initial priors come from – they’re guesses!  Searching for a “hard to vary” theory is similar to avoiding belief in belief and applying Occam’s razor and complexity penalties.  Focussing on explanatory theories is similar to insisting that beliefs pay rent in anticipated experience (Deutsch would say the explanation precedes the prediction).  And in many, many cases a (Bayesian) rationalist and a Popperian would come to the same conclusions.  So perhaps I need to explain why I think that every rationalist should read his work.

Part of the magic is the simplicity.  For example, Everettian quantum mechanics will be familiar ground to any reader of the Sequences.  In The Fabric of Reality, however, Deutsch presents the simplest account I have read, qualitatively, with no mathematics, and concludes in favour through simple argument in a way that follows inexorably from Popperian epistemology.  Deutsch describes the outcomes of the double slit experiments, and then describes how the behaviour of each photon is impacted (as evidenced by a different pattern of shadow) when other slits are opened by something that is invisible but behaves exactly like an ordinary photon (a “shadow” photon).  This shadow photon is itself impacted by an invisible counterpart to the barrier that behaves exactly like an ordinary barrier (a “shadow” barrier).  Even in a small experiment, it is possible to establish a billion slits, the opening of which impact the outcome, thus implying at least a billon shadow photons for each ordinary photon (and implying an infinity for want of any reason to suppose a limit).  Each shadow photon appears to interact, other than in cases of interference, only with its corresponding shadow barrier in exactly the same way that the ordinary photon interacts with the ordinary barrier.  And there is nothing to distinguish each shadow photon (and shadow barrier) from the ordinary photon (and ordinary barrier), or any other shadow photon (or shadow barrier), nor to distinguish the photons and barriers in the experiment from anything outside the experiment. The multiverse explains these explicanda without any need to engage with the wave function or the mathematics associated with it.  A simple, hard to vary, explanatory theory that neatly accounts for its explicanda.

Does this justify Everettian quantum mechanics as true?  No.  There are no justified true beliefs.  But Deutsch explains why, when viewed through the lens of Popperian epistemology, this is sufficient to regard the multiverse as real.

More importantly, however, having concluded that the best explanatory theory of reality is Everettian quantum mechanics, he is not content to leave it there as an intellectual boondoggle with no implications for the real world.  Rather, he engages fully with the philosophical implications of it being real.  How does it interact with the other strands of his worldview?  What are its implications for evolution?  What does it mean for the idea of causation?  What does it mean for free will? (Short answer, it’s real!) These are deep and important questions, and treating them as separate magisteria as many do means failing to develop a coherent, consistent understanding of reality.

The above will seem nebulous, I am sure.  I can’t give an adequate summary of his collection of writings.  It is too dense with insight, too full of curiosity, too much of a viewquake.  But even if I could, I would want to leave no one with the impression that reading a summary is in any way a close substitute for engaging deeply.

Why do I want you to read his works?

For starters, it’s fun.  For years now, I have thought about these ideas every day.  Every. Single. Day.  (I haven’t been this consumed since I first stumbled across SSC and realised the most useful thing I could do was read every damn thing Scott had ever written.  This is like that (though I will say his fiction is not as fun as Scott’s).)  And selfishly, I would love more discussions about these ideas to emerge amongst the internet’s best commentariat.

Secondly, Deutsch’s ideas complement and augment rationalism.  I expect he would agree with all but 1.5 of the 12 virtues of rationalism.  I think if the rationalist community can incorporate his insights into its canon, it will be immeasurably better for it.  At a basic level, Deutch’s worldview seems to cope much better with some of the lurking problems that continue to beset the Bayesian rationalist.  I remember being troubled by the Control Group is out of Control, where Scott analysed at length why the positive results in paranormal experiments do not lead him to believe in the paranormal.  He ultimately (in the comments, not the post!) reasons that there is no plausible physical mechanism for the paranormal phenomena being tested, which never felt like a Bayesian conclusion, but would be the first place a Popperian would land, i.e. there being no explanatory theory to support the results.  And since we have good explanatory theories about confounding and experimental error, a Popperian would conclude against psi, but with less mental gymnastics required.

Popperianism is also simpler, for its expression in qualitative terms rather than quantitative terms.  One of the criticisms often levelled at Bayesianism is intractability and false precision.  Per the recent lab leak debate, the formula can be twisted to nefarious or ridiculous ends depending on how inputs are layered up.  And since multiplying wide range estimates can quickly balloon into uselessness, a level of precision on inputs is required to make calculations meaningful that in turns calls into question the validity of assigning such specific numbers to inputs.  This is reminiscent of criticisms levelled by David Chapman and Nostalgebraist years ago.  To make it work, we are often applying a “qualitative” version of Bayesianism, using the intuitions rather than the numbers.  And qualitative Bayesianism converges very quickly on the idea of seeking good explanatory theories.  Marrying Popperianism and Bayesianism would pay dividends in clarity of thinking.

Separately, Deutsch’s more recent work focusses on a new theoretical framework he has developed, constructor theory, whichreframes physics through the lens of what transformations are and are not possible.  The human ability to transform nature is limited only by the very laws of physics themselves.  I still don’t get it all, but maybe others better qualified can contribute to it.

There are other topics where I think Deutsch’s worldview does better.  I won’t get too much into AI risk, beyond saying my own preoccupation with AGI ruin dissolved as soon as I came to understand Deutsch’s views.  That is, I think one’s views of AGI are downstream of one’s views on epistemology.  If knowledge is generated by algorithmic extrapolation from observations, rather than conjecture and criticism, then it is no wonder that an extremely powerful iterative AI should achieve superintelligence beyond what even AI-assisted humans could ever hope to match.  If choice and free will do not exist, it is no wonder that a superintelligent AGI would pursue its utility function to the end of the earth (note: much worse than pursuing something to the ends of the earth!) – and there is no moral failure in imposing unbreakable vows on future persons.  If problems are not inevitable, curtailing a powerful technology is an acceptable sacrifice and Pascal can take me for all I am worth and I will smile.  But if none of those things are true… suffice to say I sleep easier now.

Finally, and perhaps my crowning motivation, is that Deutsch preaches optimism, something sorely lacking in recent times.  Not blind optimism, but optimism based on a simple extrapolation of Popperian epistemology – all problems are soluble, it’s just a matter of knowing how.  Every damn thing you or I have ever worried about in our long or short lives, or will ever worry about in the long, short or infinite remaining, it’s all soluble, if only we go about creating the requisite knowledge.  And that is our choice.

There is no stable state of perfection.  Problems are inevitable, both the ones we can imagine and the ones we can’t.  And the only known solution is endless, endless knowledge creation.