We forgot one Jew along the way! Andrew Gelman is still alive and wants to say something. Gelman is a disciple of Rubin and his blog headline begins with “statistical modeling, causal inference” so he definitely has opinions on all this, which he expresses in his own review of the Book of Why. He’s a bit upset that Pearl is so dismissive of statisticians, who he relegates as “stuck” on rung one, unable to ascend his fancy ladder. I thought this would be a good chance to see a potential outcome practitioner and Pearl actually get down to what’s the problem between them and start to agree, but of course there is no hope. If you want to see for yourself a discussion between experts, Pearl copy-pasted his debate with Gelman on his blog for convenience, and there’s also an older post by Gelman with Pearl popping out in the comments to disagree, with a recap by Gelman. (But… do they disagree for real? Who knows! Ready the printing machine!) I’ll just cite here Pearl’s commencing reaction to Gelman’s review:
Andrew,
The hardest thing for people to snap out of is the bubble of their
own language. You say:
“I find it baffling that Pearl and his colleagues keep taking
statistical problems and, to my mind, complicating them by
wrapping them in a causal structure (see, for example, here).”
No way! and again: No way! There is no way to answer causal questions
without snapping out of statistical vocabulary.
I have tried to demonstrate it to you in the past several
years, but was not able to get you to solve ONE toy problem
from beginning to end.
This will remain a perennial stumbling block until one of your
readers tries honestly to solve ONE toy problem from beginning to end.
No links to books or articles, no naming of fancy statistical
techniques, no global economics problems,
just a simple causal question whose answer we know in advance.
(e.g. take Simpson’s paradox: Which data should be consulted?
The aggregated or the disaggregated?)
Even this group of 73 Editors found it impossible, and have
issued the following guidelines for reporting observational studies:
https://www.atsjournals.org/doi/pdf/10.1513/AnnalsATS.201808-564PS
To readers of your blog: Please try it. The late Dennis Lindley
was the only statistician I met who had the courage to admit:
“We need to enrich our language with a do-operator”. Try it,
and you will see why he came to this conclusion, and perhaps
you will also see why Andrew is unable to follow him.
JP
Wow, Pearl sounds so… cranky. He must be exasperated at statisticians not getting this stuff. And the only statistician who did died from the shock. But there are also his usual tone of biblical entitlement and the forced newlines that add a sophisticated touch of mysticism.
Anyway: in the end Gelman never solves Pearl’s toy problem because he thinks it makes no sense, Pearl tries to bring the discussion more on mathematical territory in the hope of going somewhere, but Pearl doesn’t understand Gelman when he makes more specific examples of things he knows well, and Gelman doesn’t understand Pearl when he makes his own specific examples. We, the ignorant readers, are left with the task of making sense of all this.
I know that I’m not an expert, I’m just an anonymous internet dweller, but I hope to have at least tentatively convinced you in this review that it is possible to get down to the mathematical details and unify everything in a meaningful sense. (Yes I have shown only specific examples with binary variables for simplicity, but all the steps of the reasoning apply more generally.) This swamp of disagreement must be due to what programs these people are running in their heads. To investigate the matter, I rewatched an old dialogue between Gelman and Yudkowsky. My violently uncharitable two-line paraphrase of their exchange is:
Yudkowsky: Imagine the ideal Bayesian agent, the flow and trickle of probability mass over an infinite space, gliding over filaments of evidence, a never ending journey to the truth which had always been nowhere but at the center… How can we mere mortals ascend to that ideal?
Gelman: Uhm, man, I dunno have you tried using more regressors? Last week I used 12 but sometimes I even go for 18. Have you checked that you don’t have coding errors in the model?
Gelman is clearly the human being of the situation, while Yudkowsky is… I don’t know what Yudkowsky is really and I don’t care now, so I’ll say instead that Pearl is the math nerd. Like he says in the Book of Why:
When reading a scientific article, I often catch myself jumping from formula to formula, skipping the words altogether. To me, a formula is a baked idea. Words are ideas in the oven.
I guess that Gelman, judging by his blogging, thinks mostly verbally, and I have fun imagining Pearl applying his principle when he argues with Gelman, skipping everything that Gelman says. Another mathy Pearl quote closing a discussion is (emphasis mine):
All,
Well it has been fun. And, if I did not succeed
in convincing anyone to convert to the dual-perspective
camp, I hope I at least managed to convince you that
causality is about the world – chimes, seat-belts, coins and bells,
not about the method you use in your analysis and not
about what this or that gurus said or did not say.
Causality has been mathematized, so there is
no more room for difference of opinion.
=======Judea Pearl
This is from 2009 so he’s still relatively polite, he talks about the “dual-perspective camp” trying to play the syncretism card. But then he immediately doubles down with his punchline! OPINION IS NO MORE. I AM TRUTH. A typical phrase of wisdom that you have to tell kids is that “math is a universal language, it crosses all borders and differences, every mathematician can understand other mathematicians without ambiguity and they all give each other little kisses.” So while going home with his mother little Pearl is thinking “so if I want to show other people that I am right I just need to express everything into math, they won’t be able to deny my genius then!” The following day, little Pearl gets beaten and left bleeding in the courtyard.
As I illustrated, I don’t buy the strong hierarchy interpretation of the ladder of causation. But I must admit that my sympathy goes to Pearl. He’s very stubborn and a bit arrogant but he has indeed shown already many years ago how he clearly understands the mathematical structure of all this better than anyone else. All the other people I see discussing this seem to miss the big picture and get bogged down in details at some point. I’d just add a pinch of Jayesianism to make Pearl more coherent toward data and he would be perfect to my taste, if not in character at least in content. I think that Pearl’s description of causality is complete and totally generic: if you can’t express your problem within it, it’s because you haven’t mastered the language. I can understand Pearl’s disgruntlement at other experts expressing disagreement without having first learned how to make sense of his tools.
I feel the same way with frequentism vs. Bayesianism. I took my time to learn to wear both the frequentist hat and the Bayesian hat. In front of any problem, I can load the desired mindset and decide to be coherently frequentist or coherently Bayesian depending on the social occasion. I strongly prefer Bayes but I recognize that it’s not a choice you can force with a definitive proof, you can be consistent in both ways, and you don’t lose out on absolute descriptive power either way. But I’m still cornered when my interlocutor makes a puddle of everything and reasons like a human, dividing concepts in clusters as it goes and linking them with a mixture of syllogism and spit, with both Bayesian and frequentist probabilities coexisting together in the real world and arguing with each other, concepts being humans all the way down. Darn humans! Then it becomes impossible to draw rigorous conclusions from the starting point. To avoid confusion you need to work within a language which is very rigid but also totally generic, and stick to it. As if you were writing a computer program.
I see Gelman and Rubin as treading the human world, a dream which Pearl tries to wake them from, but can’t because he is trapped in the human world too, having fixated on a caste system for his formulae. Ok, without the mysticism: Gelman is very practical, he attacks concrete problems and thinks about how to do it. In his worldview, any theoretical invention is a tool designed to be ergonomic, statistics is a toolbox, statisticians are plumbers. He has a set of trusted wrenches, they do the job in his hands, and now Pearl comes along insisting obnoxiously on selling him a new fancy micro-feedback wrench thing that should avoid breaking weak pipes. Gelman already manages, he just knows by looking at the pipe, he picks the right wrench and tightens just the right amount. Pearl insists that Gelman is not really understanding how he doesn’t break pipes, but hey man, what’s the point? He fixes almost every pipe fine, and when he makes a mistake, he goes back and picks another pipe, that’s life. Pearl tries to convince Gelman by showing him the duper-wrench in action—but he doesn’t seem very good at wrenching so Gelman remains skeptical.
Maybe I shouldn’t demand that they agree. They are brilliant people, and are best left alone generating undisturbed their stream of insights however they like it and independently of each other. The more they disagree, the more ideas they spawn, the more we can pick from. On the other hand, scientific practice does end up having concrete effects and I would like to prevent Fisher-type situations. In The Book of Why, Pearl makes a lot of examples of real-life failures that have their seed in a lack of understanding of causes and effects, but I don’t know how this actually translates to the work of econometricians and other scientists. On purely theoretical and mathematical grounds, I’m convinced of the good quality of Pearl’s techniques. I’m optimistic this will continue to hold when I encounter the occasion to apply them.
There are many more things to be said on this topic, but… read the book! Here I reviewed just three chapters and a half, out of a total of ten! More than just a review, I hope to have provided a context for its technical aspects, such that you won’t end up disoriented when you read it. Because you should read it! It’s a juicy big book! Would buy! Nine women out of ten recommend it if forced! Bestseller in Poland! Ask your local libgen proxy about this book!