Helsingin yliopisto
In this essay series, I will write down my own thoughts about Eliezer Yudkowsky’s essays on the Rationality: Ai to Zombies –series from the point of view of a historian. My reason for writing these is primarily to organize my own thoughts regarding the use of rationality as a mental tool in my own profession, and as such, I do not presume to even attempt to appeal to a very wide audience. However, if you are reading this disclaimer and find my essays insightful or entertaining, more the power to you and I implore you to go and read the original essays, if you have not already.
“What is true of one apple may not be true of another apple; thus more can be said about a single apple than about all the apples in the world.”
This quote is can be found in Yudkowsky’s essay …What’s a Bias, Again?¹, in which he vaguely described the category of cognitive biases and why they can – to simplify – be viewed as an obstacle in the quest to find truth. It is also an excellent quote for historians – myself especially – to remember. I tend towards easy generalizations and intuitively prefer to paint with broad strokes, and I know I am not the only one to be tempted by the faulty generalization bias.
If you go to Wikipedia and look up the list of cognitive biases, the wide variety of packages they come in becomes apparent. As such, it is not useful to try to distill one unifying feature out of all of them. In reality all of them need to be acknowledged separately and deliberately, if one wishes to overcome their effects when they check the results of their reasoning. The best way to describe cognitive bias, as Yudkowsky does, is by referring to them as errors in our reasoning arising from the shape of our own mental machinery. It is not that the machine is broken or does not have enough energy; it is just that it was built to make spaghetti when all we would like now is tagliatelle.
Improving one’s reasoning capabilities helps to avoid biases by giving us a kind of a checklist to go over when we are re-checking our reasoning and conclusions to see if they actually make sense. Do we have tagliatelle, or did we just made spaghetti again because that’s what always tends to happen when you crank the lever. The first draft of our thinking cannot rid itself of bias, but when we look at the pile of spaghetti mindfully, we can remind ourselves about the differences between various types of pasta. Only then can we grab a roller and begin to reshape the outcomes of our reasoning into something that serves our original purpose. As with spaghetti and tagliatelle, usually the results of our thinking are not so far off from what we were trying to achieve as to be unsalvageable. They just need a bit of work.
There is value in trying to obtain the most truthful answer to our questions regarding history, but historians should never forget that we are most likely going to be wrong in one way or another. What always strikes me when I read news reports of events where a reporter was actually present and several witness accounts were heard is just how often they still manage to botch the representation of the events. Historians are on this task as well, only we were never really there and did not even directly talk to the people who were. How likely must it be that our attempts to report on past events would have made the actual people involved aghast at their misrepresentation?
The amount of untruths is infinite, and truth itself is a difficult target to hit even when you have the chance to go back and empirically test if your reasoning was sound. We as historians have to do without this luxury. All we have is the garden views through our selected and blotched windows, and often not even that, just pictures drawn of the views by someone else. Nothing short of a new window or a removal of a stain in an already existing one can give us a chance to check someone’s reasoning in filling in the blanks in their drawing. So our chance of being mistaken and is already far greater, without even accounting for the fact that testing for bias is trickier for us than for most other scientists.
But that’s alright, after all what’s the worst that could happen? Luckily for us, nobody’s life (at least directly) hangs on representations of history, and there’s at least a dozen of us who can give a second opinion if someone makes a particularly unfounded and egregious claim. So we should be bold to try.
Scientific truth-seeking is a recent endeavor compared to how useful cognitive biases have been to humans in our everyday lives for hundreds of thousands of years. Therefore, we will have to make do with the spaghetti-making-machine that is our brain and see to it that we do not just leave it at that if what we actually seek is some tagliatelle.
¹ Yudkowsky, Eliezer. ”Rationality: from AI to Zombies.” Berkeley, MIRI (2015). 19–22.