Note: This post originally appeared on Medium in 2016. This a very lightly revised version with new formatting for ease of readability. It preceded the post on historical revisionism and anthropology of family but it tackles and elaborates on some of the same themes.
Outline of the argument
History is often accused of not being sufficiently scientific. To remedy this people try to come up with all sorts of theories of history that try to look like science. They come up with some measurable variable (e.g. availability of domesticable species as Jared Diamond did or energy output as Ian Morris did or size of population, advancement of technology, size of armies, etc. as many others do). This, they hope, explains the past and predicts the future.
Based on the theories, the would-be-scientist-historians build models that seem to make perfect sense but seemingly fall apart when overtaken by events. This makes history seem like it’s not at all scientific. But that’s only because we’re comparing it to the very rare instances in science where long-term perfect prediction is possible. Like the motions of planetary bodies or the behavior of a computer.
But in most cases, science struggles with perfect prediction. In medicine, it’s impossible to predict the exact course a disease will take in any one individual, even though it is often possible to predict what course it will take over a thousand individuals. In biology, it is impossible to predict, which features will be selected over others through natural selection. But it is possible to predict that some will be. But the best analogy, to my mind, is the weather.
Weather is the result of a large number (fractally infinite), perfectly knowable, describable and predictable physical events. But yet we cannot perfectly predict it even in hours or days and not at all in weeks or months. History (or rather any given present in the past or now or in the future) is also the result of a large (fractally infinite) number of relatively knowable, relatively describable and relatively predictable social events. Yet, we cannot predict it very well on any scale worth predicting.
The reason we don’t think of history as science but we think of meteorology as science even though both build models based on observation, known regularities and constants, is because the sort of reliable predictions a historian can make are of no use to anybody, while any kind of even moderately accurate weather prediction is extremely useful to everybody.
Here are some more things about the weather that can be used to view history. On most days, the weather is the same as the day before. You can predict the weather based on the current climate and what you know about the world for a few hours to days in advance (even if you still make errors). It is impossible to predict the exact weather (or history) months in advance but you can predict a range of possible weather patterns (e.g. summers in Europe can be warm or cold but there won’t be snow; political relations in the EU are warm or cold but there won’t be war).
You can also predict trends in changes in the climate but not the exact patterns or consequences or timings of those patterns (e.g. average global temperatures will rise but will it mean it will regularly snow in the summer in France?; disagreements between EU states will continue but will it mean a return to recurring warfare known up to the 1950s?). Most common mistakes in modelling the climate/weather and history come from confusing the local with the global (weather with climate and happenstance with trend) — e.g. during a cold summer, people may question global warming or the fall of the Roman empire was being predicted pretty much throughout its history until it happened hundreds of years later. Both weather and history can be disrupted by freak (Black Swan) events.
What this means is that history can be thought of as much more scientific than is commonly claimed without having to construct models that look like those of climatology.
History’s attractors
In semi-technical terms, weather is modeled using complexity theory. One of the properties of complex systems is sensitivity to initial conditions. Which means small changes in initial conditions can lead to large swings in the system’s state. This is often described as the ‘butterfly effect’ which is completely misleadingly described as ‘butterfly flapping its wings will cause a hurricane’. [See post on dangers of taking the analogy too far]
A much better example might be a weather front moving over a wooded area results in a tiny change of temperature that can then result in a hurricane developing if it was +0.5 degree Celsius or not developing if it was +1 degrees Celsius. Neither state ‘caused’ the hurricane — which was caused by the all the big forces that cause big things like hurricanes. But it was part of an initial configuration that structures the possibilities.
- We have a certain set of contexts in which certain patterns tend to occur and others do not occur at all. With weather this could be climate (local or global) so it doesn’t snow in the tropics or rain in January in the Arctic. In history, people cannot use technologies not yet invented, or generally new agents do not appear out of the blue. [This is the obvious part.]
- Rarely, freak events occur. You may get unseasonably warm weather in the Arctic, hurricane out of season, snow flurry in the tropics. In history, the Mongols, Conquistadors, ISIS came seemingly out of nowhere (and were not in any way predictable by models based on standard assumptions).
- But outside of the freak events, you can predict the range of weather patterns in a given place during a given time period. There’s a range of weather patterns on Earth and weather stays within the limits set by those patterns. And you can do the same for history. Certain types of things are likely to happen (similar to things that have happened) and others are not — e.g. Alien invasions, raptures are unlikely to happen but invasions, civil wars, famines and epidemics are happening all the time.
- It is possible to predict local weather patterns with a decent level of precision based on models in the extreme short term (hours to days) but even these predictions are subject to significant errors due to assumptions made in the models. E.g. it may matter whether a front went over a wooded area or a field. It is possible to accurately predict historical events in the extreme short term — hours to days — simply based on various models of causation (election results, scheduled events, advancing armies) but it is still subject to frequent errors due to assumptions in the models
Fractal histories
Above, I used the term ‘fractal infinity’. This is not (as far as I could find out) a technical term. It’s a metaphor based on the coastline paradox made famous by Madelbrot’s paper ‘How long is the coast of Britain’. This is the description from Wikipedia:
the length of the coastline depends on the method used to measure it. Since a landmass has features at all scales, from hundreds of kilometres in size to tiny fractions of a millimetre and below, there is no obvious size of the smallest feature that should be measured around, and hence no single well-defined perimeter to the landmass.
This means that potentially, there can be an infinite (or an indefinetely large) number of measuring units which is the paradox bit. But more interestingly these coastlines have patterns that are similar to each other across scales. This is how Madelbrot put it:
Geographical curves are so involved in their detail that their lengths are often infinite or, rather, undefinable. However, many are statistically “selfsimilar,” meaning that each portion can be considered a reduced-scale image of the whole.
Why is this relevant to history? I don’t want to overegg the analogy. Mandelbrot’s contribution was a mathematical description of these types of geometric objects and it worked well in certain quantifiable contexts. But that’s not what the study of social objects such as history needs. Those same quantifications won’t work on more analog problems. However, the analogy can explain a huge problem in historical analysis — namely conflation of similar patterns at vastly different levels of magnification.
Imagine history (ie. events as revealed to a human observer over time) as a landscape. As you fly over the landscape from a certain height, the different areas seem very similar or possibly identical (from a large enough distance, Earth will look like a dot). So, from a large enough distance all history will seem like striving for resources of groups of people. So it will make sense describing history as that.
When you zoom in a bit closer, you will see a great inequality in the struggle for resources so you may want to describe history in terms of ability to project power. And you may want to quantify those differences.
But if you zoom in even closer, you will see patterns that look very similar. Even though the groups are of different sizes, they all have very similarly hierarchical organization — with some sort of leadership on top which has to change over time. So, you will try to come up with rules for the formation of these hierarchies.
Yet, as you zoom in even more and you will notice completely different customs, ways of negotiation, ways of legitimizing. So you will postulate a complete incomensurability and simply describe the difference (kind of like a taxonomist biologist).
But then you zoom in even more at the level of individual motivations and you will see things like lust, hunger, aims, struggle for personal success, family — and then you can postulate that we are all the same.
But the problem is that it is actually relatively easy to describe the different levels of magnification in terms of each other because they are sort of like models of one another. An individual’s desires and dreams can be recast in terms of striving for resources; and the behaviors and customs at the level of a kingdom can be talked about in terms of individual desires.
So, the ‘dimensions’ of history depend on the length of the yardstick used to measure it. You can describe the same event from the ‘big man of history’ perspective, the ‘social forces’ perspective, ‘struggle for resources’ perspective or ‘long-duree’, inevitable forces of history perspective. Just like you can describe a chemical event at the level of the structure, molecular interactions or matter transformation.
But all of these levels/perspectives have their own rules and seemingly causal patterns. They also interact with each other but in complex ways that preclude simple reductionism. That is, we cannot say that interactions at one level of magnification cause interactions at another in the same way that an individual brick (or the laying of it) does not cause a wall to exist.
Aside: Wall metaphor of causality
I’ve come to like the ‘wall metaphor’ of causality. We have completely different intuitions about the causal chain of events resulting in the existence of a wall than we have about the chain of events that come to result in a historical event.
But perhaps rethinking historical events as walls can be helpful — as long as we also keep the differences in mind. We have the brick makers, brick layers, but also the commissioners, approvers, weather conditions, foundations, historical custom of brick making and brick laying — all of those play a role in the sort of wall we’re going to get.
And it is obvious that depending on our perspective the causal chains are going to be completely different. And the same goes for when we try to destroy the wall. We need commensurate forces — big enough force to make a dent, but we also need a lot of small events down to the molecular level. And different perspectives will identify different causal chains. All of them correct but obviously belonging to certain levels. These are obvious (or fairly obvious) when it comes to walls. But not so obvious when it comes to history.
Most good historians actually seem to have the same kind of intuitions about events as most of us do about walls. But they are often seduced by the customs of the genre of history writing into jettisoning their intuitions and coming up with a reductionist perspective instead.
What this means for the notion of causation in social science
We tend to think of causality in history as happening at the lowest level of magnification and accumulating into the total. And on a most straightforward account, this makes perfect sense. A lot of apples put into a basket + the basket, will make a basket full of apples. But that’s an unhelpful way of looking at causality because it implies a certain level of atomism — a final level of further indivisible measuring units. But that level cannot exist — or if it does exist it has to be so low (subatomic) that the whole cannot be modeled using it. Even if we believe in some sort of terminal particle, it is a fool’s errand (looking at you Stephen Hawking) to try to actually use it to measure object-level phenomena with it.
Which is why the butterfly causing a hurricane is such a poor way of thinking about sensitivity to initial conditions. No one aspect of the physical world ‘causes’ the weather — not the extra +.5 degrees Celsius of temperature, nor the gravitational pull of the moon or the Gulf stream. They constantly interact in a system at all levels. But our ability to model weather is supremely dependent on tiny deviations in measurement.
And the same holds for history and other social systems. No one event causes another event except in the most trivial sense. And an accumulation of little events does not ‘cause’ big events. Because there are no smallest events that could be said to constitute the final level of magnification for initial conditions. But our ability to perceive small events does have huge implications for our ability to model the future patterns of events.
It may have made a difference that Napoleon looked to the left instead of right during a battle thus winning it, which in turn led to his toppling of the Prussian state. Which in turn may have made a difference to the shape of the First World War or the Third Reich. But did that glance left actually cause any of those things or led to those things? Not outside the imaginations of writers of time travel scince fiction. It was just a small difference in the pattern (initial conditions) that made the system come out with a different state at a certain level of magnification.
But on another level, even if Napoleon had never been born, the system may have looked very much the same from a certain level of magnification (European states struggling for resources).
We can have models where big things cause other big things and little things cause other little things. And we must be careful to always know what level of magnification we’re talking about. But we must remember that we are talking about our models of what happens, not what actually happens in its totality. The bigger the models, the bigger errors in our measurement of initial conditions — or rather with the big models we actually have no hope of complete measurement of the initial conditions or any sort of computational tractability even if such measurement were possible.
So even very good and complete models of history have by definition no way of predicting anything with any level of accuracy into any kind of future. And we cannot use their accuracy on past events because our measurement of the data for the past is already filtered by what happened (ie history is written by the winners — or at least, the record keepers). We don’t have that kind of filter for the present or recent history.
We could use what we know from the past to help us sift through the data of the present but that’s where the initial conditions lie that can completely mess up our predictions. Even if we had a lawful model of individual psychology and small event dynamics — in the same way weather modelers have accurate models of molecular and Newtonian object physics — we still could not do a better job at prediction than the weather people can. And in fact, any individual success at prediction is no guarantee of the quality of the model over the long run.
What this means for Jared Diamond, Ian Morris, Niall Ferguson or CGP Grey
This essay was inspired by a recent podcast discussion of Jared Diamond’s now classic but highly controversial ‘Guns, Germs, and Steel’. But I really started thinking about it when reading Ian Morris’ ‘Why the West Rules: For now’ and Niall Ferguson’s ‘Civilization’. They both suffered from the problem of unconscious magnification refocus.
Ferguson — who’s by far the sloppiest thinker of the three though by no means a worthless one — was the most illustrative example of this. He even could not fix on the idea of the West was for more than about half a chapter.
Morris, on the other hand, was the strictest in his assumptions and measurements — in effect creating two books — the argument and the footnotes. But even he seems to compress time periods and plays around with effect sizes and scales to make the whole thing work. The same thing Steven Pinker is doing in the infuriatingly flawed but worth reading ‘Better Angels of our Nature’.
I think of these as modern historiographical eschatology and it is important to read good anthropologists to fully understand what these people are reading out.
A perfect companion to all of these were David Graeber’s ‘Debt: First 5000 Years’ and most recently for me (although the earliest in publication date) Eric Wolf’s ‘Europe and the People without History’. Also worth reading are the most recent books on Atlantic history. But not of inconsiderable interest is even dross like Pat Buchanan’s ‘Decline of the West’ because it gives an example how many people are thinking about causes.
They all think of a theory of history as a collection of hypotheses about historical causality. But we already know what the causal chains are. Or, the good historians do. But then they forget about most of them when alighting on a good grand theory of history. Jumping from weather to climatology but then using the language of the climate to talk about the weather when they come to the lessons for us today or when they zoom in on the period of history they know really well.
CGP Grey when defending Diamond suggested that people keep criticizing Diamond for all the small things but they never substantively critique his grand narrative. That has also been my initial impression of many of the critiques.
But the criticism of Diamond is actually about him swooping from his heights of resource-utilization level of modeling history down to the level of individual events where he is much shakier and trying to use the same models on the small events. So this leads to a completely inaccurate description of what happened in the Americas — where the guns and steel mattered almost not at all and the germs relatively little (see here for more).
What mattered in the conquest of the ‘new’ world were the local politics — the conquistadors became enmeshed in local politics and their victories were results of alliances with other local factions. At the same time, the Portuguese had no chance of anything like that happening on the West coast of Africa or in India (even as they were winning some important naval victories against the Ottomans who had just as much steel, better guns and the same germs).
In some way, Diamond’s critics (who would not be caught dead at an NRA rally) are saying ‘guns don’t kill people, people kill people’. They are saying, real people killed and enslaved other real people — and we must judge what happened for what it happened. It wasn’t the guns, germs or steel that did it. It was our venerable ancestors that did it. And we’re still doing it — albeit in a less mindless genocidal way.
Diamond’s stated intention is to do away with the racial superiority explanation of the difference in the current power arrangements. And he explains a part of it. But he is not careful enough to explore the boundaries of his model. And when applied at the right level, his model does what it sets out to do. But when applied at other levels of magnification, it does exactly the opposite. It provides a way of justifying real bad human behavior as inevitable.
This is perhaps most starkly exemplified in his description of the Rwandan genocide in his other book ‘Collapse’. There he recasts it in terms of simple competition for resources. He may be right. But as recent Timothy Synder’s book on the holocaust argues, that same was true for the Nazi genocide (in the idea of lebensraum). But those were not causes. This involved people like us shooting other people like us. Up close and personal. And other people telling them to do it and benefiting from it in many ways.
Diamond’s models can explain some of the dynamics but they can only do it if they leave a lot of important information out — and the argument is that that’s the information that should matter to us here.
CGP Grey went on to ask, give me a better theory of history. This is a response to that challenge. History is like the weather. This is more a way of thinking about historiography than history itself. Kind of like the language turn in philosophy. We need to be careful with our models, just like we need to be careful with our language. As the saying goes, ‘all models are wrong, but some of them are useful’.
Stack fallacy, hierarchical structure in science and the meaning of history
As I was writing this, I came across this article on the Stack Fallacy which is making the same point in a different domain. You cannot just assume that because you’re good at building all the building blocks, you can build the whole structure. Brick makers are no good at building walls by virtue of being good brick makers and conversely brick layers are not going to be any good at brick making just because they can use bricks to build durable structures. (The post makes the point about databases and CRMs but its even starker at the physical level).
One of the commenters pointed to an old paper by P. W. Anderson ‘More is different’ that makes a similar point about physics [my emphasis].
The ability to reduce everything to simple fundamental laws does not imply the ability to start from those laws and reconstruct the universe. In fact, the more the elementary particle physicists tell us about the nature of the fundamental laws, the less relevance they seem to have to the very real problems of the rest of science, much less to those of society.
The constructionist hypothesis breaks down when confronted with the twin difficulties of scale and complexities. The behavior of large and complex aggregates of elementary particles, it turns out, is not to be understood in terms of a simple extrapolation of the properties of a few particles. Instead, at each level of complexity entirely new properties appear, and the understanding of the new behaviors requires research which I think is as fundamental in its nature as any other.
A different way of restating this thesis is that the fundamental laws of the lower levels of organization apply at the higher levels (e.g. bricks have strengths and periods of decay and that will impact on how much a wall will withstand or how long it will stand up) but the higher levels have new emergent properties that cannot be easily described in terms of the lower levels. And vice versa, the lower levels cannot be described in terms of the higher levels of organization. However, the structures emergent at all levels can interact with structures emergent at other levels.
What does this mean for history? It can very well matter that a king was a violent drunk who lusted after his subjects’ daughters and wives, and therefore alienated all his followers leading to the collapse of a dynasty. However, when we look at the patterns of falls and rises of things like dynasties, we do not benefit from describing them in terms of individual psychology interacting with their immediate social norms.
That does not mean that the individual psychology and the social norms don’t matter to that individual king or individual dynasty. But falls and rises of dynasties just don’t rest on these issues there are broader patterns and dynamics — kings with highly praised kingly behavior cannot turn around the fall of a dynasty without resources and with external pressures and incompetent despised rulers do not tend to ruin powerful dynasties overnight.
To bring it back to the reimagined ‘butterfly effect’ metaphor. There are valid patterns that can be recognized but patterns are not causes. The exact shapes of the patterns or their varieties are extremely sensitive to initial conditions but the initial conditions are not the causes. Everything that comprises that pattern interacts together (that would include external inputs) to produce all its phases/shapes. It may even be inaccurate to call the pattern itself sensitive to initial conditions. It is our ability to make predictions about its future states and possibly judgments that is extremely sensitive to tiny errors in our observation of what we decide are the initial conditions for the purposes of our prediction making.
With certain weather patterns — like storms — the initial conditions are relatively easy to locate, if relatively difficult to measure. With historical events, it is a little more difficult. It is not always clear where we should look at the initial conditions. It is not uncommon for historians to say things like ‘but the real roots of this crisis lay much further in the past’ — Niall Ferguson’s ‘Civilisation’ is a case study in how badly this can get out of hand.
But historians also tend to freely jump between the levels. So in a recent LSE lecture, Ian Morris talked about long term (in the 1000s of years) trends in war and violence rates. He described an overall long-term equilibrium in violence necessary for the survival of a species. But then he started to talk about our individual natures from an evolutionary perspective, only to turn the talk into a discussion about the motivation of the elites to keep workers happy and alive to produce goods. And then he talked about the British Empire, Cold War, American supremacy and tried to draw conclusions from that.
That’s like saying one of these:
a) ‘there was a storm because it is summer when storms happen’ b) ‘this storm really began when the icebergs melted’ c) ‘there will be a storm on Tuesday, July 3rd, next year because we are experiencing global warming’
Intuitively, none of these statements make sense, even if they may be actually true. Of these, a) is uninformative, b) is also relatively uninformative because we could equally well go back to the formation of the Earth or the Big Bang but other than the arrow of time, we would have little in the way of modelling the causation, and c) is also a statement that is not useful even if it turns out that there actually is a storm on that day. Even if accurate, since our model is consistent with there being a storm on any day in July next year, it is not a useful prediction.
Yet, historians make statements like these all the time — on the surface, they are not this starkly nonsensical but if you peel away the narrative layers and just get down to simple causation, you get things that look like these. At the extreme, you get things like attributing the decline of the Roman empire to lead piping or the rise of the British empire to boiling tea. But even more complex interpretations of past events suffer from similar difficulties.
For instance, post invasion, dismissing all the Bahtist army officers is often cited as the roots of the current violence in the region. But we could plausibly imagine violence of different type but similar scale ‘resulting’ from not dismissing them. Similarly, when things go well in South Africa, everybody’s pointing to the peace and reconciliation for not creating any new resentments and drawing a line under the past. And when they go badly, people wonder if reconciliation was a problem because it did not give people justice or just stirred up trouble.
Ultimately, we cannot say much more than both are things that could happen. We simply have no way of tracking the causal chains at the level in which we could even run the models.
Critics of the social sciences argue that unlike the meteorologists who can build their larger weather models on solid physics, chemistry and geography, historians don’t actually have solid enough psychological models underlying their bigger sociological models. But that is to make the same error. Meteorological models can just about predict the weather tomorrow and the day after while sometimes making huge errors. Historical models are just as good. Equally, historical models can predict larger trends about as accurately as climatology. But we often treat them as if we thought it was possible to predict the weather a year from now.
So what is the point of history then? Its accurate predictions are not very useful and its useful predictions are not very accurate. Any statement like ‘people who forget their history are doomed to repeat it’ are nonsense. Knowing exactly what happened in the past is no better than knowing what the weather was 10 years ago. Other than knowing that anything that happened in the past can happen again, we’re no better off.
But the weather analogy can come to the rescue. On most days, for most people, it does not actually matter what the weather is going to be like tomorrow. Yet, people obsessively check their forecasts. It is interesting. And also it is something to talk about. And that is not nothing. Historical knowledge — you could argue — is even more valuable. It is something to talk about but unlike the weather (unless you think God sends it as reward or punishment), it can be used to help us make sense of today — not in a causal manner but in a narrative one. And the smart historians know that.