Popular Meme versus Expert Theory

The idea of 4d chess turned into a popular meme with Trump's victory in 2016 (helped along by narratives like the breathless Master Persuader one peddled by Scott Adams).

In the popular and intuitive perception, the idea of higher-dimensional games goes along with higher-dimensional strategies. This is 4d chess in the sense portrayed by Bradley Cooper in the movie Limitless, where he takes an intelligence-enhancing drug and is able to think on more levels, more moves out, in pursuit of more complex intentions, and powered by deeper insights. This understanding of 4d chess -- and what constitutes effective play within it -- is why the ridiculous QAnon conspiracy theory has unironic adherents. QAnon is the theory that a Master Persuader is executing a Master Plan too subtle for lower-dimensional mortals to comprehend. There are bigger problems with this idea of 4d strategy besides the obvious one that Trump doesn't look like he's thinking on more levels or more moves out (in fact, it is fairly obvious he's thinking on fewer levels and fewer moves out; if that's acting, he deserves an Oscar).

The bigger problem is that this understanding of higher-dimensional games is at odds with our best theoretical understanding of higher-dimensional thinking, which suggests that higher dimensional games require lower-dimensional strategies. In other words, thinking on more levels, more moves out, isn't even theoretically the right thing to do. You have to go in the other direction: fewer levels, fewer steps ahead.

Often, when a popular understanding conflicts with an expert understanding, (and it's not obviously due to pure ignorance or a superstitious belief system), there is some sort of shallow semantic confusion. For instance, we commonly talk of "steep learning curves," which technically correspond to easier learning (since you learn more of the subject matter in a shorter time). But here the conflict is shallow: we're really just using the metaphor of a steep hill being harder to climb, not talking about actual learning curves. Nobody actually believes that tougher things can be learned more easily. You can fix the confusion very easily by avoiding talk of steepness and speaking instead of long versus short learning curves, where popular and expert understandings of the words coincide.

But the 4d chess meme is different, and here the reality matches our counter-intuitive theoretical understanding of higher-dimensional games better. The popular and intuitive perception is an actual, consequential, and deep misreading of things going on. Why?

Deterministic versus Randomized Strategies

Here's the key insight from computer science: under fairly general conditions, more random strategies are cheaper to extend to higher dimensions than comparable deterministic ones. 

A simple illustration (see picture) can be found in the problem of approximately computing the area of an irregular shape. One way is to superimpose a square grid and count the fraction of grid points that fall inside. Another way is to draw a square around it, generate random points inside the square, and count the fraction of points that fall inside (your basic Monte Carlo method).

Turns out, as you increase the number of dimensions, the first (structured and deterministic) approach rapidly becomes computationally intractable. The second approach however, easily scales to high dimensions. This important insight (which won a major prize in 1991) is at the heart of many modern algorithms of practical importance, including the latest and greatest "deep" learning algorithms in AI, and many classic adaptive control algorithms.

I don't know of any work systematically applying this insight in the context of higher-dimensional chess in particular (since there are many ways to construct the rules of such a game), but it's easy to see why and how the game might get simplified and why random moves might get less costly.

In regular 2d chess for instance, achieving checkmate generally requires attacking the opponent's king in 2 ways simultaneously. The king can't move. But in 3d, the king might be able to escape vertically, so you'd generally need three simultaneous attacks to achieve checkmate. Defending against checkmates gets easier, achieving checkmates gets harder.

This is not an arbitrary, isolated effect. A closed planar curve partitions a plane into an inside and outside, but in 3d a bug trapped inside a circle can simply hop out. You need a sphere to trap it. In 3d, knots are possible. In 4d, knots are not possible. In 2d, laying out a network of roads (or a circuit board) without unwanted intersections is hard. In 3d, you just build flyovers and tunnels.

Life is just easier in higher dimensions. Unless you want to "win". 

In general, these sorts of qualitative effects in higher dimensional games suggest it is harder to achieve deterministically defined win states like checkmate. On the flip side, the downside risk to more thoughtless action is mitigated, because there are fewer ways to get fatally trapped. Or to put it another way, the number of ways to escape from traps increases faster than the number of ways to get trapped. Antifragilistas among you might have spotted an obvious implication: Extreme downsides get mitigated and you can gain more easily from uncertainty.

In 2d, moving your king carelessly might expose it to checkmate. In 3d, the king can move more carelessly with less risk. The extra dimension weakens the attacker more, and empowers the defender more.

The short of it is this: in higher dimensions, it gets rapidly harder to win, but rapidly easier to not lose. So adding dimensions turns symmetric games into asymmetric games: people who want to win get weaker. People who just want to keep playing get stronger. Ineffective players try to "win" in some finite sense. Effective ones try to goad opponents into going for the clear win, while making sure they simply stay in the game and don't lose.

A very nice illustration of this is in an episode of Star Trek: TNG where Data faces off against Sirna Kolrami, the "galaxy's best strategist" in the game of Strategema. Data initially loses, but then decides to play for the draw rather than win, leading to his opponent quitting out of frustration, since as an Android, Data could effectively keep going without getting bored or frustrated, drawing on an effectively infinite reservoir of what for him (but not for Kolrami) was a free resource: time and attention.

This should remind you of guerrilla warfare.

Asymmetric Game Regimes

Let's talk about war for a bit, though not all higher-dimensional games map to war.

Over the past century, war has gotten more asymmetric as it's gotten higher dimensional. World War I, with weak air power and primitive tanks was all about the highly symmetric condition of attrition warfare. World War 2, where air power reigned supreme and advanced tanks combined with motorized infantry to emable the Blitzkrieg model, was less symmetric. Vietnam, where complex political dimensions and the global Cold War backdrop entered the equation, was strongly asymmetric.  And by the time we get to Iraq and Afghanistan, there is no meaningful definition of winning, but there is a definition of losing: getting into nation-building and failing. And today we are in the age of cyberwarfare, which is so highly asymmetric, a small team of state-sponsored hackers could use the NotPetya ransomware to bloodlessly bring huge multi-national corporations to their knees and cause billions of dollars of damage.

During this evolution, war went from stylized conflict with rules derived from honor codes, to a pattern of "total war" with no rules, to a blurring of lines between war and peace, to software eating war.

One interesting effect was that military objectives for typical pairs of adversaries diverged. As Henry Kissinger noted, "the conventional army loses if it does not win, the guerrilla wins if he does not lose". The asymmetry lies not just in the relative weakness of the guerrilla, but in the fact that the conventional army is compelled by doctrine and tradition to try and "win." The guerrilla is more powerful in being doctrinally enabled to fight for a less conventionally honorable outcome. There is, in a way, a higher-level symmetry here. The guerrilla army is weaker in terms of resources but stronger in terms of doctrinal freedom of action.

Chess models conventional warfare, where both sides share a win condition: checkmate. If you do not achieve checkmate, you either lose, or you end up in a "draw." But note that outside the formal rules of the game, a draw in chess is in a sense a loss for the party with lower endurance (for example, if you didn't recognize the inescapable draw condition and continued a futile, infinite, trivial end game until your frustrated opponent conceded just to get on with their life).

This kind of "draw" of course, is analogous to the long, terrible trench warfare stalemate of WW I.

But in guerrilla warfare, the "draw" is not equally harmful for both sides. For the conventional army, a stalemate condition is enormously expensive. A continuous drain of resources. For the guerrilla, it is a cheap way to continue the game and live to fight another day. 

A very clear example of this effect is drones versus air defenses. It only takes a cheap hobbyist drone, worth a few hundred dollars of off-the-shelf-parts, to pose almost as much of a threat to an adversary today as a fighter jet 20 years ago. But defending against such an attack is approximately just as expensive: tracking and shooting down an airborne enemy with a missile is a problem that takes a few hundreds of thousands of dollars. So a guerrilla army can cause a conventional army to slowly bleed billions while itself only spending hundreds of thousands.

The key to note here is that guerrilla warfare is not particularly complex in a deterministic sense. It does not rely on the strategic wisdom of a long military tradition, complex officer education, or training in advanced weaponry for the regular troops. Even illiterate farmers can run guerrilla warfare playbooks.

What makes it work is enough support of the populace that you can disappear across the civilian/combatant boundary easily, and the limited, simple objectives: survival and random, low-level harassment of the bigger adversary. 

Sophisticated Naïveté Eats Naive Sophistication

So the popular theory of Trump playing 4d chess is both right and wrong. Yes he's playing 4d chess effectively. No, it is not by employing a naively "sophisticated" 4d strategy and thinking on more levels and further out than his adversaries. He's been doing it the theoretically sound way: by trying not to lose, like Data or the Vietcong. In higher dimensions, sophisticated naïveté eats naive sophistication for lunch.

In algorithmic terms, Trump adopted a low-complexity, highly randomized "algorithm" that had only a few key elements: pushing buttons, baiting and name-calling, distracting from negative attention by stoking new controversies, and simply adding more complexity/dimensionality to the game itself, by opening up new fronts constantly. As he himself noted once, "anything that makes things more complicated, I do." This is an excellent folk articulation of the principle that adding dimensions hurts the player going for the explicit win more.

Viewed in the light of 4d chess, the US political system is in fact set up for such asymmetry. In a formal sense, the 2016 election ended in a draw of sorts: Hillary won the popular vote while Trump won the electoral college vote. The system translated this outcome into a loss for the stronger side: the one capable of winning the popular vote. The US system is a 4d chess board compared to simpler political systems.

The electoral college system is not designed to allow less populous states to win; it is designed to ensure they stay in the game. The "game" here isn't the narrow electoral game, but the story of America itself. This broader game is set up to ensure political power, once gained, is not lost as easily as economic or cultural power.

Is this a good design? Depends on what you care about. Arguably, in something like a unicameral parliamentary system with true proportional representation, the Trump "base" would have been electorally wiped out and lost all political power over the last 30 years. The system worked as designed, acting as a check against a potential tyranny of the majority, and keeping the minority political interest alive in the game. For now.

While I have other, more ideological, problems with Trumpism, I can't really argue with the principles underlying the design of the system, or the outcomes it delivers. Protecting against the tyranny of the majority is an important principle to follow in general, whatever the bugs of a specific design or your issues with a particular minority political ideology. 

Now, Trump is engaged in another game of high-dimensional chess against the investigation of Robert Mueller, but from the other side. Again we see a mythologization of the game. This time, all sorts of godly multi-level, multi-move-ahead strategic thinking is being attributed to Mueller.

I suspect he is following a relatively simple and randomized strategy of just probing for any available legal vulnerability, while making sure not to provide any excuse for the investigation to be shut down. 

Mueller doesn't have to win. In fact, he cannot, since he cannot indict a sitting President the way he can an ordinary citizen. All he can do is play to not lose, by keeping the investigation open and opening up more and more fronts. Once he completes his investigation, it is up to Congress to decide whether or not to move for impeachment on the basis of the evidence. So long as he merely defends against being shut down, he cannot lose. So long as he completes his investigation with any findings at all, only Trump, Congress, or both can lose. 

Trump, on the other hand, is forced to try and go for the win -- which means trying to shut down the investigation. Which means he can lose.

Why did Trump enjoy the advantage of asymmetry in one case and not the other?

In the election, the game was irreducibly complex for Clinton, due to the constraints of being a conventional candidate bound by conventional party norms -- she was doctrinally bound like a conventional army. She could not simplify and randomize her strategy without losing all her resources. Trump had no such dependence on a political machine with its rules; he was doctrinally unbound, which more than made up for his weaker resource base.

But in the case of this investigation, Mueller, not being a politician with an office to lose, is free to act in ways Trump is not. But for Trump, the complex, debt-ridden, and murky business empire is the high-dimensionality 4d chessboard he cannot escape. To make things worse, as President, he has more constraints limiting his actions than he had as either a businessman or a candidate. Simplify-and-randomize is still available to him as a strategy to keep his base restive and active, but is useless in his legal battles.

Whether or not that proves to be a fatal liability for him remains to be seen.

A High-Dimensional Future

Software eating the world is the same thing as the world getting higher-dimensional. Software loosens constraints, and enables more degrees of freedom in more places, allowing more complicated, higher-dimensional drama to unfold everywhere. More variables, more movement, more confusion, more noise.

The Saga of Trump is just one early story in what promises to be a really high-dimensional future. Playing to win is foolhardy.

Playing not to lose, however, is smart. This means being open to deceptively simple-seeming strategies -- so long as the embrace the full dimensionality and have the right order of randomness -- is something anyone can do, regardless of the number of game levels they can handle or number of moves ahead they can foresee.

So how do you play to not lose at 4d chess?

Simplify your actions, increase your randomness, complicate the game for the adversary, and try to stay alive. 

And perhaps the most paradoxical principle, to win you have to avoid trying to win.