Complexity: A Guided Tour by Melanie Mitchell

Summary

Seeks to explain how large scale, complex, organized and adaptive behavior can follow from simple rules among many individuals

Key Takeaways

Complex systems – a system in which large networks of components with no central control and simple rules of operation give rise to complex collective behavior, sophisticated information processing and adaptation via learning or evolution
1. How large numbers of relatively simple entities organize themselves, without benefit of any central controller, into a collective whole that creates patterns, uses information, and, in some cases, evolves and learns.
2. Many simple parts are irreducibly entwined, and the field of complexity is itself an entwining of many different fields
3. Systems in which organized behavior arises without an internal or external controller or leader are sometimes called self-organizing. Simple rules produce complex behavior in hard-to-predict ways, the macroscopic behavior of such systems is sometimes called emergent
4. Another definition of complex systems – a system that exhibits nontrivial emergent and self-organizing behaviors
5. Order is created out of disorder, upending the usual turn of events in which order decays and disorder (or entropy) wins out. A complete account of how such entropy-defying self-organization takes place is the holy grail of complex systems science
6. Brain network, ants, immune system, world wide web, economy are all excellent examples of complex systems
  1. Ants are one of the simplest organisms but when millions of them are working together they can achieve “collective intelligence”
  2. Brains, like ant colonies, have billions of neurons (ants) working in parallel without central control
  3. Information processing has taken an ontological meaning similar to mass/energy, namely as a third primitive component of reality. In biology in particular, the description of living systems as information processing networks has become commonplace.
    1. Information processing seems to play a leading role in natural systems – immune system, ant colonies, cellular metabolism
7. Prediction of complex systems impossible as can never know starting conditions precisely and small changes lead to huge differences in outcomes
  1. However, there are universal traits to chaotic systems: period doubling route to chaos (bifurcation) and Feigenbaum’s constant
Revolutionary ideas from chaos
1. Seemingly random behavior can emerge from deterministic systems, with no external source of randomness
2. The behavior of some simple, deterministic systems can be impossible, even in principle, to predict in the long term due to sensitive dependence on initial conditions
3. There is some “order in chaos” seen in universal properties common to large sets of chaotic systems
Dynamical systems – description and prediction of systems that exhibit complex, changing behavior emerging from interaction of many components
Nonlinear system – whole is different from the sum of the parts
1. Attractors – fixed point, periodic, chaotic (logistic map)
Entropy – energy which can’t be converted to work and turns to heat
1. The second law of dynamics is said to define the “arrow of time” in that it proves there are processes that cannot be revised in time (heat spontaneously returning after work is done). The “future” is defined as the direction of time in which entropy increases. Why the second law of thermodynamics is different from all other physical laws in that it should distinguish between the past and future while all other laws of nature do not is perhaps the greatest mystery in physics
Thermodynamics describes energy’s interaction with matter
Reductionism great but it fails (so far) to explain chaos theory. Anti-reductionism systems are situations where the whole is more than the sum of its parts. Chaos theory, systems biology, evolutionary economics and network theory move beyond reductionism to explain how complex behavior can arise from large collections of simpler components. These disciplines require multi-disciplinary thinking from fields such as cybernetics, synergetics, systems science and complex systems
How does intelligence and consciousness arise from nonmaterial and nonconscious substrates?
Statistical mechanics – proposes that large-scale properties (heat) emerge from microscopic properties (motion of trillions of molecules)
Information is processed via computation
Turing’s accomplishments – defined notion of “definite procedure”; definition, in the form of Turing machines, laid the groundwork for the invention of electronic programmable computers; showed what few ever expected in that there are limits to what can be computed
Darwin had single best idea ever – “in a single stroke, the idea of evolution by natural selection unifies the realm of life, meaning and purpose with the realm of space and time, cause and effect, mechanism and physical law.”
1. Evolution gives appearance of design with no “designer”
Self-reference in DNA – complex cellular machinery – mRNA, tRNA, ribosomes, polymerases and so forth – that effect the transcription, translation, and replication of DNA are themselves encoded int that very DNA
1. It is both information and input!
Shannon entropy – one simple measure of complexity is size so Shanon entropy is the average information content or “amount of surprise” a message source has for a receiver
1. The most complex entities don’t have the most order or randomness but fall somewhere in between
Fractals have non-integer dimensions. Koch curve has 1.26 dimension and this is what makes them so strange
Simon contends that evolution can design complex systems in nature only if they can be put together like building blocks – have hierarchy and are non-decomposable. Cell can evolve to become a building block for a higher level organ, which itself can become a building block for an even higher-level organ and so forth
Most agree life includes autonomy, metabolism, self reproduction, survival instinct and evolution and adaptation
1. Dual use of information (as instructions AND data) avoids self-referential loop (how DNA replicates)
2. Van Neumann proved in principle that computers can self-replicate
3. Holland studied if programs could breed, adapt and evolve (professor at UMich)
Genetic algorithm – output is solution to a problem
1. Job of genetic algorithm is to find (evolve) to good strategy once encoded
2. Many real world applications and solves problem often hard for people to see why it works
Parallel traced scan – many, if not all, complex systems in biology have a fine grained architecture, in that they consist of large numbers of relatively simple elements that work together in a highly parallel fashion. Several possible advantages arise out of this type of architecture including robustness, efficiency and evolvability. One additional major advantage is that a fine-grained parallel system is able to carry out a parallel traced scan which is a simultaneous exploration of many possibilities or pathways in which the resources given to each exploration at a given time depend on the perceived success of that exploration at that time. The search is parallel in that many different possibilities are explored simultaneously, but is “terraced” in that not all possibilities are explored at the same speeds or to the same depth. Information is used as it is gained to continually reassess what is important to explore
1. Allows many different paths to be explored and allows the system to continually change its exploration paths since only relatively simple micro-actions are taken at any time
2. The redundancy inherent in fine-grained systems allows the system to work well even when the individual components are not perfectly reliable and the information available is only statistical in nature. Redundancy allows many independent samples of information to be made and allows fine-grained actions to be consequential only when taken by a large number of components
3. Continuous interplay of unfocused, random explorations and focused actions driven by the system’s perceived needs. Early explorations, based on little or no information are largely random and unfocused. As information is obtained and acted on, exploration gradually becomes more deterministic and focused in response to what has been perceived by the system.
4. This balancing act between unfocused exploration and focused exploitation has been hypothesized to be a general property of adaptive and intelligent systems
Meaning – the meaning of an event is what tells one how to respond to it
Computers, unlike humans, lack sensitivity to context, a lack of ability to use analogies
1. Humans are very good at perceiving abstract similarities
Idea Models – relatively simple models meant to gain insights into a general concept without the necessity of making detailed predictions about any specific system
1. Maxwell’s demon – exploring the concept of entropy
2. Turing machine – defining “definite procedure” and exploring computation
3. Logistic model and logistic map – minimal models for predicting population growth, dynamics and chaos in general
4. Von Neumann’s self-reproducing automaton – exploring the “logic” of self-reproduction
5. Genetic algorithm – exploring the concept of adaptation. Sometimes used as a minimal model of Darwinian evolution
6. Cellular automaton – complex systems in general
7. Koch curve – exploring fractal-like structures such as coastlines and snowflakes
8. Copycat – human analogy making
Prisoner’s dilemma – pursuit of self-interest for each leads to poor outcome for all
1. Tit for Tat is the best strategy with the first being cooperation
2. Predictability is important for cooperation
3. Close proximity aids cooperation
People have poor intuitive understanding of coincidence
Network thinking will permeate through all human activity and inquiry
1. Scale-free degree distributions, clustering and the existence of hubs are the common themes. These features give rise to networks with small-world communication capabilities and resilience to deletion of random nodes. Each of these properties is significant for understanding complex systems, both in science, technology and business
2. Means focusing on relationships between entities rather than entities themselves
3. A major discovery to date of network science is that high-clustering, skewed degree distributions and hub structure seem to be characteristic of the vast majority of all the natural, social and technological networks that network scientists have studied
  1. Hubs – high-degree nodes and are major conduits for the flow of activity or information in networks (Google)
  2. Small-world property – a network with relatively few long distance connections but has a small average path-length relative to the total number of nodes
    1. A network with 1,000 nodes, slightly rewired with random links brings down the average path length from 250 to 20…
    2. Evolved because information needs to travel quickly within the system and creating and maintaining reliable long-distance connections is very energy expensive. Nature has selected for it – robust, resilient, effective, efficient, energy-cheap…
    3. Web is scale-free, small world network (fractal) – relatively small number of very high-degree hubs (Google), nodes with degrees over a very large range of different values (heterogeneity of degree value), self-similarity
    4. Scale-free network = power-law degree distribution
4. What seems to generate the complexity of humans as compared to plants is not how many genes we have but how those genes are organized into networks
5. Focus on the hubs as that is where the power, influence, network, etc. falls to and relies on (Google, Facebook, GrubHub, LinkedIn, Zillow, Amazon, etc.) Winner take all systems!!
6. Dangers of networks is that a small problem can quickly balloon into a major one if it is allowed to reach its tipping point
Scaling – how one property of a system will change if a related property changes. The scaling mystery in biology concerns the question of how the average energy used by an organism while resting – the basal metabolic rate – scales with the organism’s body mass
1. Metabolic rate proportional to body mass ^3/4
  1. Larger animals are more efficient than smaller ones and this leads to heart having to work less hard and the larger animal, on average, to live longer
2. Circulatory system is fractal
3. Metabolism is universal to all life so this touches every aspect of biology
Evo-Devo (evolutionary development) – genetic switches main cause for large differences between species with very similar DNA. “Junk DNA” and allows for punctuated equilibrium in evolution
Life exists at the edge of chaos
Natural selection is in principle not necessary to create a complex creature. Once a network becomes sufficiently complex, that is, it has a large number of nodes controlling other nodes, complex and self-organized behavior will emerge
Life has an innate tendency to become more complex which is independent of any tendency of natural selection
Good, short overview of networks

What I got out of it

Awesome book on chaos and complexity, how it arises, what its real-world implications are, how they might shape our world moving forward, the importance of networks and hubs, scaling, parallel traced scan, some idea models