Tag Archives: John Holland

Worlds Hidden in Plain Sight: Thirty Years of Complexity Thinking at the Santa Fe Institute by David Krakauer


  1. Over the last three decades, the Santa Fe Institute and its network of researchers have been pursuing a revolution in science. This volume collects essays from the past thirty years of research, in which contributors explain in clear and accessible language many of the deepest challenges and insights of complexity science.

Key Takeaways

  1. Things can be hidden in space, and they can be hidden in time…But the way in which complex phenomena are hidden, beyond masking space and time, is through non-linearity, randomness, collective dynamics, hierarchy, and emergence – a deck of attributes that have proved ill suited to our intuitive and augmented abilities to grasp and to comprehend.
  2. Linearity should not be an issue. Economic systems are obviously nonlinear, as are many, if not most, systems of current interest in physics. A more controversial question concerns the direction of feedback. Whereas a strictly linear system can have only negative feedback if divergence is to be avoided, positive feedback can occur in nonlinear systems of a saturation mechanism operates. Such systems tend to have multiple equilibria or resting points and great sensitivity to initial conditions. Traditionalists find it hard to relinquish uniqueness and global stability, but physicists are easily convinced and find positive feedback natural.
  3. In 1966, Robert Paine introduced the concept of “keystone species,” top predators such as starfish and sea otters, whose removal can lead to cascading effects in system properties. Since then, the concept has been extended to species other than top predators. Some, for instance, consider the distemper virus that kills lions in Africa to be a keystone species. Levin cites “a quarter century of research on keystone species – predators, competitors, mutualists, pathogens, among others – demonstrates a diversity of situations in which individual species play critical roles, at least in determining community structure.
  4. The authors wish to thank our co-organizer, Jennifer Dunne, for reminding us that the laws of life are hierarchical and must look upward to ecology as well as downward to physics and chemistry.
  5. Ludwig Boltzmann, in about 1884, coined the term ergodic for situations with identical time averages and ensemble averages. Not every situation is like this, however; there exist “nonergodic” situations as well, and these are often as counterintuitive as the ergodic situations seem trivial. So, do we have to be more careful when we talk about expected returns and average performances? There are two averages, not one – two ways of characterizing an investment, two quantities with different meanings…Herein lies the danger: if we don’t actually play many identical games at once, then such an average only has practical relevance if it is identical to the quantity we’re interested in, often the time average. There may be many possible paths from here into the future, but only one will be realized. In our game, you are risking your entire wealth, which obviously cannot be done many times simultaneously, so the ensemble average is not really the relevant quantity. Technically, it stems from a thought experiment involving other universes
  6. What is good for groups is not always good for the individuals comprising them. For example, both multicellular organisms and social insect colonies are functionally specialized and hierarchically organized collectives that are highly successful in maintaining and transmitting accumulated knowledge, in the form of genetic instructions, to the next generation; but they also have little regard for the fates of most cells or insects. This same pattern is apparent, in an attenuated way, in human societies. For example, economist George Steckel and anthropologist Jerome Rose (2002) examined health indicators for Prehispanic New World societies and found that the median health of individuals declined as societies grew more complex. This suggests social complexity emerges from mechanisms that promote coordinated behavior even if it is not in the best interest of each individual. In the case of multi-celled organisms and insect colonies, the solution was to make the coordinating individuals (cells, insects) genetics clones or siblings. That way, genes that promote cooperation could spread even if the most cooperative individuals left no offspring.
  7. Instead of assuming agents were perfectly rational, we allowed there were limits to how smart they were. Instead of assuming the economy displayed diminishing returns (negative feedback), we allowed that it might contain increasing returns (positive feedback). Instead of assuming the economy was a mechanistic system operating at equilibrium, we saw it as an ecology – of actions, strategies, and beliefs competing for survival – perpetually changing as new behaviors were discovered.
  8. Thermodynamics is the study of the macroscopic behavior of systems exchanging work and heat with connected systems or their environment. The four laws of thermodynamics all operate on average quantities defined at equilibrium – temperature, pressure, entropy, volume, and energy. These macroscopic variables exist in fundamental relationships with each other, as expressed, for example, in the ideal gas law. Thermodynamics is an extremely powerful framework as it provides experimentalists with explicit, principle recommendations about what variables should be measured and how they are expected to change relative to each other, but it is not a dynamical theory and offers no explanations for the mechanistic origins of the macroscopic variables it privileges.
  9. This introduces two important concepts: first, the idea of scaling, which refers to how measurable properties of a system change with its size; second, the concept of economies of scale. The latter means that, as cities grow, they need less of something per person: roads, sewers, or gas stations, for example
  10. The study of complex systems, like all of science, is a search for order. Traditionally, science seeks order by understanding the simplest parts of a system. How does a single gas particle behave given a certain temperature? Which gene in our DNA determines eye color? Scientists then try to develop theories that explain more general observations based on their detailed understanding of the individual parts.
  11. We know from the application of the scientific method – that is, from observation, then explanation, then prediction, and finally verification – that gravity causes the apple to move toward the ground at a specific and constant rate of acceleration

What I got out of it

  1. A series of articles on complexity that helps give a broad overview of the field and how far it has come in the last several decades. The physical book also has some fun and interesting ways to help categorize and organize the chapters and knowledge 

Hidden Order: How Adaptation Builds Complexity by John Holland

  1. Holland walks us through how coherence emerges from unstructured agents in environments of continuous flux; coherence under change and complex adaptive systems (CAS)
Key Takeaways
  1. Behavior depends much more upon interactions of agents than their actions
  2. Catalog of all activities does not equal understanding the effect of changes in the ecosystem
  3. Many complex systems show coherence in face of change through extensive interactions, aggregation of diverse elements and learning/adaptation
    1. Must understand the interactions and dynamics of the system before can hope to make any significant, lasting changes
  4. Theory can help detect lever points where small changes lead to big outcomes – Amplifier Effect
    1. Cross-disciplinary comparisons are vital as subtle characteristics in one context can be easily drawn out in others
  5. CAS systems made up of a large number of active elements diverse in form and capability
    1. Makes system stronger and more robust. Weeding out weak actors so that only the strong remain counter-intuitively leads to worse performance
  6. Rules are used to describe agent’s strategies – few, simple rules can lead to complex behavior
    1. A major part of the modeling effort for any CAS, then, goes into selecting and representing stimuli and responses, because the behaviors and strategies of the component agents are determined thereby. Once we specify the range of possible stimuli and the set of allowed responses for a given agent, we have determined the kinds of rules that agent can have
  7. Adaptation – process by which an organism best fits itself to its environment
    1. Time scale of adaptation varies drastically and they are very important to take into account in any system (humans vs. trees)
      1. The fast dynamics will shape the slow
    2. Overall, we will view CAS as systems composed of interacting agents described in terms of rules. These agents adapt by changing their rules as experience accumulates. In CAS, a major part of the environment of any given adaptive agent consists of other adaptive agents, so that a portion of any agent’s efforts at adaptation is spent adapting to other adaptive agents, co-evolution (Red Queen). This one feature is a major source of the complex temporal patterns that CAS generate
  8. The 7 Basics
    1. Aggregation
      1. Simplifies complex systems by grouping similar things which leads to constructing models as these are prime building blocks
      2. Emergence of complex, large-scale behavior from aggregate of small, simpler behaviors (ants and “intelligent” ant colony)
    2. Tagging
      1. Facilitates the formation of aggregates as tags manipulate symmetries (flag as a rallying cry which helps group people together)
      2. Tags enable us to observe and act on properties previously unobservable due to symmetries (spinning white cue ball harder to spot but when a stripe is added you can easily tell in which direction it is rotating)
      3. Facilitates selective interaction – filtering, specialization, cooperation leads to emergence of meta-agents and organizations through individual agents are always changing
      4. Tags are the mechanism behind hierarchies
    3. Non-linearity
      1. whole is greater than the sum of the parts
      2. Behavior in aggregate more complex than the parts would indicate
    4. Flows
      1. Nodes (processors, agents), connectors (designate possible interactions), Resource
      2. Adapt as time elapses and experience accumulates
      3. Tags almost always define the network by delimiting the critical interactions, the major connections. Tags acquire acquire this role because the adaptive processes that modify CAS select for tags that mediate useful interactions and against tags that cause malfunctions
      4. Multiplier Effect – resource injected in one node spreads over network which leads to chain of changes (big in network/flows modeling)
      5. Recycling Effect – the effect of cycles in the network can drastically increase output of the system over time as the system retains resources and these resources can be further exploited as they offer new niches to be exploited by new kinds of agents. This process leads to increasing diversity through increasing recycling (virtuous cycle)
    5. Diversity
      1. Each agent fills a niche which is determined based on interactions centering on that agent
      2. Nature abhors vacuums and will fill empty niche with new agent – typically similar in form and habit (the convergence effect, mimicry)
      3. CAS systems get diverse via adaptation which leads to further interactions and new niches – symbiosis, parasitism, mimicry, biological arms races
      4. Perpetual novelty is a hallmark of CAS
    6. Internal models
      1. Mechanisms CAS used to anticipate – eliminate details so that selected patterns are emphasized. Agent must select patterns in the torrent of input it receives and then must convert those patterns into change sin its internal structure
        1. A model allows us to infer something about the thing modeled
      2. Tacit and overt models
        1. Tacit simply prescribes a current action, under an implicit prediction of some desired future state
        2. Overt model is used as a basis for explicit, but internal, explorations of alternations, a process often called lookahead 
      3. Natural selection selects for better internal models
    7. Building blocks
      1. Deconstruct complex problem into simpler parts which can be used and reused in different circumstances
      2. The search for powerful building blocks is the most effective way to make the best internal models
        1. We can a significant advantage when we can reduce the building blocks at one level to interactions and combinations of building blocks at a lower level: the laws at the higher level derive from the laws at the lower level building blocks. This does not mean that the higher level laws are easy to discover but it does add a tremendous interlocking strength to understanding systems and hierarchies
  9. CAS exhibit coherence under change via conditional action and anticipation and do so with no central controller.
  10. Can discover lever points if can uncover general principles which govern CAS dynamics
  11. Agents must act somewhat similarly if a uniform approach to CAS is feasible
  12. Adaption – a rule’s ability to win based on its usefulness int he past – older rules are likely best as they’ve been tested by time
    1. Credit Assignment to best rules easiest when have immediate feedback – tests the rule’s utility
      1. Bucket Brigade – the credit assignment procedure which strengthens rules that belong to chains of action terminating in rewards
    2. Agent should prefer rules which use more information about a situation
      1. Higher specificity leads to stronger rules (higher in the hierarchy)
    3. Default hierarchy – early on, agents will depend on overly general default rules that serve better than random actions. As experience accumulates, these internal models will be modified by adding competing, more specific exception rules. These will interact symbiotically with the default rules and the resulting model is called a default hierarchy. Default hierarchies expand over time from general default to specific exceptions (the young man knows the rules, the old man the exceptions)
    4. Adaptation by rule discovery – trial and error may work but doesn’t leverage system experience
      1. Plausibility – take strong rules and apply to new areas which seem promising
        1. Innovation / creativity – simply combining tested building blocks in new ways
    5. Recombination of rules leads to discovery and occasionally mutation which can produce a more fit offspring
      1. More fit building blocks are used more frequently which are then passed on more often to succeeding generations
      2. More complicated building blocks usually formed by combining simpler blocks
        1. The higher level are typically composed of well-tested, above-average simpler blocks. Over time, the hierarchy becomes more elaborate, providing for the persistence of still more complex behavior. A
      3. Reproduction, recombination and replacement (genetic algorithm) found in nearly every CAS system
      4. Implicit parallelism – individuals (no matter how great) don’t recur but their building blocks do
        1. Evolution “remembers” combinations of building blocks which increase fitness
      5. Discovery of new building blocks leads to a slew of new innovations (punctuated equilibrium)
  13. With any model, must know what has been emphasized (exaggerated) to make a point and what has been left out to keep focused on that point
  14. Hierarchy – the appearance of new levels of an organization in this evolution depends on one critical ability: each new level must collect and protect resources in a way that outweighs the increased cost of a more complex structure. If the seeded aggregate collects resources rapidly enough to “pay” for the structured complexity, the seed will spread.
  15. Successful approach to any theory – interdisciplinary; computer-based thought experiments, a correspondence principle (models should encompass standard models from prior studies in relevant disciplines); a mathematics of competitive processes based on recombination
What I got out of it
  1. Fascinating book on how the universe seems to produce order for free via coherence, spontaneous self-organization and complex adaptive systems.