Applying the Science of Emergence to Solve the Mysteries of Innovation (Scientific American)

Applying the Science of Emergence to Solve the Mysteries of Innovation (Scientific American)

Applying the Science of Emergence to Solve the Mysteries of Innovation (Scientific American)

Why do groups stop innovating well when they grow large? How can large teams or companies or research groups innovate faster and better?

Over the past two decades, scientists have been turning to the science of emergence, the study of surprising collective behaviors, to help us understand a broad range of systems: how birds flock, fish swim, brains work, diseases erupt, ecosystems collapse, and, more recently, how groups of people behave. Now we can use similar principles to help us design teams that are better suited for nurturing radical new ideas—the breakthroughs we desperately need to treat cancer, for example, or reverse climate change.

The recent resurgence of emergence traces back to a classic 1972 essay by the physicist and Nobel laureate Philip Anderson: “More Is Different.” Anderson railed against a widespread view that the only science that mattered was the science of the small—“fundamental” laws of nature—and argued the merits of understanding the science of the many: emergent behaviors.

Why do groups stop innovating well when they grow large? How can large teams or companies or research groups innovate faster and better?

Over the past two decades, scientists have been turning to the science of emergence, the study of surprising collective behaviors, to help us understand a broad range of systems: how birds flock, fish swim, brains work, diseases erupt, ecosystems collapse, and, more recently, how groups of people behave. Now we can use similar principles to help us design teams that are better suited for nurturing radical new ideas—the breakthroughs we desperately need to treat cancer, for example, or reverse climate change.

The recent resurgence of emergence traces back to a classic 1972 essay by the physicist and Nobel laureate Philip Anderson: “More Is Different.” Anderson railed against a widespread view that the only science that mattered was the science of the small—“fundamental” laws of nature—and argued the merits of understanding the science of the many: emergent behaviors.

A key feature we wish to understand in any complex system is the sudden change between two types of emergent behaviors: a phase transition. Water, for example, will suddenly change from liquid to solid. Metals will suddenly change from ordinary conductors to superconductors. Why?

Systems snap when the tide turns in a microscopic tug-of-war. Binding energy ties to lock water molecules into rigid formation. Entropy encourages those molecules to roam. As temperature decreases, binding forces get relatively stronger, and entropy forces get relatively weaker. When the strengths of those two forces cross, the system snaps. Water freezes.

All phase transitions are the result of two competing forces, like the tug-of-war between binding energy and entropy in water. And that’s where we begin with teams and companies: when people organize into any kind of group with a mission, and a reward system tied to that mission, they also create two competing forces—two forms of incentives. We can think of the two competing incentives, loosely, as stake in outcome and perks of rank.

As structure changes, one grows stronger and the other grows weaker. When groups are small, for example, everyone’s stake in the outcome of the group project is high. At a small biotech, if the drug works everyone will be a hero and a millionaire. If it fails, everyone will be looking for a job. The perks of rank—job titles or the increase in salary from being promoted—are small compared to those high stakes.

As teams and companies grow larger, the stakes in outcome decrease while the perks of rank increase. When the two cross, the system snaps. Incentives shift from encouraging a focus on projects and outcomes to encouraging a focus on politics and promotion. A simple—but not simplistic—model of incentives inside organizations allows us to calculate when this transition will occur.

What does this mean for innovation?

The most important breakthroughs—the ones that change the course of science, business or history—are fragile. They are rarely announced with blaring trumpets and a red carpet, dazzling everyone with their brilliance. Instead, they often arrive covered in warts—the failures and seemingly obvious reasons they could never work that make them so easy to dismiss. They pass through long dark tunnels of skepticism and uncertainty, their champions dismissed as crazy. For lack of any better term, I call them loonshots.

In the first phase of team organization mentioned above, when stake in outcome dominates, incentives favor uniting around these early stage-projects. Individuals have so much collectively at stake in the outcome of their mission, that they will come together to rescue those projects from their inevitable stumbles and wrong directions. We can call this the loonshot phase.

In the second phase, when perks of rank dominate, incentives favor a focus on careers and promotion. Early-stage projects covered in warts are rejected in favor of ideas that raise the fewest objections. Those are typically franchise projects: the next generation of an already-established product or program (the next statin drug, the next Avengers movie). We can call this the franchise phase.

The sudden change between these two emergent behaviors is a phase transition.

The bad news about these changes in organizations is that phase transitions are inevitable. All liquids freeze. The good news is that understanding the forces that cause a transition allows us to manage it. Water freezes at 32 degrees Fahrenheit. On snowy days, we toss salt on our sidewalks to lower that freezing temperature. We want the snow to melt rather than harden into ice.

We use the same principle to engineer better materials. Adding a small amount of carbon to iron creates a much stronger material: steel. Adding nickel and tungsten to steel creates some of the strongest alloys we know: the steels used inside jet engines and nuclear reactors.

Understanding the analogous control parameters that govern the transition inside teams and companies helps us engineer more innovative organizations.

None of these insights would be possible by analyzing the behavior of individuals on their own, just as we could never understand why liquids flow or solids shatter by analyzing individual molecules on their own.

Innovating well is an emergent behavior. More is different.

This was originally published on Scientific American.

Share this article

Order Your Copy Get a FREE chapter of Loonshots + monthly letter

A mix of science, business, and history, delivered to your inbox, once or twice a month. Stories that inspire and ideas you can use.

+