Geoffrey West’s Fever Dream

Geoffrey West takes a look at the unknowability of the innards of complex systems, and wonders if big data — and a new big theory to corral big data — could act like a flashlight, revealing the inner workings of financial markets, urban communities, and ecosystems under stress.

Geoffrey West, Big Data Needs a Big Theory to Go with It

Complexity comes into play when there are many parts that can interact in many different ways so that the whole takes on a life of its own: it adapts and evolves in response to changing conditions. It can be prone to sudden and seemingly unpredictable changes—a market crash is the classic example. One or more trends can reinforce other trends in a “positive feedback loop” until things swiftly spiral out of control and cross a tipping point, beyond which behavior changes radically.

What makes a “complex system” so vexing is that its collective characteristics cannot easily be predicted from underlying components: the whole is greater than, and often significantly different from, the sum of its parts. A city is much more than its buildings and people. Our bodies are more than the totality of our cells. This quality, called emergent behavior, is characteristic of economies, financial markets, urban communities, companies, organisms, the Internet, galaxies and the health care system.

The digital revolution is driving much of the increasing complexity and pace of life we are now seeing, but this technology also presents an opportunity. The ubiquity of cell phones and electronic transactions, the increasing use of personal medical probes, and the concept of the electronically wired “smart city” are already providing us with enormous amounts of data. With new computational tools and techniques to digest vast, interrelated databases, researchers and practitioners in science, technology, business and government have begun to bring large-scale simulations and models to bear on questions formerly out of reach of quantitative analysis, such as how cooperation emerges in society, what conditions promote innovation, and how conflicts spread and grow.

The trouble is, we don’t have a unified, conceptual framework for addressing questions of complexity. We don’t know what kind of data we need, nor how much, or what critical questions we should be asking. “Big data” without a “big theory” to go with it loses much of its potency and usefulness, potentially generating new unintended consequences.

When the industrial age focused society’s attention on energy in its many manifestations—steam, chemical, mechanical, and so on—the universal laws of thermodynamics came as a response. We now need to ask if our age can produce universal laws of complexity that integrate energy with information. What are the underlying principles that transcend the extraordinary diversity and historical contingency and interconnectivity of financial markets, populations, ecosystems, war and conflict, pandemics and cancer? An overarching predictive, mathematical framework for complex systems would, in principle, incorporate the dynamics and organization of any complex system in a quantitative, computable framework.

'The emergent is everywhere and nowhere.'

Complexity isn’t a clock. You can’t open one up and see it’s innards. There are no gears and cogs. If there was a way to ‘look inside’, all you’d find would be more complex systems. And those complex systems aren’t connected in purely physical way, made up of computable inputs and outputs: they are united by emergent behaviors: the system manifests it character my acting in ways that are inherently unpredictable, and incalculable. These behaviors arise from the interactions between the components, but reside in none of them. The emergent is everywhere and nowhere.

We have no math for this.

West might as well be saying ‘We need to be able to see into the future . It would be helpful.’ But that doesn’t mean we have a way to do it, or that it is doable at all.

The tempo of modern life has sped up to the point that the future feels closer, and since it’s only a heartbeat away it seems reasonable to imagine being able to glance around that corner and know what is about to transpire. But that’s just a feeling.

'The more we have wired everything into everything else, the less we can know about what will happen tomorrow.'

The future is actually farther away than ever, because we have constructed a world that is the most multi-facted astrolobe, the most incestuous interconnection of global economic interdependencies, the deepest ingraining of contingent political scenarios, and the widest pending cascade of possible ecological side-effects. The more we have wired everything into everything else, the less we can know about what will happen tomorrow.

In essence, West hopes we can create a math that can pile up all the big data and crunch it, in a Borgesian infinity. A machinery as complex as the world it hopes to fathom, allowing us — or at least it— to know everything about everything.

I suspect we will have to settle for something less.

We could start by intentionally decoupling complexity that poses threats. Derivative trading, and credit default swaps are a good example. Efforts by banks and brokerages to diffuse risks, and sharing them with other finance companies leads to increased risk, systemically. When there is a big downturn the risks are amplified, and the cascade leads to huge ‘unintended’ results. The solution to this is not predicting when and how it will happen, but stopping the increased complexity inherent in derivatives and credit default swaps. The only cure for increased complexity is decoupling components of the larger system. 

A company begins as a start-up. It creates tremendous buzz and goes through a period where anything goes. There is no concern for paying bills as the company explores new rules. Below a level of fifty employees, there seems to be a lot of random behavior. Between fifty and one hundred employees, if the company has survived, this is when the sigmoidal behavior begins. At that stage the company needs bureaucracy, human resources, compliance, and so on. The company more and more becomes the bureaucracy. The innovative phase gets phased out, unlike a city. A city tolerates all sorts of crazy people walking around. No corporation will tolerate that. Companies become very intolerant to new ideas, rhetoric to the contrary. When a company starts cutting down the bloat, it no longer can be cool. The last time I was at Google I already could feel the tentacles of the bureaucracy encroaching—and Google’s awareness of the problem. There are signs of mortality creeping in. It may well be that Apple recognizes this problem and is fighting it like crazy by being open to new ideas. The question is: Is that possible?


Santa Fe Institute physicist and organizational theorist Geoffrey West, quoted in “Inside Apple” (via buzz)

Yancey first introduced me to Geoffrey West and his way of thinking (cities never die, corporations mimic the life/death of humans.)

For a bit more on the topic you can watch Geoffrey West’s TED Talk on the topic.

(via msg)

Research by Stephen Shennan at University College London, Robert Boyd at UCLA, and others indicates that shifting demographics was an important cause of early leaps in human development. Shennan’s research—which notes that artistic and technological leaps similar to the one in Europe had occurred in Africa and the Middle East tens of thousands of years earlier—suggests that what all these leaps had in common was the growth of local population density beyond a certain threshold. (Many of these cultural blooms withered, Shennan observes, when populations subsequently shrank.) Boyd’s research shows the close relationship between toolmaking advances and population size. As people gathered into larger groups and came into contact with one another more frequently, knowledge was shared, retained, and advanced more easily. From the earliest periods of modern human history, cultural development and technological development have been closely linked to rising population density.

They still are today. One simple indication of the economic advantages held by large, dense cities is their explosive growth over the past century and more—growth that is still continuing rapidly. America’s largest cities, each of which held no more than a few hundred thousand people in the mid-19th century, surpassed 1 million by the century’s end. By the middle of the 20th century, New York City had surged past 10 million residents; today, Greater New York contains more than 20 million people.

More than half the world’s population now lives in cities and metro areas, a proportion that is projected to grow to more than 70 percent by mid-century. China’s economic development in recent decades has been propelled by the rise of its cities—130 of which are home to 1 million people or more, including Shanghai with a metro population of 19 million and Beijing with 17.5 million. The Tokyo metro area, home to more than 35 million people, produces almost $1.2 trillion in economic output each year, roughly as much as Australia.

Researchers [Geoffrey West and Luis Bettencourt] affiliated with the Santa Fe Institute have identified the mechanism that underpins city growth and development as an accelerated rate of “urban metabolism.” Unlike biological species, whose metabolism slows as they get bigger, successful cities exhibit faster metabolism as they grow—a phenomenon that the researchers dubbed “superlinear” scaling. “By almost any measure,” they wrote, “the larger a city’s population, the greater the innovation and wealth creation per person.” There seems to be no limit, as yet, to the relationship between greater density and faster growth.

For centuries, the specific geographic advantages of cities tended to obscure their underlying social role. When agriculture powered economic development, cities grew near fertile soils. In the industrial age, access to raw materials and ports became critical, along with the presence of enough physical labor to run large factories. But as those factors become less important, we can see more clearly what has arguably mattered the most all along.

Cities are our greatest invention, not because of the scale of their infrastructure or their placement along key trade routes, but because they enable human beings to combine and recombine their talents and ideas in new ways. With their breadth of skills, dense social networks, and physical spaces for interactions, great cities and metro areas push people together and increase the kinetic energy between them.

Richard Florida, Where the Skills Are

Florida looks at US cities and wonders if something big is coming as our cities grow, and as we are concentrating certain skills in different areas. And he cautions that cities can’t do what they are designed to do — efficient creation of ideas and their application — unless we take care of the physical, infrastructure side of urbanism. Since they are meant to be a place for people to interact, we have to make sure that they are social spaces.

But Florida never delves into the post-industrial city, where online interaction is just as critical and deep as off, where cooperation is easier, and chance insights are even more low-cost. More to follow in ‘Liquid City’, my book, in process.

Messiness At Scale

I stumbled onto a hilarious but unenlightening Twitter flame war instigated by Dave Winer — the Godfather of RSS — in response to MG Siegler’s 'RSS is dead' wisecrack.

At the risk of putting my fingers in the sausage machine, let me add a touch of nuance:

  • RSS has declined in use, as web heads shift their source of ‘things to read’ away from RSS readers — like Google Reader — to tools like Twitter and Flipboard.
  • The role of RSS in web infrastructure is being threatened by non-RSS based architectures, like Flipboard’s. That product ignores RSS and fetches through the URL to get directly at images, text, and other content.

Winer is ideologically opposed to closed, proprietary approaches like that of Twitter (or, by extension, of Flipboard):

Dave Winer, What I mean by “the open web”

Anyway, here’s what I meant by “open web.”

I meant not in a corporate blogging silo.

If I put stuff in Twitter, the only way to get it out is through a heavily regulated and always-changing API. It will change a lot in the coming months and years. It will certainly narrow more than it expands. I feel very confident in predicting this, because I understand where Twitter is going.

If you put stuff in Facebook, it’s even more silo’d than it is in Twitter.

However, if you put stuff in WordPress, even on wordpress.com, you have full fluidity. You are not silo’d. You can get data in and out using widely-supported APIs that are implemented by Drupal, Movable Type, TypePad, etc etc. At least there’s some compatibility. And in a pinch you could probably move your content to a static website and have it be useful.

If you write in static HTML and RSS, you’re very portable, there will be no lock-in at all.

So to the extent you’re locked in, that’s the extent you are not on the open web. The perfectly open web has zero lock-in. The silos are totally locked-in and therefore not on the open web.

Winer’s complaints are about control of our content: that we should be able to easily manage what we write. It’s a political argument. 

But his points fly in the face of innovation, where a Twitter or Quora or Facebook create very different — and not solitary — models of open social discourse, which need to be managed in ways that are different from old school blogging. It’s not every man for himself, anymore. Time is a shared resource on today’s web: our time is not our own, anymore. And that’s largely good.

I liken this problem to the trade offs inherent in living in large cities versus towns or the country. There’s more noise, bigger crowds, and longer lines at the DMV: more things that we can’t control, or where our control is restricted, relative to folks living in bucolic Des Moines.

Only in cities we get superlinear scaling, as Geoffrey West and his colleagues have shown:

Jonah Lehrer, A Physicist Turns the City Into an Equation

When a superlinear equation is graphed, it looks like the start of a roller coaster, climbing into the sky. The steep slope emerges from the positive feedback loop of urban life — a growing city makes everyone in that city more productive, which encourages more people to move to the city, and so on. According to West, these superlinear patterns demonstrate why cities are one of the single most important inventions in human history. They are the idea, he says, that enabled our economic potential and unleashed our ingenuity. “When we started living in cities, we did something that had never happened before in the history of life,” West says. “We broke away from the equations of biology, all of which are sublinear. Every other creature gets slower as it gets bigger. That’s why the elephant plods along. But in cities, the opposite happens. As cities get bigger, everything starts accelerating. There is no equivalent for this in nature. It would be like finding an elephant that’s proportionally faster than a mouse.

I maintain that Twitter, Facebook, and other ‘closed’ systems are really something else: they are dense and complex social systems, more like modern cities than Web 1.0 publishing platforms. And, like cities, there is more going on, less being controlled by specifications like RSS, and the food is better, the music is better, and there is more dangerous sex taking place.

Brian Eno uses the term ‘scenius’ to define the quality of the great cities, their ability to foster deep shared understanding and purpose for large networks of people. This collective intellect arises from messiness at scale, not carefully mediated and clearly defined standards. 

Said differently, the best food comes from cities with the highest number of health code violations, and the best art is produced where the largest number of building code infractions are found.

So, if you are looking for clean bathrooms and no traffic jams, stay in Iowa. But it is in cities — dense, loud, unplanned, messy — where the breakthroughs emerge.

Getting back to the specific case, here, let’s look at Flipboard. Flipboard rejects the use of neat-and-tidy RSS, and reaches through the URLs it finds in Twitter to directly paw the text, images, and links placed into articles and posts, and then it chooses what to display based on a proprietary algorithm inside the guts of the app, not based on the publisher’s RSS specification. 

Flipboard, Twitter, and other dense, complex social tools create a messier world, one that has superlinear scale. The tradeoff between complete ‘openness’ (or individual control of information and its experience) and superlinear social density is one I am willing to make. And so are all the users of these tools, or should I say, residents of these cities?

It’s when West switches the conversation from infrastructure to people that he brings up the work of Jane Jacobs, the urban activist and author of “The Death and Life of Great American Cities.” Jacobs was a fierce advocate for the preservation of small-scale neighborhoods, like Greenwich Village and the North End in Boston. The value of such urban areas, she said, is that they facilitate the free flow of information between city dwellers. To illustrate her point, Jacobs described her local stretch of Hudson Street in the Village. She compared the crowded sidewalk to a spontaneous “ballet,” filled with people from different walks of life. School kids on the stoops, gossiping homemakers, “business lunchers” on their way back to the office. While urban planners had long derided such neighborhoods for their inefficiencies — that’s why Robert Moses, the “master builder” of New York, wanted to build an eight-lane elevated highway through SoHo and the Village — Jacobs insisted that these casual exchanges were essential. She saw the city not as a mass of buildings but rather as a vessel of empty spaces, in which people interacted with other people. The city wasn’t a skyline — it was a dance.

If West’s basic idea was familiar, however, the evidence he provided for it was anything but. The challenge for Bettencourt and West was finding a way to quantify urban interactions. As usual, they began with reams of statistics. The first data set they analyzed was on the economic productivity of American cities, and it quickly became clear that their working hypothesis — like elephants, cities become more efficient as they get bigger — was profoundly incomplete. According to the data, whenever a city doubles in size, every measure of economic activity, from construction spending to the amount of bank deposits, increases by approximately 15 percent per capita. It doesn’t matter how big the city is; the law remains the same. “This remarkable equation is why people move to the big city,” West says. “Because you can take the same person, and if you just move them to a city that’s twice as big, then all of a sudden they’ll do 15 percent more of everything that we can measure.” While Jacobs could only speculate on the value of our urban interactions, West insists that he has found a way to “scientifically confirm” her conjectures. “One of my favorite compliments is when people come up to me and say, ‘You have done what Jane Jacobs would have done, if only she could do mathematics,’ ” West says. “What the data clearly shows, and what she was clever enough to anticipate, is that when people come together, they become much more productive.”

West illustrates the same concept by describing the Santa Fe Institute, an interdisciplinary research organization, where he and Bettencourt work. The institute itself is a sprawl of common areas, old couches and tiny offices; the coffee room is always the most crowded place. “S.F.I. is all about the chance encounters,” West says. “There are few planned meetings, just lots of unplanned conversations. It’s like a little city that way.” The previous evening, West and I ran into the novelist Cormac McCarthy at the institute, where McCarthy often works. The physicist and the novelist ended up talking about Antarctic icefish, the editing process and convergent evolution for 45 minutes.
Of course, these interpersonal collisions — the human friction of a crowded space — can also feel unpleasant. We don’t always want to talk with strangers on the subway or jostle with people on the sidewalk. West admits that all successful cities are a little uncomfortable. He describes the purpose of urban planning as finding a way to minimize our distress while maximizing our interactions. The residents of Hudson Street, after all, didn’t seem to mind mingling with one another on the sidewalk. As Jacobs pointed out, the layout of her Manhattan neighborhood — the short blocks, the mixed-use zoning, the density of brownstones — made it easier to cope with the strain of the metropolis. It’s fitting that it’s called the Village.

In recent decades, though, many of the fastest-growing cities in America, like Phoenix and Riverside, Calif., have given us a very different urban model. These places have traded away public spaces for affordable single-family homes, attracting working-class families who want their own white picket fences. West and Bettencourt point out, however, that cheap suburban comforts are associated with poor performance on a variety of urban metrics. Phoenix, for instance, has been characterized by below-average levels of income and innovation (as measured by the production of patents) for the last 40 years. “When you look at some of these fast-growing cities, they look like tumors on the landscape,” West says, with typical bombast. “They have these extreme levels of growth, but it’s not sustainable growth.” According to the physicists, the trade-off is inevitable. The same sidewalks that lead to “knowledge trading” also lead to cockroaches.

Consider the data: When Bettencourt and West analyzed the negative variables of urban life, like crime and disease, they discovered that the exact same mathematical equation applied. After a city doubles in size, it also experiences a 15 percent per capita increase in violent crimes, traffic and AIDS cases. (Of course, these trends are only true in general. Some cities can bend the equations with additional cops or strict pollution regulations.) “What this tells you is that you can’t get the economic growth without a parallel growth in the spread of things we don’t want,” Bettencourt says. “When you double the population, everything that’s related to the social network goes up by the same percentage.”

West and Bettencourt refer to this phenomenon as “superlinear scaling,” which is a fancy way of describing the increased output of people living in big cities. When a superlinear equation is graphed, it looks like the start of a roller coaster, climbing into the sky. The steep slope emerges from the positive feedback loop of urban life — a growing city makes everyone in that city more productive, which encourages more people to move to the city, and so on. According to West, these superlinear patterns demonstrate why cities are one of the single most important inventions in human history. They are the idea, he says, that enabled our economic potential and unleashed our ingenuity. “When we started living in cities, we did something that had never happened before in the history of life,” West says. “We broke away from the equations of biology, all of which are sublinear. Every other creature gets slower as it gets bigger. That’s why the elephant plods along. But in cities, the opposite happens. As cities get bigger, everything starts accelerating. There is no equivalent for this in nature. It would be like finding an elephant that’s proportionally faster than a mouse.”

- Jonah Lehrer, A Physicist Turns the City Into an Equation

(via underpaidgenius)

Related Posts Plugin for WordPress, Blogger...