Brian is one of my closest friends, and I owe him huge for letting me live on his boat for a few months a few years back. The boat was based in the marina behind Willie Mays field in San Francisco, and my office was only a block away, at that time.
Since then, Brian has become a highly influential and well-known author and speaker, with works like The End of Business as Usual, Engage! and What’s the Future of Business (WTF). I am honored to have him consider me a friend and influence, as he is to me.
Brian Solis is principal at Altimeter Group, a research firm focused on disruptive technology. A digital analyst, sociologist, and futurist, Solis has studied and influenced the effects of emerging technology on business, marketing, and culture. Solis is also globally recognized as one of the most prominent thought leaders and published authors in new media. His new book, What’s the Future of Business (WTF), explores the landscape of connected consumerism and how business and customer relationships unfold and flourish in four distinct moments of truth.
Stowe Boyd: I’m glad we could get together and chat. I wonder what your take is on big data. There is so much buzz, and over the top metaphors. It’s the new oil, it’s the new internet! I know you have some reservations, so I’ll ask you, how do you see big data playing in the transformations we know are needed in the future of business?'No matter how smart we get with predictive algorithms it doesn’t matter, because without understanding social science, without aligning with a bigger mission or vision with what we are trying to do — something that is going to matter to people — we are just managing businesses the way we always have. We are not moving in any new direction.'
Brian Solis: I’ll start with a caveat. My day job is as a digital analyst. I study tech disruption, and I try and make sense of it for business. And what I hear quite a bit is that big data is big, it is the savior of the future of business, because it finally puts business in alignment with customer activity and customer expectations, so that businesses can move from the rigid format of today to a miraculous state of prediction, or being able to look into the crystal ball to make really great decisions. And I hear a lot these days that business professionals and executives are making decisions based on experience or gut, and with big date they will have the answers they need to do the right thing at the right time…
SB: Sounds like you really think the hype about peering into the future through big data, or based on past management practices, is a bit suspect. What kind of mistake is latent in that thinking?
BS: It’s a technology-first decision. People are reacting to technology. People are assuming that the data coming in is going have a tremendous amount of insight baked in. I think the future business lies at the intersection of data science, digital anthropology, sociology, ethnography, and psychology. Because without someone on the human side to track behavior we can’t necessarily map the journey. We can’t identify new touch points. We can’t really surface new expectations or opportunities without understanding human beings.
SB: Well, I completely concur. As Williams James said, ‘you judge a man’s intelligence by how well he agrees with you.’ By that measure, you are a wise man indued. I’m going to raise a second question, which is almost unrelated, but comes back to another debunking of the Santa Claus, Tooth Fairy perspective of big data that people seem to have. And that is the fundamental limits of certainty in a world that has grown so massively complex that certainty fails. There is a great deal of science that supports the theory that large complex systems can’t be peered into. They are too chaotic, and the mathematics suggests that even if we could understand them, we don’t know the initial state of the system, and so it’s unknowable. So that suggests there is a fundamental limit on what we can project, no matter how much data we collect.'We have access to incredible data, but we put it into the same processes, and use the same methodology and philosophy to determine what we do with these technologies. And then we expect different results.'
BS: I think part of this is the hope or the optimism that there has to be an answer to fix business, and I think people are looking to any kind of technology or business practice that they can. There’s that old saying about history repeating itself…
SB: ‘History doesn’t repeat itself, but it rhymes.’ Mark Twain.
BS: Exactly! Thank you. If you think about it, theres a big disconnect between where technology comes into the organization and where decision making takes place in the organization. We’ve learned that from past mistakes. The people making the decisions about what to do and where to head the company are disconnected from technology. A lot of decision-makers don’t even read their own email. And what ends up happening is then on one hand you have one person reporting to shareholders and stakeholders, making decisions based on spreadsheets and powerpoints. Then on the other, you have people who have real access to data or are in the real world. You almost have an Undercover Boss moment, where the decision makers need to go back in and think about what it’s like to be the employee or that customer. You have a little bit of empathy that comes back into the mix. Big data is that representation. It’s either coming through the marketing department or some silo because it’s not getting anywhere it needs to because, number one, there’s no one on the inside to translate those insights into actionable insights and tie it to tangible benefits to the organization. And the second thing is, no one on the organizational level is listening. That’s the problem. And no matter how smart we get with predictive algorithms it doesn’t matter, because without understanding social science, without aligning with a bigger mission or vision with what we are trying to do — something that is going to matter to people — we are just managing businesses the way we always have. We are not moving in any new direction.
SB: So, if it’s going to turn out that big data is going to something, but it won’t be a panacea, what should the most prescient of CEOs be considering? What should they be spending their time on? How are they going to get that extra productivity, or competitiveness, or intensity that they are reporting they are after. And they don’t think they can get that extra boost with the techniques used before, liking making people working long hours or automating business processes. Those seem to be tapped out.
BS: What’s that Rita Mae Brown quote? The definition of insanity is doing the same thing over and over but expecting different results. And you look at how we are making decisions about this new technology, and how we are looking at the other technologies that are producing these disruptions like social, mobile, and real time. We have access to incredible data, but we put it into the same processes, and use the same methodology and philosophy to determine what we do with these technologies. And then we expect different results. That’s just not going to happen. As an analyst, one of the things I have answer in my research is ‘what are the questions that are going to be answered based on the research?’ The problem I have is that in an era of uncertainty people aren’t asking the right questions. Those question might be like ‘What are the early warning signals I need to pay attention to that determines that my business is off the rails?’ or ‘How do I convince shareholders that we need to do things differently even though we are incredibly profitable right now?’ These are questions that leaders or visionaries ask. Not necessarily incredible managers or CEOs. And the problem is that someone is going to have to become that champion within the organization. And I do think big data is an opportunity, but it takes someone on the outside to say ‘This is so telling! Did you realize fourty-seven thousand people have had the same problem and we haven’t done anything about it?’ By the way the title of my new book is What Is The Future of Business, which stands for WTF for a reason.
SB: You sneak.
BS: I wanted to show I can talk the talk and walk the walk, so I spent a good couple of years as an aspiring digital anthropologist and ethnographer, and studied — using big data — the decision making cycles using different kind of customers and different types of industries, and I mapped it. And I showed how you can distill your business into four moments of truth. What you can do in those four moments of truth to grow your business to increase awareness for a much more distracted and connected customer than ever before. You take the data, you take the social science, you take just a bit of empathy, and you can do a lot of amazing things. And that to me is where we have to start thinking. And by the way, I’m doing what I’m telling everyone else to do. You have to fight the fight. Revolutionaries are revolutionaries for a reason. You have to speak the language of the C suite, if you will, and you have to show how this matters to them and how this is going to help them.'The human psyche can tolerate a great deal of prospective misery, but it cannot bear the thought that the future is beyond all power of anticipation'. - Robert Heilbroner
SB: You’ve made a great example of the sorts of things that I think are necessary, the things that underlie this series, Socialogy — the theory and practice of social business based on scientific principles — looking to outside disciplines, outside of the conventional skill sets of business people, looking to fields like cognitive science and network analysis. A great example is Paul Kedrosky’s Ladder Index. Paul learned that the state of California was publishing data on what sorts of debris was being recovered on the state’s roads, stuff that was falling off of trucks and cars. He tracked this data for a few months and one of the things he discovered was that ladders had an interesting trajectory, first rising and then starting to fall. He then started to plot that against housing prices, and it turned out to be a perfect predictor of housing prices. Because, after all, as housing starts fall, the number of carpenters driving to job sites with ladders on their pickups starts to fall, and so too will the number of ladders falling off those trucks. So it’s an example of social data, the behavior of people aggregated neatly. But it wasn’t very big, actually, only a few hundred or so ladders per month, and only a few months of data. And he applied the insight about who are the owners of these ladders falling onto the highway? Why construction workers, mostly. And he correlated that with housing data, and discovered the Ladder Index of Housing. Obviously, the state of California was not providing that analysis along with the raw data in the spreadsheet.
BS: I love that example, and you bring it home to everyone. When we look on the horizon about what big data can do and what businesses need, you realize that just as much as the data you need people like that, to make the inference, and to check it out. And that kind of sense is not common.
SB: Kedrosky is a world-class economist.
BS: Businesses are going to have to pay attention, because as soon as two or three of your competitors figure it out, they are going to have a tremendous competitive advantage over you. Why react to that? That’s when digital Darwinism starts to happen: when society and technology change faster than your ability to adapt. Get it in place now.
SB: Brian, next time we have to do this face to face, with champagne, so I can toast you to say thanks.
BS: Thanks for having me.
So the critical factor isn’t just amassing more data about people, but deepening our understanding about people, people connected in complex social systems, like markets, cities, and companies. I recall the quote by Robert Heilbroner,
The human psyche can tolerate a great deal of prospective misery, but it cannot bear the thought that the future is beyond all power of anticipation.
People want to believe we can see around the corner to the future, in order to make positive changes. But we can’t let our desires cloud our thinking, and give in to the dream of big data solving all of the big questions. I agree with Brian: we need a dollop of empathy to make sense of the human world, not just number crunching.
This post was written as part of the IBM for Midsize Business program, which provides midsize businesses with the tools, expertise and solutions they need to become engines of a smarter planet. I’ve been compensated to contribute to this program, but the opinions expressed in this post are my own and don’t necessarily represent IBM’s positions, strategies or opinions.
As it becomes harder and harder for Google to avoid the spam sites, search becomes a less helpful way to find answers. Paul Kedrosky says that curation is the answer, and always has been.
Paul Kedrosky, Curation is the New Search is the New Curation
Any algorithm can be gamed; it’s only a matter of time. The Google algorithm is now well and thoroughly gamed, as I first wrote about late last year, and as now become an entire genre of web writing, and that has grown to include my friend Vivek Wadhwa’s smart piece on TechCrunch not long ago. Google has, they argue, lost its mojo — which is true, but it’s more interesting and complicated than that.
What has happened is that Google’s ranking algorithm, like any trading algorithm, has lost its alpha. It no longer has lists to draw and, on its own, it no longer generates the same outperformance — in part because it is, for practical purposes, reverse-engineered, well-understood and operating in an adaptive content landscape. Search results in many categories are now honey pots embedded in ruined landscapes — traps for the unwary. It has turned search back into something like it was in the dying days of first-generation algorithmic search, like Excite and Altavista: results so polluted by spam that you often started looking at results only on the second or third page — the first page was a smoking hulk of algo-optimized awfulness.
There are two things that can happen now. (Okay, three. We could stop search, which won’t happen.). We could get better algorithms, which is happening to some degree, with search engines like Blekko and others. Or, we could head back to curation, which is what I see happening, and watch new algos emerge on top of that next-gen curation again. Think of Twitter as a new stab at curation, but there are plenty of other examples.
Yes, that sounds mad. If we couldn’t index 100,000 websites in 1996 by hand, how do we propose to do 234-million by hand today?
The answer, of course, is that we won’t — do them all by hand, that is. Instead, the re-rise of curation is partly about crowd curation — not one people, but lots of people, whether consciously (lists, etc.) or unconsciously (tweets, etc) — and partly about hand curation (JetSetter, etc.). We are going to increasingly see nichey services that sell curation as a primary feature, with the primary advantage of being mostly unsullied by content farms, SEO spam, and nonsensical Q&A sites intended to create low-rent versions of Borges’ Library of Babylon. The result will be a subset of curated sites that will re-seed a new generation of algorithmic search sites, and the cycle will continue, over and over.
In short, curation is the new search. It’s also the old search. And it’s happening again, and again.
I take a different view, which is that meaning is the new search:
10 Minute Sprint From 140 Characters Conference: Social Business
Abundance economics means that we won’t rely on search: search is based on scarcity.
Imagine that all critical information is available, publicly, and the most important breaking news is a few seconds (at most) away. In this world the problem won’t be finding what you want, but minimizing the torrent so that you have a small number of things to look at.
This is as true inside of a 1000 person company as in the open web.
Increasingly, we will switch to a social connection mode to filter and find for us. Our networks will become engines of meaning, as Bruce Sterling said.
Everything we want to find has been found, and will find us through our social connections. Like head colds and happiness.
We will find everything through social relationships: what washing machine to buy, or the best Thai restaurant in Beacon NY, or the company that makes the horizontal corduroys. people that care about these issues, and to who we matter, will share meaning with us: they have beliefs that they can justify, also called knowledge.
Google is only the echo of our linking behavior, a second-order derivative of our combined gestures. But generally, we would be happier with fewer results from trusted sources, and the rise of social tools makes that almost as fast as Google search.
Google must plan to adapt to the social revolution or fall into the spam darkness.
Steven Johnson, What A Hundred Million Calls To 311 Reveal About New York
New Yorkers are accustomed to strong odors, but several years ago a new aroma began wafting through the city’s streets, a smell that was more unnerving than the usual offenders (trash, sweat, urine) precisely because it was so delightful: the sweet, unmistakable scent of maple syrup. It was a fickle miasma, though, draping itself over Morningside Heights one afternoon, disappearing for weeks, reemerging in Chelsea for a few passing hours before vanishing again. Fearing a chemical warfare attack, perhaps from the Aunt Jemima wing of al Qaeda, hundreds of New Yorkers reported the smell to authorities. The New York Times first wrote about it in October 2005; local blogs covered each outbreak, augmented by firsthand reports in their comment threads.
The city quickly determined that the odor was harmless, but the mystery of its origin persisted for four years. During maple syrup events, as they came to be called, operators at the city’s popular NYC311 call center—set up to field complaints and provide information on school closings and the like—were instructed to reassure callers that they could go about their business as usual.
But then city officials had an idea. Those calls into the 311 line, they realized, weren’t simply queries from an edgy populace. They were clues.
On January 29, 2009, another maple syrup event commenced in northern Manhattan. The first reports triggered a new protocol that routed all complaints to the Office of Emergency Management and Department of Environmental Protection, which took precise location data from each syrup smeller. Within hours, inspectors were taking air quality samples in the affected regions. The reports were tagged by location and mapped against previous complaints. A working group gathered atmospheric data from past syrup events: temperature, humidity, wind direction, velocity.
Seen all together, the data formed a giant arrow aiming at a group of industrial plants in northeastern New Jersey. A quick bit of shoe-leather detective work led the authorities to a flavor compound manufacturer named Frutarom, which had been processing fenugreek seeds on January 29. Fenugreek is a versatile spice used in many cuisines around the world, but in American supermarkets, it’s most commonly found in the products on one shelf—the one where they sell cheap maple-syrup substitutes.
This piece reminds me of the fantastic presentation Paul Kedrosky gave at Defrag a few weeks back on his ‘Ladder Index’ — the frequency of ladders found on southern California’s highways — as a leading indicator of the housing market.
Big data is everywhere, and can be tapped in mysterious — and smelly — ways.
Mark your calendars for the upcoming (and rescheduled) mesh — Canada’s web 2.0 conference - Toronto May 15 & 16. mesh will bring together great keynotes and speakers, including Om Malik, Paul Kedrosky, Andrew Coyne, Michael Geist, Tara Hunt, Paul Wells, Steve Rubel, Jason Fried, Stowe Boyd (yes, me), Amber McArthur, Ren Bucholz, Andrew Baron, Chris Messina, David Crow (whew!) and many others. Organizers include Rob Hyndman, Matthew Ingram, Mike McDerment, Stuart MacDonald, and Mark Evans.
Looks like a great conference, and a great venue. Toronto is a fabulous city.
[Long aside: I truly love Canada: even before my sister moved there and became a ‘landed’ immigrant after living in Toronto 20-odd years, I had traveled much of the country. In the past few decades, I have been to the country literally a hundred times or so, and I am increasingly enamored of this very foreign country so close by. I also hope that if I continue to say nice things, I will be allowed to emigrate, which looks like a better and better idea considering America’s political situation and progressive global warming. Although Toronto may be one day be under water as the Great Lakes slowly turn into a giant inland ocean.
Karp carps about the terminology, as if using ‘subscribing’ instead of ‘syndicating’ would solve the real broken parts of the whole RSS mess. Paul does a better job enumerating real problems, which can be summarized as feed overload.
But the real problem is that the entire user experience offered up by RSS newsreaders is wrong. I wrote about this at some length last year in a post called RSS Readering: Why RSS Readers Are No Good For Me (And You Too, I Bet). In particular, I made the core point:
I tried them for a time, and then dropped out. These annoy me for similar reasons: I don’t like the Pez dispenser feel, where all posts are like another, and you assume the role of a pigeon in a Skinner box, hitting the button to make the pellets roll out.
I have been lusting for something, a new solution, that actually parallels my most rewarding reading experiences. The way this generally works is like so:
I stumble across some link, or reference — perhaps in an email, or in the midst of reading a post in a browser — and I decide that I would like to invest some attention to this concept, or meme. Note: I am not just deciding to click a link and go to a specific page — which is all typical browsers do. I am deciding to investigate the theme, thread, meme, or whatever, and assimilate and collate information about it.
I then use a variety of techniques to uncover what I am interested in:
- I might click on tags embedded in the post, that take me to Technorati, or I might simply decide to search at Technorati or Del.icio.us for references to the piece or for tags to the topic or the names of individuals writing about it.
- I might follow backlinks, from the post back to earlier sources: other posts, or articles.
- I might ask specific contacts of mine what they know about the object of my interest.
- I might write a post, summarizing what I have uncovered, and offering some thoughts on the subject
But what I seldom do is just sit there reading a stream of posts, based on their chronology, or other intrinsic factors. No, I am on a hunt, skipping from place to place, and these tools constrain me more than they free me.
So the problem is not RSS, which should be just a low-level protocol that tools rely on. The problem is the amazingly static and non-innovative way we are using RSS.
The basic metaphor of having all RSS streams converge into an app like NewsGator or Bloglines is too limiting.
I want RSS threaded into other social aspects of the web, like the Nerdvana concept I have been hawking for a long time: an integration of RSS feeds into the instant message buddylist, so that I can be notified when someone I am interested in has posted something recently, just like I can about their online presence, except in this case it is their onblog presence.
At any rate, Scott and Paul are attracting attention to a real problem, although the problem is the RSS reader model we have adopted.