Old guys rule — or at least that’s what the bumper stickers say along the beaches of Encinitas.
In the face of “little data”, experience counts. Old guys, step up to the mike.
But when blessed with “big data” (i.e., lots of it), analytics shine. Sorry, geezers, take a nap.
Below is a summary by Caron Carlson of FierceCIO about a fascinating, well-written opinion piece by Viktor Mayer-Schönberger and Kenneth Cukier in Wired.
Emphasis in red added by me.
Brian Wood, VP Marketing
———-
Big data can’t supplant instinct
Data alone can lead to efficiency, but probably not genius.
Data is the key to making decisions based on fact rather than feeling, but are we losing something by eliminating feeling from the process all together?
When human judgment is overruled by data, the way we value knowledge will be transformed and the subject-area expert will lose supremacy as the statistician gains status, write Viktor Mayer-Schonberger and Kenneth Cukier, authors of “Big Data: A Revolution That Will Transform How We Live, Work, and Think”.
Human judgment absent of concrete metrics often falls short of the mark, Mayer- Schonberger and Cukier write at Wired. Professional sports learned this when the Oakland As replaced a time-honored method of relying on instinct to value players to a new method relying on math.
When there isn’t enough information, experience and intuition (in the form of the expert) are critical. “In such a world, experience plays a critical role, since it is the long accumulation of latent knowledge–knowledge that one can’t transmit easily or learn from a book, or perhaps even be consciously aware of–that enables one to make smarter decisions,” they write.
But here’s the kicker: While big data is sure to help companies become faster, smarter and more efficient, it won’t necessarily make them dazzle. That will still depend on humans. “There is an essential role for people, with all our foibles, misperceptions and mistakes, since these traits walk hand in hand with human creativity, instinct, and genius,” the authors write. “The same messy mental processes that lead to our occasional humiliation or wrongheadedness also give rise to successes and stumbling upon our greatness.”
http://www.fiercecio.com/story/big-data-cant-supplant-instinct/2013-03-05
———-
Big Data
In the movie Moneyball, about how the Oakland A’s became a winning baseball team by applying analytics and new types of metrics to the game, there is a delightful scene in which grizzled old scouts are sitting around a table discussing players.
The audience can’t help cringing, not simply because the scene exposes the way decisions are made devoid of data, but because we’ve all been in situations where “certainty” was based on sentiment rather than science.
“He’s got a baseball body … a good face,” says one scout.
“He’s got a beautiful swing. When it connects, he drives it, it pops off the bat,” chimes in a frail, gray-haired fellow wearing a hearing aid. “A lot of pop off the bat,” another scout concurs.
A third man cuts the conversation short, declaring, “He’s got an ugly girlfriend.”
“What does that mean?” asks the scout leading the meeting.
“An ugly girlfriend means no confidence,” the naysayer explains matter-of-factly.
“OK,” says the leader, satisfied and ready to move on…
The scene perfectly depicts the shortcomings of human judgment. What passes for reasoned debate is really based on nothing concrete. Decisions about millions of dollars’ worth of player contracts are made on gut instinct, absent of objective measures.
Yes, it is just a film, but real life isn’t much different. Similar empty reasoning is employed from Manhattan boardrooms to the Oval Office to coffee shops and kitchen tables everywhere else.
Moneyball, based on the book by Michael Lewis, tells the true story of Billy Beane, the Oakland A’s general manager who threw out the century-old rulebook on how to value players in favor of a math- infused method that looks at the game from a new set of metrics…. Ultimately he led the long-suffering team to a first-place finish in the American League West in the 2002 season, including a 20-game winning streak. From then on, statisticians supplanted the scouts as the sport’s savants. And lots of other teams scrambled to adopt sabermetrics themselves.
In the same spirit, the biggest impact of big data will be that data-driven decisions are poised to augment or overrule human judgment.
The subject-area expert, the substantive specialist, will lose some of his or her luster compared with the statistician and data analyst, who are unfettered by the old ways of doing things and let the data speak. This new cadre will rely on correlations without prejudgments and prejudice. To be sure, subject-area experts won’t die out, but their supremacy will ebb. From now on, they must share the podium with the big-data geeks, just as princely causation must share the limelight with humble correlation.
This transforms the way we value knowledge, because we tend to think that people with deep specialization are worth more than generalists — that fortune favors depth.
Yet expertise is like exactitude: appropriate for a small-data world where one never has enough information, or the right information, and thus has to rely on intuition and experience to guide one’s way. In such a world, experience plays a critical role, since it is the long accumulation of latent knowledge — knowledge that one can’t transmit easily or learn from a book, or perhaps even be consciously aware of — that enables one to make smarter decisions.
But when you are stuffed silly with data, you can tap that instead, and to greater effect. Thus those who can analyze big data may see past the superstitions and conventional thinking not because they’re smarter, but because they have the data. (And being outsiders, they are impartial about squabbles within the field that may narrow an expert’s vision to whichever side of a squabble she’s on.) This suggests that what it takes for an employee to be valuable to a company changes. What you need to know changes, whom you need to know changes, and so does what you need to study to prepare for professional life.
Harnessing data is no guarantee of business success but shows what is possible.
The shift to data-driven decisions is profound. Most people base their decisions on a combination of facts and reflection, plus a heavy dose of guesswork. “A riot of subjective visions — feelings in the solar plexus,” in the poet W. H. Auden’s memorable words. Thomas Davenport, a business professor at Babson College in Massachusetts and the author of numerous books on analytics, calls it “the golden gut.” Executives are just sure of themselves from gut instinct, so they go with that. But this is starting to change as managerial decisions are made or at least confirmed by predictive modeling and big-data analysis.
As big data transforms our lives — optimizing, improving, making more efficient, and capturing benefits — what role is left for intuition, faith, uncertainty, and originality?
Brilliance doesn’t depend on data. Steve Jobs may have continually improved the Mac laptop over years on the basis of field reports, but he used his intuition, not data, to launch the iPod, iPhone, and iPad. He relied on his sixth sense. “It isn’t the consumers’ job to know what they want,” he famously said, when telling a reporter that Apple did no market research before releasing the iPad.
Big data is not an ice-cold world of algorithms and automatons. What is greatest about human beings is precisely what the algorithms and silicon chips don’t reveal, what they can’t reveal because it can’t be captured in data. It is not the “what is,” but the “what is not”: the empty space, the cracks in the sidewalk, the unspoken and the not-yet-thought. There is an essential role for people, with all our foibles, misperceptions and mistakes, since these traits walk hand in hand with human creativity, instinct, and genius.
The same messy mental processes that lead to our occasional humiliation or wrongheadedness also give rise to successes and stumbling upon our greatness. This suggests that, just as we’re learning to embrace messy data because it serves a larger purpose, we ought to welcome the inexactitude that is part of what it means to be human. After all, messiness is an essential property of both the world and our minds; in both cases, we only benefit by accepting it and applying it.
http://www.wired.com/opinion/2013/03/does-big-data-mean-the-demise-of-the-expert-and-intuition/
