News flash: big data is not sufficient to achieve pure efficiency or solve all business problems.
Spoiler alert: big data is one tool among many in the business executive’s organizational arsenal.
Opinion: people with insight and experience and street-smarts and common sense matter too.
Summary by Tim McElligott in FierceBigData, original opinion piece by Paul Thorley (CEO of Capgemini) in CIO.
Emphasis in red added by me.
Brian Wood, VP Marketing
——–
When big data is not enough
The first question you have to ask yourself when reading a headline like this one from CIO–“Big data necessary, but not enough” is, who ever said it was?
The point is well taken that big data is not a solution unto itself. It does not exist in a vacuum. In business, big data is an analytics tool that provides non-obvious insights to executives to assist in their decision making. In healthcare, it is an adjunct to decades of medical research and a taskmaster for adhering to best practices.
In all instances, big data works with some other process or tool–usually already in place–in order to make sure the data being used is the most accurate and relative to the task at hand.
The CIO article, written by Paul Thorley, CEO of Capgemini, states rightly that the ability to take advantage of big data is what “makes a smart organization different from one that is not so smart.” It also explains how business intelligence, which traditionally was derived from data within an organization, has now expanded beyond the borders of the enterprise to leverage data from the outside, and that this is driving interest in big data. But he never explains what big data is lacking or what makes up for it not being enough.
His company’s own data, from a 2012 global survey on big data, found that the majority of respondents (54 percent) say that management decisions based purely on intuition or experience are increasingly regarded as suspect. By this view, the headline should have read: “Intuition and experience necessary but not enough, supplement decision making with big data.”
Capgemini’s research also showed that 65 percent of executives assert that more management decisions are now based on hard analytic information and that organizations are increasingly moving towards evidence-based decision making. If only that were true of society at large.
The article is worth reading for the survey data it shares, but it never deals with the headline, which arbitrarily leaves big data shortcomings flapping in the breeze. However, Thorley offers a sage bit of advice for implementing big data solutions. He said to do a short proof of value before getting too far along. “If the proof of value (PoV) is promising, the scope [of the project] can be further expanded iteratively. The key to this is to be able to keep the PoV succinct, employ support of external organizations like Systems Integrators, big data forums to exchange experiences, to obviate the need for upfront investment in the technology/ tool set and others.”
http://www.fiercebigdata.com/story/when-big-data-not-enough/2013-04-30
——–
Opinion: Big data necessary, but not enough
In today’s inter-connected world of transactions and interactions, data has become a key factor in the production process. The ability to take advantage of this factor is what makes a smart organisation different from one that is not so smart.
Traditionally, most of the business intelligence that is of interest to an organisation comes from data that originates and/or is managed within the organisation. Also, in the past, the sole focus had been on structured data despite the fact that most of the information is generated in unstructured form. With more data important to an organisation’s activities being generated – and not necessarily in structured form – outside an organisation, these organisations have started to take an interest in big data.
In addition, with the increased use of sensors (RFIDs, smart meters, etc), we generate a lot of data that can be used to gain competitive advantage, drive productivity and make operational improvements.
This phenomenon has also been enabled by technological advances such as in-memory analytics.
Capgemini’s 2012 global survey on big data, commissioned the Economic Intelligence Unit, covered 607 executives around the world (25 per cent in the Asia-Pacific region), and found that the majority of respondents (54 per cent) say that management decisions based purely on intuition or experience are increasingly regarded as suspect.
This view is held even more firmly in the manufacturing, energy and government sectors, and 65 per cent assert that more and more management decisions are based on ‘hard analytic’ information.
The research shows that organisations are increasingly moving towards evidence-based decision making, but at the same time, face significant challenges in managing and leveraging the ever-increasing volumes of data not only from a technology perspective but also as an organisation.
With regard to the most valuable big data sets, there has been significant consensus across industries: 69 per cent of the participants agree that business activity data (sales, PoS data, purchases, etc) were the most valuable, followed by office documentation (emails, document stores, etc).
However, health care and pharmaceuticals differ, citing social media as the second most valuable big data set, possibly because of sentiment analyses reflecting consumer views and trends as keys to health care products.
With unstructured data seen as the second most valuable big data set, more than 40 per cent of respondents said that unstructured content was too difficult to interpret and manage to be used in support of decision making.
Interestingly, the survey also showed that many of the organisations that described themselves as being data driven felt that they could have made better decisions if the right data had been available.
With regard to impediments to effective decision making using big data, organisational silos topped the list of 56 per cent of the respondents. This seems to apply more to large organisations (with more than $10 billion in revenues), the response rate being 72 per cent vis-a-vis smaller firms (with less than $500 million in revenues) at 43 per cent.
The research shows that organisations are also struggling with the enormous volumes of data as well as their poor quality, together with ‘silo’ed’ nature of dealing with data to drive decision making.
The other major impediments included a shortage of skilled people to analyse the data properly, the time taken to analyse, the difficulties in interpreting the unstructured content of big data, and the high cost of storing and manipulating large data sets.
Until a few months ago, when organisations (apart from a few pioneering enterprises/groups) in Australia talked about big data it was pretty much a matter of “give me a business case for big data and we can put it up for project funding”.
However, in the last few months, there has been an increased awareness of the big data conundrum and the discussion has now come to focus on “let us take up a use case (hypothesis-based, if feasible) and do a short proof of value”.
If the proof of value (PoV) is promising, the scope can be further expanded iteratively. The key to this is to be able to keep the PoV succinct, employ support of external organisations like Systems Integrators, big data forums to exchange experiences, to obviate the need for upfront investment in the technology/ tool set and others.
We have often seen technological advances raising productivity; more often this happens over a large time frame of many years. Organisations looking for productivity gains, potential benefits even in short and medium term are possible through big data initiatives.
The aspects of big data that are more predominantly covered relate to customer sentiment analysis available through social media. Of late, we are observing this extended beyond customer satisfaction to other facets viz. top line growth, profitability improvement and risk management. One notable exception to this is the energy utilities sector. In Victoria, driven by the regulatory requirements, installation of household smart meters provides a big data platform (where the expected meter readings could be once every 30 minutes or so).
The need here is the ability to leverage the investments made, in the form of smart meters, to achieve operational improvements for these enterprises.
Some of our clients also raised the question of possible replacement of their traditional Enterprise Data Warehouse(s), Data Mart environments with big data technology.
When you consider the sizeable investments that have been made and the maturity of these applications in supporting business needs, it is unlikely that this would happen.
However, in cases where no sizeable incumbent applications are in place, but where further investments are required, there is real potential to look at the big data solutions. We believe that the traditional warehouses will co-exist with the big data platforms for some time to come. When it comes to big data investments, the key point to remember is that the analytical insight it offers together with acting on that insight make up the value proposition.
Some of the popular metaphors adopted with regards to big data, such as ‘finding the nuggets of gold’, capture only part of the picture. Australian organisations will do well by not succumbing to the hype but by focusing on the complete picture that drives the business value.
http://www.cio.com.au/article/460348/opinion_big_data_necessary_enough/
