Capgemini Consulting: Big Data trends & challenges

Although the belief among executives is wide-spread that Big Data will disrupt their industry in the next three years, only 13% has Big Data insights fully integrated into their business operations, a recent survey by Capgemini Consulting shows. As much as one in four admits that insights have not been implemented yet. The most pressing challenge to successfully implementation is, according to the firm, a lack of data integration across the organisation.

Since 2011 the search term ‘Big Data’ has become a growing Google trend, flying high above ‘the internet of things’, of which its use can be seen as both beneficiary as disruptive to organisations’ operations. In a recently released survey of senior Big Data executives, titled “Cracking the Data Conundrum: How Successful Companies Make Big Data Operational”, Capgemini Consulting asked 226 respondents globally across multiple regions and business types about their organisation’s approach to Big Data governance, data management, skill development, and technology infrastructure.

From the survey, it is clear that business leaders are expectant, and some even apprehensive, about the effects the drivers will have on their business, with 60% believing “that Big Data will disrupt their industry within the next three years.” Given these expectations, business leaders are moving down avenues of exploiting the business relevant information through mining and other analytic techniques.

In 2013 $31 billion was expended on taking advantage of Big Data, by 2018 the expenditure is expected to rise to $114 billion. With the level of implementation within firms so far showing that 5% indicate they have neither begun implementation nor allocated a budget, 19% have budgeted but not yet implemented, 35% have partially integrated Big Data insights into business practice, while 13% indicate insights are fully integrated into their business operations.  Of the respondents that have already moved to capture and exploit Big Data, 27% indicate that their Big Data solution was successful, while 8% describe it as very successful. However, according to the survey, many businesses (38%) are struggling even with proofs of concept.

The authors found that a combination of infrastructure, structural and social conditions are holding back successful implementation. In terms of infrastructure, the biggest issue (46%) is scatted data across various teams, 31% indicate that legacy systems for data processing and their management forms are holding back full scale operations, while only 18% say that infrastructure costs and specific tools are holding them back. Structural issues are the lack of a clear business case for Big Data projects (39%), ineffective governance models (27%), and a lack of skills (25%). Social issues are primarily due to ineffective management across teams (35%), poor sponsorship from top management (27%).

Of the issues, the consultants note that one of the most pressing is the effective use of the available data between teams, which prevents decision makers having all the relevant conclusions drawn from data at their fingertips – with 79% of those surveyed lacking full cross organisation data integration. Other key issues are the absence of a clear case for funding, and poor management, with decentralised pockets of resources, decentralised teams and poor central planning. Particularly best practice sharing and implementation methods are missed and resources deployment is poorly timed.

One clear driver of success according to the research is an organisation structure with some form of internal implementation strategy. Ad hoc approaches, where analytics is done by isolated teams scattered throughout the organisation has the worst incidence of success at 20%. A centralised approach, with a team that develops a business wide strategy and implementation of Big Data analytics is successful in 43% of organisations. The most successful method is creating a separate business unit whose aims are to not only develop and implement but also to set to make a profit from their efforts.

Big Data best practices
The authors identified three best practices which produce results in the relevant business units. One such approach is a strategic approach to investment, with a systematic and structure approach generating the most favourable outcomes. However, the study found that 74% lacked defined criteria to identify, qualify and select Big Data use-cases, while 60% of companies did not have clearly defined KPIs to assess initiatives. Another key on the road to success is to have a skilled driver leading the way. At issue is that only 34% of companies have a Chief Data Officer, or an equivalent role. A final key practice for a successful implementation is having analytic talent.

Related news: Capgemini Consulting and MIT launch book Leading Digital.