Does Big Data just mean Big Hype? Not entirely, but amid such a fast changing landscape it's all too easy to lose direction. Data has always been around; the only difference today is the sheer volume.

Data capture and storage have grown exponentially. CEOs are under pressure to create brand new capabilities and models based on Big Data. They are understandably worried that if they don't get to grips with it they will lose out to their competitors. Indeed, the vast majority of corporations are failing to exploit Big Data effectively for competitive advantage.

In a 2013 White Paper, 'Getting Big Value from Big Data', the German software company SAP says: "The key is to focus on the opportunities and rewards of Big Data initiatives rather than getting stuck in endless discussions about technology. The technologies supporting this space are evolving so fast that investing in capabilities is more important than investing in individual pieces of hardware and software." As the opportunities to collect ever-increasing volumes of data explode, the biggest value of this data tidal wave – its commercialisation – can become an overwhelming task. Getting into the hearts and minds of customers via the collection of data, and spotting emergent trends within that data, is key.

According to Mike Young (pictured left), CIO with Aegis Media, "the trick is to understand what you are trying to achieve by collecting all this data. CEOs can feel a little overwhelmed by new media – their teenagers are on Facebook and come along and say 'Why doesn't your company have a Facebook page?'.

For the business it's not about having a Facebook page, but how does the data collected and interpreted from the new media help sell more products? It shouldn't be data-gathering for its own sake. Data is only any good if you can draw valuable insights from it. For the CEO the important thing is to focus on the actual challenges of the business; Big Data is just another tool in the box.

This voluminous data growth has been fuelled by the astonishing rise of the internet, from just 50 million users worldwide in 1998 to 2.7 billion (about 40% of the global population) today. Global e-commerce sales have been growing by some 20 per cent annually and in 2013 surpassed $1 trillion for the first time.

Each single transaction can yield an enormous amount of information about the purchaser. Storing the monumental quantities of data gathered from e-commerce is the least of the problems: making sense of it and putting it to commercial use is giving rise to the new God of the marketplace – the statistical predictive analyst.

Daniel Singer, of Mavens, the research-led digital strategy agency, plays the same tune. "There's a temptation to be in Twitter, to engage with these things, but the first question ought to be 'What am I trying to achieve?' Then the collecting of and dealing with data will have a purpose, make sense." For corporations, data collection, whether from electronic point-of-sale, store loyalty cards, or surveys, is all about gleaning insights into consumer preferences. Once collected, the more important step is spotting trends and patterns, segmenting those ever-more finely, and then re-targeting brand advertising or marketing.

Big Data is not just about collecting greater volumes of consumer information; it's also about making that information more transparent and usable at much higher frequency. Big Data allows ever-narrower segmentation of customers and therefore much more precisely tailored products or services. This sounds scarily intrusive but predictive statistical analysis has a potentially huge benefit: the elimination of the tendency towards unconscious human bias in favour of age, appearance, ethnicity and gender. If competence is what really matters, in any field, then human bias is just a distraction.

But it's in marketing and advertising that Big Data is making the biggest inroads right now. Alan Mumby, Partner and Head of Technology, Entertainment and Communications at Odgers Berndtson London says: "Now we not only understand this data, but know how to use it in the universe of work; everything from retail to finance. This really is marketing Nirvana. I see Big Data as good for customers. No longer will they be subjected to scattershot advertising. It completely overturns the old aphorism of Lord Leverhulme about half his advertising money being wasted, but the trouble was he didn't know which half." Leverhulme might soon be history.

For Young, Aegis is sitting on the edge of an untapped gold mine, thanks to its data collection. Storage, retrieval, updating, visualising – "the churning and crunching of the vast amount of data we have and continually gather is all about building insights for our customers. Aegis provides all media support for General Motors worldwide, a $3 billion account. The ambition of GM – as for marketers in most organisations – is to know as much as they can about the person they want to sell to. Our data allows a more accurate construction of that picture. For me – for GM – perhaps the most critical part of all this data is visualisation; mapping it, so it can be grasped through immediately understandable graphics."

Viktor Mayer-Schonberger and Kenneth Cukier, in their 2013 book Big Data: A Revolution That Will Transform How We Live, Work, and Think, compared the growth of Big Data to a kind of social change ushered in by the Enlightenment's Encyclopédie. If anything, that's probably too conservative. Potentially, Big Data – even though it's a misnomer – will create the kind of profound revolution not seen since the 19th century's industrialisation.

What is big data?

Big Data is a constantly moving beast. What was considered big a few years ago is today a mere speck. The US IT research firm Gartner developed a useful definition of Big Data, often referred to as the '3Vs' – increasing Volume (the amount of data), Velocity (the speed of data in and out), and Variety (range of data types and sources).

Some organisations add a fourth V – Veracity. Over 20 years – between 1986 and 2007 – the world's technological capacity to store information grew almost 100-fold, from 2.6 exabytes (one exabyte being equivalent to one quintillion bytes) to 295 exabytes. In layman's terms, this growth in data collection capacity was equivalent to less than one CD (650 MB) per person in 1986 to almost 61 CDs in 2007.

Walmart handles more than a million transactions every hour; the consumer information it consequently collects currently amounts to almost 200 times the information contained in all the books of the US Library of Congress. Globally, the technological capacity to store data has roughly doubled every 40 months since the 1980s.

The big question facing all corporations is – how to make the best use of this tsunami of information?

Illustration: Ikon Images

Gary Mead

Gary Mead is a business journalist and former commodities editor of the Financial Times



Leadership services: answering the critical talent questions

Your Leadership Challenges Our Leadership Practice works with Chairs, CEOs, and CHROs to optimise...


Tech impact on City promises to trump Brexit

Following a recent briefing that she delivered for internal staff on the impact of Brexit, Head o...


The 20-year journey on gender: why equal pay really matters

“I came from a working-class background where my father was a carpenter and my mother ran a publi...