Some years ago, I noticed something interesting.
When I asked innovators, company founders, and entrepreneurs about the breakthrough ideas that led to their killer brands, they didn’t tell me, as I might have expected, that their great ideas had emerged from a well-planned brainstorming session or as the result of years of hard work in the lab. Their breakthrough ideas came from seemingly insignificant behavioural observations they’d made while interacting with friends, family, colleagues, or strangers. These key observations often occurred when least expected, revealing an unmet and previously unrecognised consumer need.
It was a thought-provoking insight. After all, who would have thought that Snapchat, the social media app that allows the user to create photos with an ultra-short lifespan, was invented when the founder’s friend tried desperately to find a message containing a photo of himself smoking pot? Or who would have imagined that a priest dropping a Bible on the ground and spilling all his bookmarks would lead to Post-it Notes?
In our data-obsessed world, we’ve been convinced that billions of data observations drive innovation. If you peel the historic onion, you’ll discover that the key to innovation is often a coincidental observation.
Only a couple of years ago, you wouldn’t be able to attend a conference without hearing 'Big Data' mentioned over and over again, nor could you have attended a board meeting at which Big Data didn’t dominate the agenda. Everyone was intrigued by the notion that a black box of data could magically produce deep insight into humans’ deepest needs, thereby revealing billion-dollar innovation opportunities. Like a kid in a candy store, every CEO proclaimed: “I want one of those!”
What’s needed, so to speak, is a counterbalance to Big Data.
The missing piece in the puzzle, I’ve discovered, is tiny — and though it may be tiny, the potential impact of ‘Small Data’ is huge. I’m talking about first-hand observations made in consumers’ homes, in restaurants, in night clubs, in sports clubs, when driving or on the phone. These seemingly insignificant, seemingly irrelevant observations, once connected, have the potential to identify the vital causation that Big Data has, so far, lacked.
You could say where Big Data is all about seeking a correlation, Small Data is all about seeking the causation. Small Data is the key to turning Big Data into the success story everyone has been wanting.
I tend to say that true creativity happens when one combines two ordinary things in a new way. In many ways, this is the essence of both Small Data and Big Data.
There’s probably nothing as powerful as combining creativity with structured thinking. The most exciting thing is that we’ve just begun this amazing journey.
Next time you hear the term 'Big Data', think 'Small Data', too.
Odgers Berndtson global study of university technology research reveals dearth of UK specialists.