An introduction to using Big Data for Marketing

Big Data, Big Analytics, Big Automation, Big Noise?

When I first started my career, storage was not at today’s inexpensive levels and one of the big challenges was utilising the space available in efficient an ingenious ways. Over the years the price has plummeted with space becoming freely available and available data growing exponentially. In recent years the ability to use this data has become possible with development of Big Data removing the historic problems of storage and access in reasonable, practical timescales.

But what does this storage potential mean for marketers? Is it just more data? Many I speak to are wondering about the applications and this is also suggested by Dave Chaffey's review of Online marketing trends for 2014 where Big Data was voted as having the third biggest commercial potential for Smart Insights readers.

In this post and a short series, I'll be exploring how the increase of available, ever growing, ever changing data be utilised in the Marketing arena?

What is Big Data in a marketing context?

To begin with let’s look at what we mean by Big Data. As a starting point you may have seen this visual, technical definition of Big Data which is useful since it shows that the potential of using Big Data for marketing isn't just about increasing volumes of data.

Image original source from Diya Soubra in a Data Science blog - The 3Vs that define Big Data

The Data Velocity vector shows how marketers now have access to real-time data, such as real time analytics of interactions on web and mobile sites and also social media interactions.  The question is, how do we harness it.

The Data Variety vector shows how new types of unstructured data, including again, social media interactions offer potential too. This also suggests the potential of integrating different sources of data to gain more customer insight.

I will explore the applications of each of these in future posts.

Let's now look at an example which gives a sense of what's possible with Big Data. If we start by looking at a game between you and a few other people, after a few hands you will have picked up the rough style of play of each player and get a feel of how you should play them. Taking this a step further you may have kept a tally of the cards and have an idea of how the cards are stacked (assuming not shuffled) and which cards may be in sequence. This gives you a basic understanding of your environment, which depending on how well you can memorise the information helps you play and hopefully win the coming hands.

Looking at the diagram below you can see how this can progress from basic understanding to detailed informed understanding of not just one game, but multiple games across many people all interacting in and out of the game.

This shows how basic data for a regular game of cards can expand, which if we take into an online gaming environment would cause an explosion of information covering:

• Standard game information across multiple matches, games and hand.
• Multiple sources of information, both internal and external.
• Free text data of conversations within and out of the game (think in game messaging, twitter, facebook, etc).

This volume, growth and range of data has been used to define what Big Data is, with it simplified to the 3 V’s of Big Data:

• Volume – Amount of Data. The total volume of existing digital data standing at ~2.5 zettabytes in 2012, which represents ~400 gigabytes of information for each person on the planet.
• Velocity – Speed of growth/change. The accumulation of data is growing at astonishing rates increasing by 2.5 exabytes each day or 40% annually at current rates and speeding up, with a projected volume of 7.9 zettabytes by 2015.
• Variety – Types of Data. With the ever increasing digital world the type of information is provided in many and varied way from structured data with defined elements and values, to unstructured free format of social media including text, video & images; not forgetting the additional source of metadata, data about data.

So going back to the original question what is the value of Big Data, in the card game example the value is potential to win overall, by knowing the best moments to play aggressively, conservatively, how to direct conversation, offer drinks, etc. So what is the value to Marketing if we assume that volume, variety and fast changing data is a given, available and usable? In the title I put forward two possible value points:

• Big Analytics – So does Big Data mean better more insightful analytics? Well a good way to look at this is to think of a body of water, to understand if something is happening you could analyse all sorts of data, using fluid dynamics to predict and understand changes to movement of water or you could just look for a ripple. Data is like this and currently you have an understanding and can get to some of your data, but there is a vast amount of information not available or accessible. Big Data provides the ability to use this information and the key is to understand what is useful (identifying and descriptive data) and what can help drive improved activity, be that sales, traffic, margins, etc. (predictive data). So yes Big Data will provide more opportunity to understand your customer, with more granular information available and usable, driving greater appetite for modelling and analytics.
• Big Automation – The velocity of data and ability to access will enable you to understand your customer’s behaviour in the now and have enough information to make informed decisions to improve the customer experience. This is the nirvana and potential of Big Data, when it comes to automation, but will require a move from retrospective analysis to real time analysis. In reality to respond to the availability, depth and changeability of data will require automation or teams of 1,000s to manage/deliver actions from insights. So in a short answer yes Big Data will drive Big Automation.

So does Big Data provide a new paradigm or is it an extension of Data Driven marketing, with the accessibility and availability of information enabling a greater take up.

To look at this I will take each of the 3 V’’s of Big Data and look at their potential impact on Marketing in a following series of posts.