A pensions professional's primer on Big Data

14 October 2015

You’ve all heard the term ‘Big Data’ and are perhaps wondering what on earth it’s referring to and how it relates to you as a pensions professional. Big Data is having a growing impact on the pensions industry and this is only going to continue.

So just what is ‘Big Data’? ‘Big’ is a relative term and you could be forgiven for thinking Big Data is simply a technology for dealing with large amounts of data. But it’s actually more than that. Without going into any technical detail, perhaps one way to think of Big Data is this:

Big Data is data that is so big that that the size of the data causes problems for the people trying to process and analyse it.

The problems are solvable but require a bit of technical jiggery pokery to deal with. Imagine trying to carve up and serve the world’s largest watermelon on record – 250 pounds. You could deal with it and feed a lot of people but you may need some special equipment to lift and cut it (perhaps a pulley system to lift and a machete to cut it). A 250 lb watermelon is problematic – but manageably so. Ditto with Big Data. 

That’s fine but so what – am I not just saying that Big Data is ‘big’ and we have some specialised tech that can handle it? The definition and analogy above fail to convey the most important aspect of Big Data which is that the ‘big’ is getting bigger at an exponential rate and the rate in which it is beginning to permeate society is growing similarly.  Big Data is what happens when data meets Moore’s law.

For anyone not aware of Moore’s law, it is the observation that computing power doubles every 12 to 18 months and costs halve in the same period.

Just step back for second and think about what this means. Imagine if Moore’s law applied to cars. A Ferrari purchased in the 1980s costs £200,000 in today’s money and had top-end speeds in the order of 160mph. If Moore’s law applied to Ferraris where the price halved and performance doubled every 12 – 18 months, the 2015 model would cost less than a bar of chocolate and would have top speeds nearing the speed of light.  So would a Ford Fiesta for that matter. The exponential scaling described by Moore’s law is so alien to our everyday experience of things that it’s difficult to comprehend and appears absurd when placed in the context of physical objects like Ferraris.

Although the Ferrari example is fantasy (unfortunately!), Moore’s law very much applies to data as data leverages that tech which is halving in cost and doubling in capacity. The cost of dealing with data is collapsing like the price of Ferraris in the above example, while capabilities are accelerating.  On top of that, the quantity of high (and low) volume data sources has exploded which is also an effect of Moore’s law. 

Moore’s law is powering an extraordinary virtuous circle – the cheaper it is to process data, the more we process and the more we generate. The more we generate the more we process as Moore’s law makes it cheaper and easier to process the next time round. The process is exponential and is feeding on itself, growing ever more large, pervasive and economic and ever more, well, everywhere.

That’s what we really mean by ‘Big Data’. It’s not just about size – ‘big’ also refers to the rate data usage is growing and being applied to increasing numbers of applications in all sorts of areas including the pensions industry. The revolution is in the extraordinarily increasing availability of data and the plummeting cost of leveraging it. Moreover, there is a revolution in the expectation that you can and should be using data much more intelligently than you have been before. Big Data means we can do a lot of things economically that we could not do only a few years ago.

Now, if dealing with quadrillions of bytes of data is now easily doable, albeit with some specialist kit, what do you think the cost of dealing in pension-related data that is nowhere near that volume is going to be? That’s right - cheap as chips. That’s why the Big Data revolution has arrived in the pensions industry. Big Data has made leveraging data to enhance the pensions industry economic to produce, store, process and use. It means the cost of analysing, digitising and automating processes is crashing while the demand for and acceptance of data driven analytics and automation is growing. 

In summary, the ‘big’ in Big Data is not just about quantity of data, it also refers to magnitude of impact and is a form of significant technological progress. Big Data is Big.

Big Data has arrived in the pensions industry because there are problems in the pensions space it can solve, the technology is mature and the costs have reached the point where we can apply data driven methods economically. The pensions industry on the cusp of change, and needs these tools and techniques to succeed in the new landscape.

This is the first of three articles about Big Data and its application to pensions. My second article will discuss the data driven projects that are beginning to pop up and how data driven techniques are already touching everything from fund design and governance to member communications and engagement and applies to you whether your role is trustee, manager, HR professional, provider or adviser.

Finally, in Part III I’ll look at where things are heading in the near future – seemingly futuristic technologies like “roboadvice” are getting much closer to reality than you may realise and are already being evaluated by regulators. We all know that the pressures on the industry mean it is ripe for change and that it will change. What may not be clear to many is how much Big Data and the forces behind it are driving the technological innovations that will be enabling those changes. The way services are delivered and consumed and the nature of the services themselves are about to be transformed in fundamental ways.

Connecting with you

Get in touch if you have a question or would like to explore how our solutions could help your organisation.

Get in touch