What is Big Data and Why Should Consultants Care?

Several months ago, I asked one of our members, Greg Richards, involved in the study and application of big data to comment on it's potential effect on consulting.  He has assembled some courseware on the subject that will be made available to members in the next few months.  In the interim, I thought it would be useful to reprint his original submission.

What is Big Data and Why Should Consultants Care?

Gregory Richards, MBA, Ph.D, FCMC

By now, the hype engine is in full gear about Big Data and Analytics (BDA) and many consulting firms are already providing services in this area.  To me the real question is can BDA completely transform consulting as we know it? Consider for example, Mckinsey Consulting’s ”Solutions”, an approach that embeds hardware and analytics in the client organization as an early example of the disruptive potential of BDA.

Before we get too far ahead of ourselves however, we should clarify what we mean exactly by BDA. Once we understand the phenomenon, we can then address how it fits with our standard consulting models and whether indeed, it represents a true disruption.

What is Big Data?

Big Data is commonly defined as data that cannot be processed by standard information technology systems within organizations. However, some “standard” databases can now handle over 100 terabytes of data (equivalent to about half-a- billion pages of text) so although “Big” has captured the imagination of vendors and practitioners alike, it’s not really about the volume of data.  Storage and processing speed are relatively cheap commodities nowadays. The reality, however, is that data takes different forms:  numbers, text, sound, images and its moving quickly…over 10 terabytes per day on Facebook for example.  To me, term BDA refers to three things:

  1. The data itself-internal data and streaming data of different forms available from various social media sites, sensors and satellites;
  2. The technologies available to process this data; and
  3. The techniques available to use this data for descriptive, prescriptive and predictive analytics.

Potential Impact on consulting

Advances in available data, technologies and techniques can potentially change some of what we do in traditional consulting. Consider for example the standard 5-phase consulting model.

Generally, client and consultant work through a joint problem solving approach and the consultant then conducts some form of data collection.  Analysis and synthesis follow leading to recommended solutions and eventual implementation.

In a BDA world, the client has monitoring devices available that defines the problem in real time. Consider the situation where our clients are continually monitoring things they care about (such as their customer comments through social media sites or through continual on-line surveys). Therefore, customer related problems are flagged early and more or less continually. In this scenario, joint problem solving might not be as important.

Assume then, that the problem can be flagged through near-real-time monitoring.  The data is also readily available, so what might be needed from consultants is assistance with analysis and synthesis.  In addition, consultants might help with interpretation and implementation of solutions.  This approach is completely consistent with trends in consulting which calls for consultants to deliver tangible results for clients; nowadays the thick report is not always enough.

Another important point to consider is that in 2011 there were about 7 universities in the US and Canada offering degrees in BDA.  In 2015, that number has grown to approximately 80.  This is good news in that it means that there will be a supply of talent available if we want to build a BDA service in our consulting practices.  It also means that our clients-to-be will be familiar with BDA and will therefore be informed consumers of consulting help in this arena.  Clearly, it can also be a threat in that many potential clients can hire this new data-savvy talent directly.

What does it mean for consultants?

First, we can see from the scenario above that our typical consulting process could be considerably shortened. This could be good news for both consultant and client, but it calls for different consulting skills. Second, the emphasis could be shifted to implementation if indeed, many of the tools available can automate or semi-automate the diagnostic stage (think of IBM’s Watson computer and its ability to diagnose Jeopardy questions).

Despite the foregoing and the excitement about BDA, it is still early days.  In my view, as consultants we don’t need to completely retool, but we should consider the following:

  1. Become familiar with the terminology and the processes involved in BDA (the data, the technology and the techniques). It’s not necessary to transform ourselves into “data scientists”, but we should be able to talk to clients intelligently about the topic.
  2. Invest some time in brushing up on diagnostic skills. Diagnosis is one of the main phases in the CMC-Canada 5-phase model. Clearly BDA techniques are evolving and influencing how we might conduct this phase. We need to stay current with recent advances in this arena.Reviewing those statistics courses you might have followed in college would be a good place to start to recall the basics in order to better understand the new world of analytics.
  3. Stay in touch with the field through credible sources. Perhaps we need to consider formation of BDA special interest groups or connecting with groups such as Data for Good to better appreciate where the field is going.

Gregory Richards is Director of the MBA program at the University of Ottawa’s Telfer School of Management. He manages 2 research clusters focused on the use of data and analytics in improving organizational performance.