According to analyst firm Gartner, Big Data is at the portion of the hype cycle called the “peak of inflated expectations”.

The business world is awash with all sorts of claims about the magic of Big Data and how it will transform industries by increasing productivity and profits and opening up opportunities that nobody even knew existed.

But this will only happen if companies are able to hire enough people who actually understand what Big Data is, how to collect it, and preserve it. Computing and analytical skills are also required to get Big Data to reveal its hidden secrets and visualise it in novel ways. And there unfortunately, is the rub. There are just not enough data scientists, people with the required skills to satisfy this unmet demand.

The shortfall in Big Data experts is set to rise and in the UK alone, one digital industries employer body has predicted there will be a need for 69,000 of these experts in the next five years. This claim is not original. Back in 2011, McKinsey & Co was claiming a US shortfall in Big Data experts of 140,000 - 190,000 by 2018.

The shortfall in Big Data experts is being manifested in a number of ways. The first and most obvious is through recruiters casting an ever-widening net in their search for appropriate talent.

There is some agreement that Big Data analysis and data visualisation requires skills in computing as well as statistics and mathematics. This has meant that university graduates with statistics, computer science and engineering have been the main source of potential employees. Lately this has widened to include subject areas such as astrophysics and computational chemistry.

The demand for Big Data experts has driven a more direct approach to tackling the shortfall problem with industry partnering with universities to create courses, majors and degrees that focus on these specific skills. In Australia, Macquarie University will offer a Master’s degree in Data Science. In the US there are a number of universities that offer postgraduate degrees in disciplines that cover skills required for Big Data professionals.

Even MOOC provider Udacity has partnered with Big Data database provider Cloudera to put on specific courses on Big Data. Rival MOOC provider Coursera, has three Big Data related courses available.

In the midst of the Big Data hype, some clear notes of sanity have come from an unexpected quarter. UK supermarket giant Tesco has claimed it has no problems hiring graduates with the right skills. Basically its strategy is to look for smart people who may be mathematicians, scientists or engineers.

Surprisingly enough, this vindicates the long-held belief that universities should first and foremost produce graduates who are equipped with the ability to think, apply those thoughts and solve problems.

Although over-emphasised by some in industry, product or technology specific skills are largely useless. By the time universities mobilise to refactor their courses to the latest hyped technology and graduate students with those skills, the world will have moved on to the next “Big” thing.

Big Data is real and some of the challenges it poses will need to be solved by scientists and engineers and mathematicians in the coming years. Setting aside truly massive data being generated by radio astronomy and some other fields of science, we are actually mostly there in terms of having techniques and technology that allows us to process and make sense of Big Data. Underlying this all however are the general skills required to handle and make sense of all data, big and small. These skills still rely on knowledge of basic mathematics, statistics, science and computing.

What the interest in Big Data has done is to highlight to companies the importance of data generally. It’s not as though companies have never looked at data analytics before, they simply may not have recognised its central importance to the business, nor understood what the data has been trying to say for years. The task is to convince them that universities are already producing graduates with the right skills and answers, industry only needs to ask the right questions.