Reviewing Gartner’s predictions from 2010

Is it nearly three months since I last updated this blog? I don’t know if other bloggers have the same challenge I find – often issues come up which I want to blog about, but then my thoughts get interrupted, and then the urgency to blog goes away, and that opportunity is lost. Anyway, I now have the issue, the time and the urgency.

I recently found a report by the Gartner analyst organisation from 2010 with some predictions, most interestingly for 2012, but some beyond that. Predictions, like forecasts, are mostly going to be wrong – some in a small way, others in much bigger ways. If your organisation has a sales and operations planning activity, one of the key elements is feeding actual data back into the forecast, so you can see where the forecast was wrong, if incorrect assumptions were made, and most importantly, how you can make more accurate forecasts. I have never seen or heard Gartner do that with their forecasts. So here am I to do it.

Gartner’s top end user predictions in 2010 were:

  • by 2012:
    1. 20% of businesses will own not IT assets
    2. India-centric IT service companies will represent 20% of the leading cloud aggregators
    3. Facebook will become the hub for social network integration and web socialization
    4. 60% of a new PC’s total life greenhouse gas emissions will have occurred before the user first turns on the machine
  • for 2013:
    1. mobile phones will overtake PCs as the most common web access device
  • by 2014:
    1. most IT business cases will include carbon remediation costs
    2. more than 3 billion of the world’s adult population will be able to transact electronically via mobile and Internet technology
  • by 2015:
    1. internet marketing will be regulated
    2. context will be as influential to mobile consumer services and relationships as search engines are to the web

How are those forecasts looking? For their 2012 predictions, the only one that might be argued as being accurate is number 3 – Facebook. However, even that could be questioned; consider that Saleforce is pushing Chatter as its social network, and Microsoft recently bought Yammer to beef up SharePoint in the social space. Number 1, the cloud prediction, was clearly driven by analyst hype, and is wrong at the moment, so why doesn’t Gartner review this and give another (maybe more realistic) forecast?

The 2013 prediction is interesting because it is definitely coming, I’m just not sure that it will be by next year.

A lot can happen in two years, ask anyone who thought Groupon would be big. So for 2014 I am prepared to accept the second prediction about how many people will be transacting electronically. I doubt the prediction though regarding carbon remediation.

The predictions for 2015 still seem to be extravagant, but who knows what will happen in three years where the Internet is concerned.

Am I being unnecesarily harsh on Gartner about their prediction accuracy? I would be interested to hear what others think.

Understand statistics

A reasonable probability is the only certainty – E. W. Howe

These days companies are much more likely to use statistics to help with planning than they did in the past, and I’m not talking about simple statistics. Complex time series statistics for forecasting can be used quite easily without having a statistician in attendance.

The reason we use statistics with far more abandon is that the combination of large amounts of data plus greater computational capacity makes data analysis quicker, cheaper and easier. No longer do you need a SAS consultant to do your number crunching, you can do it yourself on a PC.

The problem with making statistics more ubiquitous is that a user may not understand the assumptions that go into the statistics – and all statistical calculations rely on assumptions. The value of any statistical result has to be interpreted in conjunction with the confidence and probability of that result.

When using statistics for forecasting, the important thing is to understand how likely that forecast is. A forecast is based on data from the past, and relies on the assumption that the patterns of the past can be used to predict the future with some certainty.

A local meteorological event can be used as an example. The cumulative mean daily rainfall at my house in Johannesburg, for the summer and winter seasons, is shown – the line of concern is the red summer (wet) season.

season_cumul_avg

This is the cumulative mean daily rainfall for each month – the focus here is on December.

monthly_cumul_avg

According to these two graphs, the rainfall for mid-December should be around 225mm for the season, and 60mm for the month.

However, if we look at the month-by-month actuals versus monthly mean, and median, we can see how certain months vary widely about the mean and median; in statistical terms, the standard deviation around the mean changes. That indicates in some periods the mean and/or median are less useful for prediction than other periods.

monthly_avg

When we look at the seasonal and monthly cumulative actual graphs, we can see how the real data is distributed.

Wet season cumulative actual:

season_cumul_act

December cumulative actual:

monthly_cumul_act

The interesting event mentioned above occurred this month. In the early part of the month, it looked like December was going to be a dry month. But unusual atmospheric conditions led to very heavy downpours on the 16th and 17th. As the monthly graph shows, we now have the wettest December since I started recording rainfall at my house in 1997.

For the seasonal graph, the drier than usual conditions this summer can be see by the arrow at the bottom. But all the heavy rain has done is to lift the seasonal rainfall to around the average for this time of the summer.

So while December is exceptionally wet, the season overall is as would be expected from the mean.

What do we learn from this in terms of analysis and prediction:

  1. be careful how you use the mean (sometimes use the median instead) and understand the variability around that point as described by the standard deviation;
  2. the nature and length of the data record is important, in this case December looks wet (for a 30 day period), but for a season (over 200 days) the rainfall is normal;
  3. analyse your data in different ways so you see alternative perspectives;
  4. discuss any forecasts in terms of probability of accuracy.

In the ERP world, the amount of data being stored is giving rise to more business intelligence (BI) and analytical tools. But because these tools can be applied by people without the requisite understanding of data analysis, the results and predictions from the analysis can be faulty.

If you are using a forecasting or optimisation tool for business planning, who is doing the analysis and do they understand the ramifications of how they use the tool?