How relevant are Gartner and other tech analysts really?

How relevant are Gartner and other tech analysts really?

I was at an event recently organized by Sage software in Johannesburg. This was an event to promote Sage’s independent software partners to other consultants and customers in it’s large community. I spent a lot of time talking with various people, many of whom operate fairly small businesses but are very knowledgeable of their area of business. It was only afterwards that I realized I was never asked how analysts rate our platform, rather we were asked what we did and how we might help. In other words, none of the attendees cared, or even knew, about Gartner. Continue reading


Do analyst organizations “speak with forked tongue”?

In the old cowboy movies, when a ‘red’ Indian said that someone was being duplicitous or exhibiting a double agenda, the words they were given to say were ‘XXX speaks with forked tongue”.

Analyst organizations often extol the values of the Internet, allowing teams to work together virtually, while being disconnected physically. But when you see a job opening for an analyst company it is frequently for a location which coincides with the analyst main office.

So when I saw this blog from the TechnologyEvaluation group for a new analyst position, the “forked tongue” comment came to mind.

Do you think I am being unfair?

Is IDC out of step?

I am coming to the conclusion that the research group International Data Corporation (IDC) is out of step with just about everyone else when it comes to the research they publish.

This first came to my attention when I was referred to an IDC report on the top ERP vendors. Looking at the report’s table of contents, I was surprised to see that companies such as TOTVS, Activant, Micros Systems, CGI and Torex Retail, were included. In no way can these companies be called top ERP vendors. So I was left wondering on what basis did IDC identify its vendors.

Now, there is a report that IDC predicts 27% compound annual growth in the Software-as-a-Service (SaaS) market, only a few days after Gartner is reported as “backing off the software as a service (SaaS) bandwagon“.

Whether you are a customer or a vendor, do you wonder where IDC are getting their data?

Presenting for a Gartner Magic Quadrant

Sometime in the next few months, the Gartner analyst group is going to be releasing an updated Magic Quadrant (MQ) of the mid-market ERP software vendors. The reason I know this is because I was heavily involved in preparing the presentation that SYSPRO gave to the Gartner analysts.

Its an interesting experience to prepare for an MQ session – interesting as in the Chinese curse: may you live in interesting times. Or as a former mentor wrote to me: “the prep for Gartner is a killer and usually disproportionate to the benefit.”

For me, it started several weeks before, documenting the company’s product roadmap. But the work became really intense in the week leading up to the presentation.

If you aren’t familiar with it, Gartner evaluates product and service providers on an MQ against two criteria, made up of different measures:

  1. Completeness of Vision
  2. Ability to execute

Under “Completeness of Vision”, the measures and scoring are:

Measure Scoring
market understanding high
marketing strategy standard
sales strategy high
product strategy high
business model low
vertical/industry strategy standard
innovation low
geographic strategy standard

For “Ability to execute”, they are:

Measure Scoring
product high
overall financial viability standard
sales execution/pricing high
market responsiveness, track record low
marketing execution low
customer experience high
operations low

What first struck me was when I saw Innovation with a low scoring. On further reading of their preparation document, Gartner explained that this score was set because mid-market companies are usually not innovation focused. This corresponds with SYSPRO’s positioning in the market – what we call the ‘pragmatic buyer’. However I was surprised that Track Record was scored low as mid-market decision-makers tend to use word-of-mouth in evaluating suppliers.

Another discrepancy for me was that while Marketing Strategy was scored standard, marketing execution was low. Does this show that Gartner care more about spin than substance? Probably not, as the scores for Sales Execution and Customer Experience are high, indicating that Gartner do look at results and real-life experiences.

One caveat – while Vertical/Industry Strategy has a standard rating, don’t assume this isn’t significant. Other members of our presenting team knew from experience that Gartner like to see industry-focused strategy clearly spelt out.

Before the first call we debated whether we should show the product or discuss it. Our decision and approach was greatly assisted by reading Carol Rozwell’s blog ‘Vendors: suggestions to maximize briefing value’.

We have now done the first conference call, and are waiting for Gartner to get back to us to present their initial ideas and findings – this is to give us an opportunity to respond and suggest amendments.

If you have gone through a Gartner MQ briefing, how did you find it? Do you think it is worth it? It occurs to me that a little wiki could be set up to provide ideas, guidelines and good practices for vendors approaching an MQ call.

Gartner’s prediction accuracy

Care of a Twitter note from Hutch Carpenter, I have been reading a Gartner report on the state of the hype cycle for social software.

It was interesting reading, but I am going to save it (or social bookmark it) for reading in a few years time. The reason: back in about 2001/2002, I remember Gartner doing a report on the state of ERP II – being inter-company (between company) transacting, as opposed to intra-company (within company) which is where ERP’s strength was supposed to be.

At the time, I recall Gartner making major predictions about how ERP II would be a new revolutionising tool for business. 6-7 years later I am still waiting to see that.

I wonder if the Gartner analysts ever review old predictions and see how accurate they were? It would be interesting to see the analysis.

My take on latest analyst views

My favourite analyst site, AMR Research, often has raises some issues on its ‘First Thing Monday‘ page that provide me with blogging ideas:

A Google chrome comment – summarises the prospects of Chrome from an enterprise perspective as a

browser, a platform for Google Apps, omni-client platform strategy and a replacement for the Windows environment for desktop and mobile applications

The state of enterprise software skills in the US – AMR now believes only 2 ERP vendors, SAP and Oracle, are the main players in the US. I wonder what Microsoft thinks of that?

I find the Gartner site to be pretty poor in terms of getting quick news analysis, but one of their analysts was on ClassicFM last night (again) discussing some discontinuities that are coming from the Internet. At least he was aware of SA’s local issues around the Internet, e.g., why Software as a Service (SaaS) is not taking off here due to our Internet bandwidth problems. Also discussed was the future impact of social software – very much aimed at the big corporates rather than SMB market – which makes it sound unimportant to the majority of SA companies.

Why analysts don’t know what’s going on in SA

When I read analyst reviews of what’s going on in various markets, I often get the feeling that none of it applies to South Africa. Now I understand why.

In a recent AMR note about Oracle in Africa and the Middle East, Bruce Richardson made the telling comment that AMR hadn’t visited SA in 11 years.

11 years ago I was the Alliance Manager at Digital Equipment SA responsible for, among others, 10 ERP players. In the hardware vendor market, there were at least twice as many players as there are now. In South Africa, we still had Nelson Mandela as president, and there had been only one open general election. The local IT environment had developed significantly, and we aren’t so dominated by international players.

Shame on you AMR. South Africa has changed hugely in 11 years, and you haven’t even bothered to update yourselves.