Gartner’s Magic Quadrant is stirring up emotions again. This time ZL Technologies have launched a law suit against the analyst firm, essentially claiming that its methods are biased and obscure. We’re not industry analysts, or partners of any of the vendors, so we’re not too bothered about who’s in Gartner’s good books. It makes a big difference to the vendors, however, since Gartner is such a dominant influence in the industry and so many clients assume that if a product’s in the Magic Quadrant, it must be the best.
And yet, this precisely contradicts Gartner’s own advice:
Gartner advises organizations against simply selecting vendors that appear in the Leaders quadrant. All selections should be buyer-specific, and vendors from the Challengers, Niche Players or Visionaries quadrants could be better matches for your business goals and solution requirements.
But what clients and many consultants see is the graph, and this is what they decide on. We’ve worked with many of the WCM products assessed by Gartner and conducted many technology selections for clients. They want the best product, not a niche player.
But what do you want to do with your CMS? Don’t you want to achieve things that other people aren’t doing, within business structures that will be difficult to change, aimed at specific audiences? Isn’t that a niche? Then why wouldn’t you consider a niche product?
Just because a vendor has a more complete vision, doesn’t mean it offers all the features that niche products do. In fact, the completeness of vision is based on many other criteria, including market understanding and strategy, sales strategy, business model and geographic strategy. These are all important, but do they really have a bearing on your business requirements?
We’d rather come and ask you what you’re trying to achieve, point out the things that any CMS will do and some of your issues that only certain products are likely to solve well. We’ll suggest you look at those but warn you about some of their weak points. If you’re then concerned that the vendor’s marketing strategy isn’t up to scratch, go and take a look at their financial viability. But every vendor Gartner assessed had WCM revenues in excess of $8 million in 2008, so they aren’t small fry.
Nevertheless, you have to question the neutrality of a firm that takes a significant proportion of its revenue from advising the vendors on product development, but doesn’t disclose what that revenue is. As a buyer, you should question whether the criteria are relevant and whether the assessments are fair.
So what benefit can you get from the report?
Firstly, you get a list of products. That’s not a trite observation. In a market with several hundred vendors — and seemingly more each day popping out of the Scandinavian CMS womb — it’s useful to be able to limit the products you’re considering to those that have a considerable industry presence. Gartner will shortly be adding open source WCM to the proprietary software it currently evaluates.
Secondly, you get some ammunition with which to question vendors. If EPiServer is heavily focussed on expanding into the US market, you should be asking how much of their core team is still in Europe and able to deal with your concerns. (This is true of many of the European vendors.) Similarly, if you read between the lines on cautions about Vignette, you’ll need to ask how many of their clients are actually using the latest version of their product which they’re so keen to sell you.
So how should you read Gartner? With interest, and with caution.
Some further reading: