Gartner’s Magic Quadrant is stirring up emotions again. This time ZL Technologies have launched a law suit against the analyst firm, essentially claiming that its methods are biased and obscure. We’re not industry analysts, or partners of any of the vendors, so we’re not too bothered about who’s in Gartner’s good books. It makes a big difference to the vendors, however, since Gartner is such a dominant influence in the industry and so many clients assume that if a product’s in the Magic Quadrant, it must be the best.
And yet, this precisely contradicts Gartner’s own advice:
Gartner advises organizations against simply selecting vendors that appear in the Leaders quadrant. All selections should be buyer-specific, and vendors from the Challengers, Niche Players or Visionaries quadrants could be better matches for your business goals and solution requirements.
But what clients and many consultants see is the graph, and this is what they decide on. We’ve worked with many of the WCM products assessed by Gartner and conducted many technology selections for clients. They want the best product, not a niche player.
But what do you want to do with your CMS? Don’t you want to achieve things that other people aren’t doing, within business structures that will be difficult to change, aimed at specific audiences? Isn’t that a niche? Then why wouldn’t you consider a niche product?
Just because a vendor has a more complete vision, doesn’t mean it offers all the features that niche products do. In fact, the completeness of vision is based on many other criteria, including market understanding and strategy, sales strategy, business model and geographic strategy. These are all important, but do they really have a bearing on your business requirements?
We’d rather come and ask you what you’re trying to achieve, point out the things that any CMS will do and some of your issues that only certain products are likely to solve well. We’ll suggest you look at those but warn you about some of their weak points. If you’re then concerned that the vendor’s marketing strategy isn’t up to scratch, go and take a look at their financial viability. But every vendor Gartner assessed had WCM revenues in excess of $8 million in 2008, so they aren’t small fry.
Nevertheless, you have to question the neutrality of a firm that takes a significant proportion of its revenue from advising the vendors on product development, but doesn’t disclose what that revenue is. As a buyer, you should question whether the criteria are relevant and whether the assessments are fair.
So what benefit can you get from the report?
Firstly, you get a list of products. That’s not a trite observation. In a market with several hundred vendors — and seemingly more each day popping out of the Scandinavian CMS womb — it’s useful to be able to limit the products you’re considering to those that have a considerable industry presence. Gartner will shortly be adding open source WCM to the proprietary software it currently evaluates.
Secondly, you get some ammunition with which to question vendors. If EPiServer is heavily focussed on expanding into the US market, you should be asking how much of their core team is still in Europe and able to deal with your concerns. (This is true of many of the European vendors.) Similarly, if you read between the lines on cautions about Vignette, you’ll need to ask how many of their clients are actually using the latest version of their product which they’re so keen to sell you.
So how should you read Gartner? With interest, and with caution.
Some further reading:
I agree with you on all the points. But maybe this lawsuit is a cheap way to do marketing.How strange: http://www.zlti.com/company/press/releases/archives/2009/10-19-09_PR_70LargeEdiscovery.html
They annouced a new product release…
Comment by R. — 26 October 2009 @ 9:51 am
Hello,
On December 4, 2009, ZL Technologies filed an amended complaint against Gartner, Inc. in the United States District Court for the Northern District of California. The Court granted ZL the opportunity to clarify and augment our earlier allegations of defamation and trade libel.
In the first round of ZL’s legal dispute with Gartner, Gartner argued to the Court that its rankings and other statements in the proprietary “Magic Quadrant Reports” are merely opinions that are not based upon fact, and that they are understood as such by the readers of those reports. However, Gartner’s past statements in marketing materials, white papers, blogs and even the Magic Quadrant Reports themselves, assert that their research and analysts’ opinions are based on a body of facts compiled through what is asserted to be a rigorous process.
The amended complaint clarifies ZL’s contentions about the inaccuracy of Gartner’s reports, the inherent conflict of interest arising out of Gartner’s voluminous business with the vendors it reviews, and its subsequent bias towards large and established vendors. The amended complaint also adds new detail about Gartner’s repeated claims that its research is based on objective fact—a position exactly opposite to the stance forwarded by Gartner in court.
While this case is focused on ZL’s dispute with Gartner over the erroneous statements in Gartner’s publications, the issues here also implicate Gartner’s larger business model. Gartner plainly admits that it attempts to leverage value from its largest clients, many of whom are also vendors covered in the company’s research. ZL’s legal filings describe how that business model causes Gartner to favor those large companies at the expense of identifying the best technologies, thus misleading not just the vendors who are inaccurately reviewed by Gartner, but the consumers who base their IT purchasing decisions on Gartner’s biased research.
ZL is seeking injunctive relief as well as compensatory and punitive damages from Gartner.
The amended complaint can be found here: http://www.zlti.com/courtdocs/docs/First_Amended_Complaint.pdf
Comment by Rob — 8 December 2009 @ 4:59 pm