Where’s My Magic Quadrant?


Stephen Pace

When I got involved with Kalido over 15 years ago, we had big plans to change the world of data warehousing, and we still do.  However, one frustration we’ve had over the years is recognition from the major analyst firms that how we build data warehouses is fundamentally different and merits a “Quadrant” or “Wave” of our own.  In a nutshell, we think that the old way of building models with ER modeling tools, having DBAs create physical tables from those models, and then hardcoding the population of those tables with an ETL tool can never allow a business to be as agile as they need to be.

Fill in the blanks

Gartner Magic QuadrantForrester Wave

In the early days, we generally got a very good reception from the analysts, but were told that while what we were doing was innovative, they couldn’t have a quadrant with just one vendor.  Fair enough.  I recall one skeptical analyst told us to come back and see him after we got five customers, and when we came back after 8 he seemed genuinely surprised.

We were also told, “if there is no competition, there isn’t a market.”   Given that we always considered our primary competition the traditional approach (custom warehouses), this surprised us.   Kalido felt that data warehousing was facing a build vs. buy situation, just like how every other piece of software on the market had evolved.  That said, we did see other companies in the space.  Examples included AppsMart and Marc Demarest‘s DecisionPoint Software.  I never liked the term “Packaged Data Warehouse”, especially for us, but along those lines Acta had their Rapid Marts, PeopleSoft had EPM, Sybase had Industry Warehouse Studio, and even lumbering SAP eventually came out with BW.  When you are a startup, getting lumped in with those vendors wouldn’t have been a bad thing for us, but as far as I know, a quadrant was never developed for those packages either.

Some analysts like our founder Andy Hayler, now CEO of analyst firm Information Difference, have chosen to include us in with other database products:

information-difference

While I’m happy to get recognition on any analyst chart, I think melding us together with other players in the data warehousing space like this can lead to confusion.  For instance, for people that don’t understand what we do and don’t read the text down below the chart, it might look like we compete with Teradata or Oracle.  In reality we partner with those vendors because we require a relational database to function and indeed offload most of the processing to the database itself.  This Extract Load and Transform (ELT) approach fits very well with these vendors, especially in the MPP space where Kalido can fully exploit the power of these high-performance database platforms.

Eventually some analysts noticed the trend and started to look at this space in greater detail.  For instance, Wayne Eckerson  (now at his own consultancy, BI Leader) in his role as Director of TDWI Research dubbed the space ‘data warehouse automation’ software (a name I really like) and in May 2009 even held a webinar showcasing what he felt were the top products in the space at the time:

tdwi-webinar

In this webinar, speakers from Balanced Insight, BIReady, Kalido, and WhereScape each got 15 minutes each to present and answer questions.  I thought the format was great.

Fast forward almost 5 years, though, and still no quadrants.  Meanwhile Kalido has hundreds of implementations around the world in almost every vertical you can name.  And while I wouldn’t consider any of the competing vendors as solving the problem in the same way we have, nor to the extent we have (including BI tool semantic layer generation and having a related MDM product), there is competition.  There are even early stage open source offerings, like Quipu and Lars Rönnbäck’s tool to support Anchor Modeling!

To be fair to the analyst firms, they haven’t ignored us.  For instance, Forrester’s Boris Evelson wrote a paper in April 2010 on “Agile BI Out of the Box” comparing the leading vendors in our space.   The thing I liked about it is that it aligns well with the Kalido way of thinking that no matter what your BI tool is, it is impossible to have Agile BI if the data foundation that the BI tool is using is not itself agile. (Boris used the term “metadata-driven”, which I also like.)  But for however many analyst mentions we get, I feel it still isn’t a substitute for full recognition of the space.

So what’s it going to take?   I know it really isn’t up to us and that the analyst firms move at their own pace, but perhaps there is something we can do to accelerate it:

  1. As vendors, we should probably try to agree on what to call the space and start referring to ourselves in those terms.  I personally like Eckerson’s  “Data Warehouse Automation”, but I’m open to other suggestions.  Agile Data Warehouse?  Metadata Driven Warehouse?  Agile BI Foundation?  I would like to avoid “Packaged Data Warehouse” as it tends to indicate an inflexible, rigid set of tables which is contrary to the agile foundation businesses demand.
  2. Encourage our customers to present their successes at conferences and trade shows and sing the praises of data warehouse automation until this space can no longer be ignored.

We have the customers.  We have the success stories.  We have a track record that spans more than a decade.  We have vendor competition.  It’s time for a quadrant!


Author’s note:   Please send me your feedback!  If this is useful, I can work on related pieces:

  1. What would a quadrant look like if current business needs and capabilities were taken into account?
  2. Have vendor ranking tools used by analysts kept pace with the evolution of business needs and vendor responses of the past five years?

Tags: ,

One Response to “Where’s My Magic Quadrant?”

  1. Stephen Pace
    Stephen Pace February 26, 2014 at 11:11 am #

    One important update to this quest is that TDWI now offers a class in Data Warehouse Automation! The details can be found here:

    http://events.tdwi.org/Events/Las-Vegas-World-Conference-2014/Sessions/Monday/Data-Warehouse-Automation-Better-Faster-Cheaper-You-Can-Have-It-All.aspx

Leave a Reply