Forrester’s 2018 PAML “Waves”
Forrester just published two “Wave” reports for predictive analytics and machine learning. The first, covering “multi-modal” solutions, is available here for free. A second report, covering notebook-based solutions, is available here (registration required.)
Forrester plans to publish a third report, covering automated machine learning vendors, in 2019.
Kudos to Forrester for understanding the diversity of the data science tools market. Software with a visual interface does not compete with code-centric software — it appeals to a different class of users. Instead of trashing code-based tools as “too hard to use,” Forrester recognizes that they belong in a separate category.
Let’s take a quick look at how vendors fared in each report
Multimodal Predictive Analytics and Machine Learning Platforms
Here’s the “Wave”:
— SAS did well. Forrester gives a glowing review to SAS Visual Data Mining and Machine Learning. That squares with what I hear from the few customers willing to pay for it. Use a wizard to automatically train a model is a bit of a stretch, though. VDM/ML supports automated parameter tuning, but data engineering, feature engineering, experiment management, model evaluation, and model selection are all manual tasks. Oh, and for model management, you need to license another SAS product.
— Forrester’s assessment of IBM makes less sense to me. Watson Studio is a quodlibet of previously available services, cobbled together and pushed out the door just in time for analyst review season. Those “SPSS-inspired” workflows look an awful lot like — wait for it — SPSS, which IBM did not submit for review because it’s so done. IBM Watson Studio is only available on IBM Cloud, everyone’s fifth choice in cloud platforms, which makes it seem more like a niche product. Does anyone actually pay for Watson Studio? I’ve only run into it when some Blue customer gets free credits with an IBM enterprise agreement.
— Forrester notes that RapidMiner helps 380,000 users. If only more of them paid for the privilege.
— Angoss (Datawatch), FICO, KNIME, and SAP all fell out of the “Leaders” category, which was getting pretty crowded in last year’s report. All fell victim to Forrester’s changing metrics.
— TIBCO remains in the “Strong Performers” category, but Forrester rates its current offering much lower than it rated Alpine and Statistica, which TIBCO acquired last year. This demonstrates the maxim that in business, one plus two doesn’t always add up to three.
— Dataiku scored about the same this year as last.
— Microsoft took a big hit, falling from “Strong Performer” to “Contender,” with markedly lower ratings on both dimensions. Bit of a puzzler, IMHO, the MSFT offering seems better than that.
— MathWorks joins the Wave this year and lands about where you would expect.
— World Programming and Salford Systems trail the pack. SAS has not yet litigated the former out of business. Minitab acquired Salford last year. I can remember using Minitab back in the 1970s. Yeah, I’m that old.
Forrester did not rate Alteryx.
Notebook-Based Predictive Analytics and Machine Learning Solutions
Note: this is the updated “Wave” published by Forrester on September 7.
Most of the vendors in this wave are new to Forrester. My comments:
— Domino Data Lab leads the pack, and rightly so. Domino invented this category and leads in every respect.
— Forrester’s assessment of Oracle as a leader seems, well, aspirational. Customer adoption, per the detailed tables, is zero. Insiders from DataScience.com, which Oracle acquired recently, throw shade at the product’s stability and maturity. Presumably, Oracle has the deep pockets to fix the product and make it work. Even so, it’s not nearly as good as Domino; I’d share a detailed feature/function analysis, but it would take more than a paragraph. Oracle lacks Domino’s street cred with the data science community, and the folks in Oracle Cloud who drove the acquisition don’t talk to the folks in Oracle Data Mining, who have actual customers and experience in the field.
— For this wave, Forrester did not evaluate H2O.ai‘s Driverless AI. Forrester wasn’t impressed with Sparkling Water and Flow UI. Enterprises looking for a notebook-based PaML solution will find better solutions from the other vendors in this evaluation. Ooh, burn.
— My former colleagues at Cloudera should be pleased with their positioning in the middle of the pack. Databricks did well, too. Forrester dings Cloudera and Databricks for using proprietary IDEs instead of Jupyter. I’m sorry, but the folks at Cloudera and Databricks aren’t stupid — they understand that Jupyter isn’t suitable for production software development. Don’t @ me.
— Civis Analytics‘ main asset is its founders’ political connections. Forrester rightly notes that Civis is not yet the platform for everyone, though, as it is currently cloud only, it doesn’t support many machine learning frameworks out of the box, and Spark is still in the pipeline. That’s like saying dinner is ready, but we have no rolls or salad and the roast is still in the oven.
— It must sting Anaconda to score below OpenText, but there it is.
— Google brings up the rear with Cloud DataLab which, according to Forrester, does little to improve data scientist productivity, such as through project capabilities, team collaboration features, and other modeling tools that are important criteria in this evaluation. Yeah, that’s about right.
Surprisingly, Forrester did not evaluate Amazon Machine Learning.