Big Analytics Roundup (March 30, 2015)

Lots of Spark news this week, following last week’s Sparkalanche, plus some other non-Spark news just to show that Big Analytics isn’t entirely about Spark.

Alteryx

In IntelligentHQ, Maria Fonseca interviews Alteryx COO George Mathew, argues that analytics is for people. Left unanswered: who else it could be for.

Analytic Startups

Analytics vendor Ayasdi lands a $55 million “C” round.
Localytics, which specializes in analytics for mobile and web apps, secures a $35 million “D” round.

Apache Drill

MicroStrategy announces certification of Apache Drill with MicroStrategy Analytics Enterprise Platform.

Apache Spark

Analysis

IBM Big Data “evangelist” James Kobelius confirms that IBM has no idea what to do with Spark.
In TechRepublic, Matt Asay argues that Hadoop won’t disappear just because it’s slow, knocking over several straw men in the process. On readwrite, he makes similar points; and on InfoWorld, he goes for the hat trick.
In InfoWorld, Platfora’s Peter Schlampp offers five reasons why Spark is the next big thing.

Applications

On the Cloudera blog, Sam Shuster of Edmunds.com describes a dashboard built with Spark Streaming, SparkOnHbase and Morphlines.
In InfoQ, Srini Penchikala of Pinterest explains why he’s using Spark Streaming, Kafka and MemSQL for a real-time application.

Data Science

On the Databricks blog, Joseph Bradley writes an excellent article on Topic Modeling with Spark’s new Latent Dirichlet Allocation capability.

Developer

On the Databricks blog, Michael Armbrust describes new Spark SQL features in Spark 1.3
On Slideshare, Vida Ha and Holden Karau share tips for writing better Spark programs; video here.

Deep Learning

Tomasz Malisiewicz of Vision.ai blogs on Deep Learning versus Machine Learning versus Pattern Recognition.

RapidMiner

RapidMiner publishes a white paper on code-free analytics in Hadoop, and another on Hadoop security.

Big Analytics Roundup (March 23, 2015)

This week, Spark Summit East produced a deluge of news and analysis on Apache Spark and Databricks. Also in the news: a couple of ventures landed funding, SAP released software and SAS soft-launched something new for SAS Visual Analytics.

Analytic Startups

Venture Capital Dispatch on WSJ.D reports that Andreeson Horowitz has invested $7.5 million in AMPLab spinout Tachyon Nexus. Tachyon Nexus supports the eponymous Tachyon project, a memory-centric storage layer that runs underneath Apache Spark or independently.

Social media mining venture Dataminr pulls $130 million in “D” round financing, demonstrating that the real money in analytics is in applications, not algorithms.

Apache Flink

On the Flink project blog, Fabian Hueske posts an excellent article that describes how joins work in Flink.

Apache Spark

ADTMag rehashes the tired debate about whether Spark and Hadoop are “friends” or “foes”. Sounds like teens whispering in the hallways of Silicon Valley High. Spark works with HDFS, and it works with other datastores; it all depends on your use case. If that means a little less buzz for Hadoop purists, get over it.

To that point, Matt Kalan explains how to use Spark with MongoDB on the Databricks blog.

A paper published by a team at Berkeley summarizes results from Spark benchmark testing, draws surprising conclusions.

In other commentary about Spark:

TechCrunch reports on the growth of Spark.
TechRepublic wonders if anything can dim Spark.
InfoWorld lists five reasons to use Spark for Big Data.

In VentureBeat, Sharmila Mulligan relates how ClearStory Data’s big bet on Spark paid off without explaining the nature of the payoff. ClearStory has a nice product, but it seems a bit too early for a victory lap.

On the Spark blog, Justin Kestelyn describes exactly-once Spark Streaming with Apache Kafka, a new feature in Spark 1.3.

Databricks

Doug Henschen chides Ion Stoica for plugging Databricks Cloud at Spark Summit East, hinting darkly that some Big Data vendors are threatened by Spark and trying to plant FUD about it. Vendors planting FUD about competitors that threaten them: who knew that people did such things? It’s not clear what revenue model Henschen thinks Databricks should pursue; as Hortonworks’ numbers show, “contributing to open source” alone is not a viable business model. If those Big Data vendors are unhappy that Databricks Cloud competes with what they offer, there is nothing to stop them from embracing Spark and standing up their own cloud service.

In other news:

On the Databricks blog, the folks from Uncharted Software describe PanTera, cool visualization software that runs in Databricks Cloud.
Rob Marvin of SD Times rounds up new product announcements from Spark Summit East.
In PCWorld, Joab Jackson touts the benefits of Databricks Cloud.
ConsumerElectronicsNet recaps Databricks’ announcement of the Jobs feature for Databricks Cloud, plus other news from Spark Summit East.
On ZDNet, Toby Wolpe reviews the new Jobs feature for production workloads in Databricks Cloud.
On the Databricks blog, Abi Mehta announces that Tresata’s TEAK application for AML will be implemented on Databricks Cloud. Media coverage here, here and here.

Geospatial

MemSQL announced geospatial capabilities for its distributed in-memory NewSQL database.

J. Andrew Rogers asks why geospatial databases are hard to build, then answers his own question.

RapidMiner

Butler Analytics publishes a favorable review of RapidMiner.

SAP

SAP released a new on-premises version of Lumira Edge for visualization, adding to the list of software that is not as good as Tableau. SAP also released Predictive Analytics 2.0, a product that marries the toylike SAP Predictive Analytics with KXEN InfiniteInsight, a product acquired in 2013. According to SAP, Predictive Analytics 2.0 is a “single, unified analytics product” with two work environments, which sounds like SAP has bundled two different code bases into a marketing bundle with a common datastore. Going for a “three-fer”, SAP also adds Lumira Edge to the bundle.

SAS

American Banker reports that SAS has “launched” SAS Transaction Monitoring Optimization for AML scenario testing; in this case, “launch”, means marketing collateral is available. The product is said to run on top of SAS Visual Analytics, which itself runs on top of SAS LASR Server, SAS’ “other” distributed in-memory platform.

Spark Updates

Here is a quick roundup of some recent Apache Spark news.

(1) Databricks and Typesafe released results from a survey of 2,136 individuals (mostly developers). Some key findings:

13% of respondents run Spark in production, 20% plan to use Spark in 2015
Most say they expect to use the 82% Spark core to replace MapReduce
88% say they use the Scala API
Respondents split on deployment: 54% deploy Spark standalone, 42% co-located with Hadoop under YARN
62% load data from HDFS, 46% from unspecified databases, 41% from Apache Kafka, 29% from Amazon S3

Analysis from GigaOm here. Copy of the report available here (registration required).

(2) On the Databricks blog, Jeremy Freeman introduces streaming k-means, a capability included in Spark 1.2. Excellent article outlining some of the practical differences between streaming and static analytics.

(3) NewSQL vendor MemSQL announced availability of its Spark Connector, which it claims offers seamless connectivity with Spark. More coverage here, here and here; analysis here.

(4) Learning Spark, by Holden Karau, Andy Konwinski, Patrick Wendell, and Matei Zaharia is now available from O’Reilly Media.

(5) InfoWorld selected Apache Spark (along with 31 other products and open source projects) for its 2015 Technology of the Year Award.

Smart Money: More Funding for Analytics

Funding for analytic ventures remained robust in January, with 17 significant funding transactions and three acquisitions. Key themes:

Outcomes-based medicine and health care
Vertical solutions for the energy industry
Solutions for risk management
Mobile analytics, including location-based targeting and app metrics
Social media sentiment analysis
Graph engines (and solutions based on graph engines)
In-memory SQL engines

All funding news via Crunchbase.

Funding

Health Catalyst led the way with $41 million in Series C funding. Health Catalyst offers a solution stack consisting of a proprietary data warehouse optimized for electronic medical records, plus analytic applications designed to support outcomes-based health care.

Other transactions greater than $1 million include:

— MemSQL, provider of a high performance in-memory distributed database, raised $35 million in a Series B round.

— Still in stealth mode, marketing analytics provider OrigamiLogic closed on $15 million in Series B funding.

— Kreditech scored $15 million in debt financing. Kreditech uses machine learning and Big Data to offer credit scoring for microlending.

— Radius closed on $13 million in Series B funding. Radius supports B2B targeted marketing and lead generation for small businesses.

— Smart grid analytics provider AutoGrid landed $12.8 million in Series C funding.

— GNS Healthcare leverages Bayesian Networks and Monte Carlo Simulation to deliver solutions for outcomes-based medicine to hospitals, health insurance plans, pharmaceutical companies and other entities in the health care delivery chain. GNS completed $10 million in Series B financing.

— Simple Energy raised $6 million in Series B funding. Simple Energy offers utilities services to improve customer interactions through microtargeting and social gaming.

— Binary Fountain, provider of software integrating social sentiment analysis with BPM, raised $5.7 million.

— 4C Insights integrates social media sentiment analysis with public data to support media planning and targeting. The firm raised $5 million in Series B funding.

— Kontagent secured $4.8 million in venture funding. Kontagent offers mobile analytic solutions to mobile app developers and marketers.

— Offshore analytic services provider Axtria received $4.8 million in venture funding.

— Enigma Technologies raised $4.5 million in Series A funding. Enigma provides a platform for the analysis of public data that includes a repository and directory to sources, plus tools for search, export and simple analytics.

— Lumiata raised $4 million in Series A funding. Lumiata leverages graph engine technology to deliver evidence-based predictions to medical practitioners.

— BI vendor Chartio received $2.2 million in venture funding

— Bottlenose, purveyor of dashboard and insight tools for social sentiment analysis, raised $1.1 million in debt financing.

— Geofeedia, a provider of open source location-based social media mining tools, received 1.25 million in Series A funding.

Acquisitions

There were three acquisitions of note; purchase prices were not disclosed.

— yp, the corporate successor to AT&T Interactive and AT&T Advertising Solutions, acquired Sense Networks on January 6. Sense Networks uses predictive analytics to drive location-based behavioral targeting for mobile ad platforms.

— Pinterest acquired VisualGraph on January 6. VisualGraph, a two-man operation, has developed a distributed in-memory visual search engine.

— Apigee, an API management company, acquired InsightsOne on January 8. InsightsOne offers cloud-based infrastructure for predictive analytics based on Hadoop, plus an in-memory graph engine.