Thomas Dinsmore's Blog

16 responses to “The Year in Machine Learning (Part Two)”

1. 

   Chris Nicholson

   January 2, 2017 at 1:55 pm

   Hi Thomas –

   Thanks for including us in your roundup.

   The mission statement about bringing Google’s deep learning to the rest of the world dates from 2014, and they only released TensorFlow in late 2015. We actually don’t say that anymore. To put a finer point on it, Google has open-sourced some tools tools. But the infrasctructure that Tensorflow requires to run fast, the proprietary TPUs, and the massive datasets Google uses to train its nets are not public and may never be. Our framework is optimized to run on publicly available chips, and is faster than TensorFlow on multiple GPUs. We also offer a higher-level API. TensorFlow as you may know is relatively low-level like Theano, and works best with a framework like Keras on top.

   What you term a tiny venture round was actually a decent-sized seed round, the first of what may be many, and larger, funding rounds. Everybody’s got to start somewhere. 😉

   Seven of our employees have created profiles on Crunchbase, where I assume you got that data. We actually have 16, and they are listed on our team page: https://skymind.ai/about Our team includes PhDs, a former principle software architect at Cloudera, and an engineer who helped design chips at NVIDIA, among others.

   You’re right to point out that data scientists prefer Python and R. We will be giving them a Python API soon, and currently they can import models to DL4J from Python frameworks: https://deeplearning4j.org/model-import-keras

   Businesses need an AI stack that provides the right tools for at least three teams: data engineers building the pipelines and storage, data scientists, and DevOps. Only the second group works predominantly in Python or R. The first and third groups use a lot of JVM-based tools such as Hadoop, Spark, Kafka, ElasticSearch, Pig, Hive and HBase. We give all three a cross-team solution that allows data scientists to work in Python and then deploy to production on the JVM, where Deeplearning4j is the dominant deep-learning framework. We’re number 6 over all.

   Deep learning frameworks need to optimize for other problems beyond easy prototyping. This includes integrations and containerization. We’re the only DL framework that certified on CDH and HDP. We’re dockerized to run on any OS. And we’ve worked with the teams at NVIDIA, Intel and IBM to run fast and well on GPUs, CPUs and Power chips.

   If you believe updates and activity are indicators of a healthy project, please check out our Github and Gitter pages, which are pretty active:

   https://github.com/deeplearning4j
   https://gitter.im/deeplearning4j/deeplearning4j

   Chris Nicholson
   Skymind.ai

   Reply
   1. 

      Thomas W. Dinsmore

      January 2, 2017 at 3:32 pm

      Chris,

      Thank you for reading. I appreciate the clarifications. A few comments:

      — Crunchbase characterizes your 2016 funding as a venture round, which makes sense since most of the investors in the latest round rarely do seed rounds, and Skymind did a couple of seed rounds in 2015.

      — I agree with you that Google has designed TensorFlow to drive inference business to its cloud platform. Where I disagree with you is in thinking that there is a great benefit for organizations to build out their own deep learning back ends rather than training in the cloud.

      — Can you provide a link to support the claim that DL4J is faster than TensorFlow? If it’s a legitimate benchmark, I’ll publish it. A word of caution, though — unless DL4J is an order of magnitude faster, nobody will care.

      — A Python API will be great when you deliver it. You are correct that data scientists are just one part of the team. They are, however, the most important part, and machine learning projects that lack R or Python APIs will go nowhere.

      — DL4J’s Hadoop integration is a strength for inference, but not a compelling strength for training. Model training workloads are moving to the cloud; frameworks that support this will thrive, others will lag behind.

      — A higher-level API is a good thing if you take it to the logical conclusion and abstract it completely from the back end, as Keras has done. That is the direction of deep learning.

      Every entrepreneur believes that his solution is the killer app. While I can see that Skymind has a distinct approach to deep learning, I’m not convinced that it’s a winning approach. But time will tell.

      Reply
      1. 

         Chris Nicholson

         January 2, 2017 at 5:02 pm

         Hi Thomas –

         Thanks for your thoughtful response.

         > Crunchbase characterizes your 2016 funding as a venture round, which makes sense since most of the investors in the latest round rarely do seed rounds, and Skymind did a couple of seed rounds in 2015.

         Crunchbase is a crowd-sourced database, so not all the information in the profiles is up to date or initially correct. I’ve fixed the profile now. In April we raised money from a group of angels and smaller funds, including Ron Conway’s seed fund SV Angel, as well as some strategics and larger funds as you noted. The round came at the end of YC. I’d qualify the 2015 money as pre-seed, which is not an available tag in Crunchbase, but that’s just splitting hairs.

         > I agree with you that Google has designed TensorFlow to drive inference business to its cloud platform. Where I disagree with you is in thinking that there is a great benefit for organizations to build out their own deep learning back ends rather than training in the cloud.

         It has been widely written that about 85% of all workloads run on prem at the moment, even though many of them will move to cloud eventually. So yes, cloud is the future, but on prem is the present, even if those figures are overblown, and companies want deep learning now, which means it has to be on prem. The best solution will be platform neutral, as we are, running on prem, cloud and hybrid during the intermediate stages. In our case, the backend is already built and works on top of technology they’ve adopted, like Hadoop, Spark, Kafka etc. TensorFlow doesn’t pay a lot of attention to the JVM stack.

         One of things that has tripped up ML/DL in the cloud, especially as a service, is data gravity. The companies with budgets have very large datasets, and those are costly to move by definition. Any time you have to choose between moving your data to an algorithm, or an algorithm to your data, the latter is more efficient, all other things being equal. You probably also know that large organizations apply different security measures to various tiers of data. Some of it they’ll send to the cloud soon. Other datasets they don’t even want connected to the Internet. So there will be data to process on prem for a long time to come, and the tools that customers adopt there they will bring with them to the cloud.

         > Can you provide a link to support the claim that DL4J is faster than TensorFlow? If it’s a legitimate benchmark, I’ll publish it. A word of caution, though — unless DL4J is an order of magnitude faster, nobody will care.

         Here are the results and specs for the benchmarks we’ve run. https://github.com/deeplearning4j/dl4j-benchmark They’re reproducible, but obviously not conducted by an objective third party, which would be preferable. I mostly agree with you that only order of magnitude gains matter in speed, at least at the practitioner level. That said, the efficiency of deep-learning software has a huge impact on cost whether your are running on the cloud or buying your own hardware. So it’s not merely a matter of time but of budget, where gains of less than 10x can have an impact for the people with the purse strings.

         > A Python API will be great when you deliver it. You are correct that data scientists are just one part of the team. They are, however, the most important part, and machine learning projects that lack R or Python APIs will go nowhere.

         We will be using Keras as our Python API, hooking it up to the DL4J backend with py4j. Work will restart soon, with the holidays over: https://github.com/deeplearning4j/deeplearning4j/issues/2556

         > DL4J’s Hadoop integration is a strength for inference, but not a compelling strength for training. Model training workloads are moving to the cloud; frameworks that support this will thrive, others will lag behind.

         For training, we use Spark as a data access layer for fast ETL, pulling data out of HDFS. Spark is inefficient as a computation layer, but very useful for orchestrating multiple host threads over many chips.

         > A higher-level API is a good thing if you take it to the logical conclusion and abstract it completely from the back end, as Keras has done. That is the direction of deep learning.

         We have done that already with our Java and Scala APIs. If you are curious to see them, Java examples are here: https://github.com/deeplearning4j/dl4j-examples and the Scala API mirrors Keras/Torch https://github.com/deeplearning4j/scalnet

         > Every entrepreneur believes that his solution is the killer app. While I can see that Skymind has a distinct approach to deep learning, I’m not convinced that it’s a winning approach. But time will tell.

         We believe we occupy an important niche, and that niche is where deep learning meets the production environment. Many other libraries are very good tools, but in a sense they are just libraries. The makers of Tensorflow, Theano, Caffe, Keras and MxNet do not offer commercial support or sign SLAs, and given their penchant for research, they are unlikely to do so. The organizations purporting to support those libraries did not create them, and will have a harder time maintaining and extending them. In open source, the biggest committers have the most authority, and the companies working with TF have committed very little. We created DL4J and support it, and for enterprise customers seeking to de-risk tech adoption, this is an important consideration. Our model is “Cloudera for deep learning”: the classic open-source playbook of support, services and training. Google doesn’t like to get its hands messy with enterprise support.

         The other deep-learning libraries are “just libraries” in another sense as well, in that they focus on the training stage. As I’m sure you know, a lot of the work that goes into building deep learning solutions is the data pipeline. We’ve built an open-source data preprocessing library called DataVec that handles binary data well, and can normalize, clean and vectorize most major data types, like images and video, sound and voice, text, and time series. https://github.com/deeplearning4j/DataVec It’s like Trifacta’s Wrangler but open source. With DataVec, you can create persistent and reusable data pipelines for both training and inference, rather than one-off jobs each time. We created a fast tensor library for n-dimensional arrays using ND4J, JavaCPP and libnd4j to bridge the gap between the JVM, C++ and academic hardware acceleration. https://github.com/deeplearning4j/nd4j. And a model evaluation tool called Arbiter: https://github.com/deeplearning4j/Arbiter.

         Together, these libraries are more like an integrated deep learning environment (IDLE is an unfortunate acronym, but there you go…). That means Kafka connectors, and packaging these libs with DCOS and Mesos, among other things. For the inference stage, we use Lightbend’s Lagom to expose neural net models as a micro-service that autoscales elastically with traffic, and communicates via a REST API. A lot of libraries aren’t solving for all the problems that surround model training.

         Best,

         Chris

2. 

   Thomas W. Dinsmore

   January 2, 2017 at 6:45 pm

   Chris,

   I’ve updated the story based on your feedback. Let me know if you think it’s a fair assessment.

   Some additional comments:

   — 85% of all workloads may be on-premises, but the relevant measure is analytic workloads, and machine learning workloads in particular. I have not seen good statistics showing the percentage of machine learning workloads that run in the cloud, but anecdotal evidence says it’s extremely high. For many firms, model training in the cloud isn’t AN option — it’s the ONLY option.

   — Cloud is the primary platform for model training because the workloads are inherently lumpy and unpredictable. You need a huge amount of processing power for a short period, and most training projects are ad hoc and time-boxed. For most organizations, it simply does not make sense to build out an on-premises platform for model training, even with data center virtualization.

   — The data gravity and security arguments against cloud are red herrings. All data moves at some point in the process, and the cost of data movement is just one factor among many to consider. Data is more secure in the public cloud than it is on premises, and there are work practices data scientists can adopt (such as hashing and anonymization) to reduce or eliminate the risk of data breaches.

   As I note in the updated story, I’m skeptical that your strategy will succeed, but it’s a credible strategy.

   Regards,

   Thomas

   Reply
3. 

   Chris Nicholson

   January 2, 2017 at 7:01 pm

   Many thanks, Thomas. We see a lot of training happening on prem, although you may be right that the vast majority is running on the cloud. We work there, too. On prem, many lines of business have access to the resources of a Hadoop cluster managed by central IT, usually CPUs with the occasional GPU mixed in, and we let them train models on those resources by sending in the model config as a Hadoop job/JAR file. For those teams, who don’t control their company’s hardware purchases or cloud access/budget, this is their only option to experiment with new DL tools. I don’t think the data gravity argument is a red herring. MLaaS will probably succeed first with customers already generating and storing their data in the same cloud where the service is offered. Data security may be a red herring, but you’re talking about risk-averse organizations that are path dependent on some tech commitments they made long ago and have invested heavily in. I think the cloud vendors have a long way to go to reassure people about security, and there’s some exciting work going on in ML around learning from encrypted data. Unfortunately, that comes with additional computation costs.

   Reply
4. 

   Thomas W. Dinsmore

   January 2, 2017 at 7:32 pm

   Chris,

   Anything can work in the cloud, but you should be seeking out partners who can offer a managed service in the cloud, so it becomes an integral part of your strategy.

   Data science teams with clout choose their platforms. Show me a data science team that is stuck with whatever IT provides, and I’ll show you a data science team that hasn’t figured out how to deliver business value. Teams like that will never succeed with deep learning.

   If data gravity mattered, we’d be hearing about all of the people who use IBM Intelligent Miner in DB2 and Teradata Warehouse Miner in Teradata. Training models inside a data store only eliminates data movement if all of the data you need is already in the data store, which is never the case. Nobody runs deep learning on production databases — all data moves at least once, so unless you believe that the IT organization perfectly anticipates business needs when it creates its data lake/data warehouse/data mart, every analysis project requires data movement. Believe that and I have a bridge to sell you.

   You make a good point about ML in the cloud working with data sources in the cloud, but a growing volume of data sources ARE in the cloud, especially in fields like digital marketing.

   Risk averse? I work with banks and insurance companies that have shifted all or most of their model training to the cloud. If an organization is so risk-averse and platform-centric that they aren’t willing to work in the cloud, what are they doing with deep learning? They should stick with linear regression; it’s much safer.

   Re security, you don’t actually have to train the models on the encrypted data to ensure good security, but that’s a different discussion.

   Reply
5. 

   Adam Gibson

   January 2, 2017 at 8:18 pm

   Hi Thomas, Chris’ cofounder here.
   >> Anything can work in the cloud, but you should be seeking out partners who can offer a managed service in the cloud, so it becomes an integral part of your strategy.
   We will be doing that with microsoft here soon.

   >> If data gravity mattered, …

   It does to a certain extent for things like cyber security applications. While you’re saying data moves once, indeed it does: from oracle to HDFS. Actually, it’s often replicated there. Batch scoring for predictive analytics is a very common use case. Data scientists do all of their analytics in a data lake. That lake still has security controls in place.

   >> but a growing volume of data sources ARE in the cloud, especially in fields like digital marketing.

   While this is important, it will still take time for this to work. 1 thing I would counter with here is that a lot of the services doing machine learning as a service where the data sources are tend to have a lot of lock in. This is where the competition will heat up though. A multi cloud offering is actually very appealing to reduce lock in. This keeps the components neutral. We see the importance of this by having 1 of our primary deployment mechanisms be containers.

   >> I work with banks and insurance companies that have shifted all or most of their model training to the cloud. If an organization is so risk-averse and platform-centric that they aren’t willing to work in the cloud, what are they doing with deep learning? They should stick with linear regression; it’s much safer.

   Half of it is bureaucracy. A lot of our market share is actually in asia where data controls are very different from western markets. Many of these organizations are on rules engines. 1 point I’d make (and I am biased for saying this, so feel free to ignore) is that deep learning is enough of an accuracy win with representation learning that some of these organizations will actually consider deep learning solutions. This is some of what we’re seeing now.

   >> Security

   I think we’re mainly talking about data access at this point. By that I mean anything with roles and an access control list. Training on encrypted data is an interesting idea though. Hope that helps!

   Reply
   1. 

      Thomas W. Dinsmore

      January 2, 2017 at 9:19 pm

      Adam,

      Great news about your plans for a partnership with MSFT, I’ll be watching for an announcement.

      Re: data movement. As I’ve noted at least twice in this thread, model training and model scoring are two completely different things. It’s logical to implement data scoring in the data store. It’s not logical to implement model training in the data store. Model training workloads are moving to the cloud. Cyber security applications are not an exception to that rule.

      Re: data sources in the cloud. This already HAS happened. In digital marketing, it happened years ago.

      Also, you should not confuse MLaaS with the broader practice of provisioning ML in the cloud. Lots of folks are standing up their data science platforms on Amazon instances, and they could move their platform to Azure or Google or whatever tomorrow without too much effort; Linux is Linux, after all. (Qubole’s data science-as-a-service platform runs in all three.) It’s hard to argue that cloud poses a general risk of lock-in when all of the cloud vendors support open standards and open source software. Of course, if you use Oracle on Amazon, you’re stuck with Oracle.

      Reply
6. The biggest R stories from 2016 – Mubashir Qasim

   January 4, 2017 at 2:18 am

   […] ML/DL blog also features a roundup of 2016 news from the R Project, Python, and other open source data science […]

   Reply
7. The Year in Machine Learning (Part Three) | ML/DL

   January 9, 2017 at 12:23 am

   […] emergence of cloud machine learning platforms. In Part Two, I surveyed significant developments in Open Source machine learning projects, such as R, Python, Spark, Flink, H2O, TensorFlow, and […]

   Reply
8. The Year in Machine Learning (Part Four) | ML/DL

   January 16, 2017 at 12:45 am

   […] Part Two surveyed significant developments in Open Source machine learning projects, such as R, Python, Spark, Flink, H2O, TensorFlow, and […]

   Reply
9. The Year in SQL Engines | ML/DL

   February 1, 2017 at 12:33 am

   […] September, the Hivemall project entered the Apache Incubator, as I noted in Part Two of my machine learning year-end roundup. Originally […]

   Reply
10. A look back at the year in R and Microsoft – Use-R!Use-R!

    February 1, 2017 at 5:18 pm

    […] Part 2: Open source machine learning and deep learning projects, including  R, Python, Spark, H20 and Tensorflow […]

    Reply
11. A look back at the year in R and Microsoft | A bunch of data

    February 2, 2017 at 5:48 am

    […] Part 2: Open source machine learning and deep learning projects, including  R, Python, Spark, H20 and Tensorflow […]

    Reply
12. A look back at the year in R and Microsoft – Cloud Data Architect

    February 3, 2017 at 1:33 am

    […] Part 2: Open source machine learning and deep learning projects, including  R, Python, Spark, H20 and Tensorflow […]

    Reply
13. Confluence: Predictive Analytics

    May 2, 2017 at 9:10 am

    External Articles – Up your Data Game!

    A collection of articles on Predictive Analytics a

    Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.