TensorFlow and Caffe are each deep learning frameworks that deliver high-performance multi-GPU accelerated training. Deep learning frameworks offer initial building blocks for the design, training and validation of deep neural networks and training for image, speech and text based data, via a high level programming interface.
Machine Learning and Predictive Analytics are now used in a wide cross-section of business operations; taking this further and using the correct deep learning framework in conjunction with Artificial Intelligence will amplify the scale of what can be achieved and obtained.
There are multiple deep learning software options available, including roNNie.ai, BigDL, Deeplearning4, Chainer, Darknet, Microsoft Cognitive Toolkit, Apache MXNet, TensorFlow and Caffe.
Intro to TensorFlow vs. Caffe
Beginners tend to favor TensorFlow because of its programmatic approach to network creation. Caffe has been panned for its convoluted code and disorganized documentation. In this post, we will look in depth at both frameworks and consider their pros and cons.
TensorFlow
TensorFlow is an open-source ML library for research and production. It was developed at Google by the Google Brain team (initially purely for internal use) and since its release under the Apache 2.0 open source license in late 2015, has built an impressive community around it. Google’s intention in releasing it was exactly that: to accelerate the evolution of AI through the support of the larger community. As we’ve discussed elsewhere,
TensorFlow had reached 21,000+ commits from outside contributors by late last year. It continues to be in rapid development and Google continues to use TensorFlow internally for research and production. Google also offers the community a series of online training tutorials via its YouTube Channel, and a video series called TensorFlow meets in which Google’s Developer Advocate, Laurence Moroney, interviews people in the business community about their work with TensorFlow.
TensorFlow is a multi-platform framework, running on Linux, macOS and Windows. Its architecture runs on various CPUs, GPUs and TPUs, including mobile and embedded platforms; and from desktops to server clusters, with optional modular CUDA and SYCL extensions for graphics computing.
Its computations are expressed as stateful dataflow graphs.
TensorFlow is available in the TensorFlow GitHub org, as are an ecosystem of other repos, including models, serving, TensorBoard, Project Magenta, and others. TensorFlow has APIs in multiple languages (Python, C++, Java, and Go) available; in addition to other bindings the wider community has developed, including C#, Haskell, Julia, Ruby, Rust, and Scala.
Google has been developing a host of other programs around the core TensorFlow solution, including its Tensor processing unit (TPU), an application-specific hardware chip, or integrated circuit, built entirely for machine learning and specifically geared toward TensorFlow. It is a programmable AI accelerator, which offers low-precision arithmetic at a high throughput, geared at using or running models as opposed to training them. Earlier this year, Google announced it was making TPUs available in beta on its Google Cloud Platform.
Caffe
Caffe is a deep learning framework that was developed by Yangqing Jia during his PhD at UC Berkeley along with Berkeley AI Research (BAIR) and community contributors. It is now in version 2, released under the BSD 2-Clause license. It is a highly popular deep learning framework. Its name stands for Convolution Architecture for Feature Extraction.
It offers several key benefits:
Expressive architecture. Rather than hard-coding, models and optimization are defined by configuration, which encourages application and innovation. By setting a single flag to train on a GPU machine, you can MSwitch between CPU and GPU then deploy to commodity clusters or mobile devices.
Extensible code, which has fostered active development within the community. Across the first year of Caffe, it was forked by more than 1,000 developers and underwent many significant changes contributed back. The framework currently tracks the state-of-the-art in both code and models.
Community. Caffe powers numerous academic research projects (including at its source, Berkeley), startups, and large-scale industrial applications in vision, speech, and multimedia. Community resources/access points can be found at caffe-users group and Github.
Speed. Caffe is capable of processing over 60M images every day with a single NVIDIA K40 GPU*. That translates to 1 ms/image for inference and 4 ms/image for learning, The more recent library versions and hardware are feven faster. Caffe is among the fastest convnet implementations available.
Caffe sets itself apart from other deep learning frameworks through its modularity and the fact that it is designed for scale. It has been described as an “un-framework” due to its flexibility and modularity.
Caffe2 was released in 2017 by Facebook as the successor to the original Caffe. However, Caffe continues to be available as a separate framework.
Caffe2 improves Caffe 1.0 in several ways:
- Improved support for large-scale distributed training
- mobile deployment
- new hardware support (as well as CPU and CUDA)
- flexibility for future directions, including quantized computation
- stress tested by the huge scale of Facebook applications
The first Caffe framework was especially useful for large-scale product use cases due to its unparalleled performance and well tested C++ codebase. Caffe has various design choices inherited from its initial use case: conventional CNN applications. As new computation patterns have emerged, however, in particular, distributed computation, mobile, reduced precision computation, and some non-vision use cases, its design has shown several limitations.
TensorFlow vs. Caffe
Aaron Schumacher, senior data scientist for Deep Learning Analytics, believes that TensorFlow beats out the Caffe library in multiple significant ways. According to Schumacher (who made the argument at the OSCON open source conference in Austin, Texas late last year), TensorFlow is easier to deploy and enjoys a more flexible API.
Deployment
Caffe does not have a straightforward deployment model for installation. It has to be developed from source. TensorFlow, by contrast, has the Python pip package manager.
APIs and Life-Cycle Management
Also, Caffe does not offer a “high-level API for building models” says Schumacher, something that TensorFlow does offer. This can make Caffe harder to experiment with, as with a higher-level API, you can experiment quickly; however, with a low-level API, it can be harder to configure things in a non-standard way. Schumacher believes that Caffe’s approach of a “middle-to-low API” offers little high-level support and limited deep configurability. “It’s not always as low as you want it to get to change things, and if you want to go higher, you have to build your own,” he said.
Schumacher explains that when Deep Learning Analytics had to build a wrapper for the PyCaffe interface to make it more straightforward to use, Tensorflow has a more suitable interface for Python (increasingly the language of choice for data scientists). Caffe’s interface tends to be more C++ centric, meaning that users need to do more tasks manually, such as creating configuration files and planting them on disk for every new machine learning job.
GPUs
As Caffe’s GPU support currently offers no tools for Python, all training must be performed via a C++ based command line interface. Caffe supports only a single style of multi-GPU configuration. Meanwhile, in TensorFlow, all the necessary adjustments are performed via the tf.device(), in which one designates the use of GPUs. No further documentation is necessary, nor are further changes to the API. TensorFlow also offers a more flexible architecture as you can run two copies of a model on two GPUs, or a single large model over two GPUs.
Support for Multiple Machines
In TensorFlow, support for multiple machines is a given. With Caffee, the MPI library must be used. It was initially built for breaking apart applications on massive multi-node supercomputers, which can mean that implementing an MPI version, which is running a Caffe training process is not always straightforward. By setting tf. Device to the number of machines a job can run on, TensorFlow offers a straightforward solution to configuring jobs for multi-node tasks.
Conclusion
The deep learning landscape is constantly in flux. The first mainstream deep learning framework was called Theano and was created and maintained by MILA and led by Yoshua Bengio, a deep learning pioneer. However, work on Theano came to a halt in September of last year after one final version was released. It was not a surprising decision as so many other deep learning frameworks had sprung into being, usually open source Python and frequently backed by one of the big tech companies like Google and Microsoft.
According to Indra den Bakker, co-founder @20tree.ai, Google’s TensorFlow appears to be the most popular deep learning framework available (he based this on GitHub stars and forks, and Stack Overflow activity). However, it does not entirely dominate the space. Other frameworks have passionate user bases as well, including Caffe. The original Caffe framework remains very popular for its community, its computer vision developments and its Model Zoo, essentially a selection of pre-trained models.
There is a wide array of different machine learning frameworks. Finding the right one for your business needs can be a juggling act. A deeper comparison between a host of them can be found here.