How Developers Can Benefit From Intel’s Optimization Of TensorFlow
Abstraction is a common trait amongst the now widely used machine learning libraries or frameworks. Dusting off the nitty-gritty details under the rug and concentrating on implementing algorithms with more ease is what any data scientist would like to get their hands on.
TensorFlow rose into prominence for the very same reason — abstraction.
Now with its latest library TensorFlow Graphics, it aims to address key computer vision challenges by incorporating the knowledge from graphics in the images, which in turn result in robust neural network architectures.
TensorFlow is a widely-used Machine Learning framework in the deep learning arena, demanding efficient utilization of computational resources.
While efforts are being made to make Deep Learning more accessible through platforms like TensorFlow, companies like Intel® are tweaking TensorFlow to extract high performance.
The TensorFlow framework has been optimized1 using Intel® Math Kernel Library for Deep Neural Networks (Intel® MKL-DNN) primitives, a popular performance library for deep learning applications.
TensorFlow follows a data flow paradigm for computations and it is a good model for doing parallelism.
TensorFlow is designed to be flexible, scalable and deployable. For example, new developers can get a quick start as this framework hides all complex distributed training and developers do not need to understand low level APIs.
Deep Learning inference can be done with two different strategies, each with different performance measurements and recommendations. The first is Max Throughput (MxT) and aims to process as many images per second, passing in batches of size > 1. For Max Throughput, best performance is achieved by exercising all the physical cores on a socket.
Real-time Inference (RTI) is an altogether different regime where we typically want to process a single image as fast as possible. Here the aim is to avoid penalties from excessive thread launching and orchestration between concurrent processes. The strategy is to confine and execute quickly. The following best known methods (BKMs) differ where noted with MxT RTI.
Most of the success of modern AI, especially deep learning algorithms, is due to its impressive results in image classification where near human-level has been observed.
To explain the advantages that Intel® brings to the table, let’s take the example of document classification.
This capability can be used for document authentication which is a common task when opening a banking account, performing checks-in at the airport or showing a driver’s license to a police officer. Today most document authentication tasks are done by humans, but AI is showing to be effective and is being increasingly employed for this activity.
Any typical document classification task would contain the following steps:
The Binary and Multiclass Classifier used in the experiments of this paper were implemented using Keras* high-level API available on TensorFlow.
So, on the CPU, when Intel® Distribution for Python* along with Intel® Optimization for TensorFlow was used, around 70% to 80% improvement was observed only by installing Intel® Optimization for TensorFlow.
This is done by setting Number of Threads to Execute in Parallel for Inter and Intra Operations in TensorFlow and Keras.
