Medical Image Analysis with Deep Learning
In my last article, I wrote a general overview of Big Data Analytics in Healthcare. Researchers at IBM researchers estimate that medical images currently account for at least 90 percent of all medical data, making it the largest data source in the healthcare industry. This becomes an overwhelming amount on a human scale, when you consider that radiologists in some hospital emergency rooms are presented with thousands of images daily which are not analyzed properly. Due to the large facets of healthcare processes, it is not uncommon to create datasets with over 10,000 or more features even after dimensionality reduction. Deep learning has shown that being able to train large models on vasts amount of data can drastically improve model performance. However, consider the problem of training a deep network with millions, or even billions of parameters. In this article, we discuss how we can achieve this without waiting for days, or even multiple weeks with parallelization as well as use of DICOM format. In the end, we will go through a case study on Nvidia’s achieved this task with through Clara Medical Imaging Platform.
Since most high-performance deep learning implementations are single-node, we walk through two paradigms fromJoeri Hermans’thesis and explain how we can use Spark and a cluster of machines to improve deep learning pipelines with TensorFlow.
Apache Spark is an amazing framework for distributing computations in a cluster in an easy and declarative way. It is becoming a standard across industries. Tensorflow uses a dataflow graph to represent the computation dependencies among individual operations. Spark’s optimization power lies into the use of Resilient Distributed Datasets or RDD. Using Tensorflow in a Distributed fashion with Spark clusters allows us to compute portions of the graph in different processes, and thus on different servers. The computational graph, containing our neural network is passed as a map function. The key ingredients of this approach consists of Central Parameter Server and the workers. On this part of the dataset, each worker applies the regular deep learning optimisation, such as mini-batch, and computes a gradient. Upon computation of the gradient, the worker commits its result to the central Parameter Server.
In model parallelism, a single model is distributed over multiple machines. The performance benefits of distributing a deep network across multiple machines mainly depends on the structure of the model. Models with a large number of parameters typically benefit from access to more CPU cores and memory, thus, parallelizing a large model produces a significant performance increase, and thereby reducing the training time.
