Practical AI-Powered Real-Time Analytics for Manufacturing: Lessons Learned From Design to Deployment
Human beings are not suited to perform repetitive and tedious tasks, no matter how important, skillful, or critical they might be. Imagine watching hours of surveillance footage with extremely rare instances of anomalous activity. Or being a doctor who must manually review volumes of medical imaging to identify tumorous cells. Similarly, in manufacturing, inspecting products for defects is exceedingly mundane when only one in a million needs to be pulled. As human beings, we are curious at our core, and our work should play to that strength.
Fortunately, today’s analytics and AI capabilities are perfect for such tasks.
AI has had a resurgence in the last five years because new machine learning techniques are now solving three particular problems in ways that are similar to humans and with comparable accuracy: playing games (reinforcement learning), reading and understanding natural language (text analytics), and analyzing images and videos (computer vision with convolutional neural networks or CNNs).
A leader from an organization’s advanced manufacturing division reached out to my team because he had heard about computer vision and had a hunch that some of the problems he encounters in his plant could be solved easily with modern techniques that use deep learning. We found out that we could solve the problem—not easily, perhaps, but with sustained creative effort.
The organization’s plant is an intricate system of conveyor belts that weaves and snakes throughout a large manufacturing floor, transforming raw materials through dozens of stations into a final product ready for shipping. As the forming product is handed off from one station to the next, it can get stuck. Like a car slamming its brakes on a busy highway, this can lead to a pile-up. Furthermore, in the same way that traffic behind the accident will get slowed down or come to a grinding halt, the production line upstream may have to be shut down if the jam cannot be cleared quickly. While such events are relatively rare, they are costly. Products involved in the collision will likely have to be scrapped, and manual intervention is required to remove the damaged products and clear the jam. Shutting down the production line causes a huge loss in daily yield.
My team was tasked with identifying and developing the right analytical solutions. The company sent us a few minutes’ worth of sample footage demonstrating normal operation versus a collision event. Given the problem and some sample data, we could begin the modeling process. I’ll describe three approaches we took in increasing order of complexity. The more complex models achieve higher robustness at the expense of taking more time to build and having slightly slower real-time performance.
Our sample videos were centered on a station in which a product comes in, has a sticker applied and then proceeds. When running smoothly, it is a regular, repetitive process. The first and simplest solution was a model that treats every image frame as a vector and computes the distance to a reference frame. Applying this model to the video footage results in a periodic signal much like a sinusoidal wave; a collision event breaks the wave and turns it into a constant flat line. This pixel level inter-frame distance method is ultra-light, simple and fast, but not robust. It has limited value in that it will only work in the specific circumstances that resemble something like the sample footage. For example, the periodicity could also break due to inactivity instead of a collision event. To build a more robust model, we turned to deep learning.
