Identifying and addressing the limitations of deep learning software

Conventional machine vision inspection methods define defect criteria—such as brightness and radius in a formal pattern—which requires the manual setting of a threshold. Because of this, quantifying every potential defect on an object in an image isnecessary.

With deep learning software, example-based algorithms are trained with images of both acceptable and unacceptable products, unlike rule-based algorithms that require manual setting of defect criteria. Through deep learning algorithms the software automatically learns and analyzes defect criteria and differentiates between normal and defective object images.

Figure 1: SuaKIT provides segmentation, classification and detection, and four different deep learning architectures.

However, training a deep learning model requires large amounts of image data showing both acceptable and unacceptable parts. While obtaining images of good parts is relatively easy, images of defects are much rarer, making it difficult to quantify defect criteria.

In order to address this lack of defect data, engineers at SUALAB (Seoul, South Korea; www.sualab.com) rely on data augmentation techniques in the “One Class Learning” function of the company’s SuaKIT deep learning software.

Related:What’s the difference between machine learning and deep learning?

“Data augmentation transforms the image by decolorizing, flipping, or rotating,” explains SUALAB Marketing Manager Hanjun Kim. “It also enables the creation of images with varying brightness, which reduces problems caused by optical condition changes.”

Figure 2: Using the “Segmentation” function, it is possible to accurately detect a defect’s position/area/shape.

The black-box nature of deep-learning algorithms makes it difficult to track down issues, leaving users at a loss when it comes to understanding why certain parts fail.

SuaKIT’s visual debugger function helps users locate the section in which the deep learning algorithm identified a defect. The visual debugger also identifies the criteria that were used by the deep learning algorithms.

Shrinking product life cycles and mixed model processing also challenge real-world deep learning applications in manufacturing. Each time a new type of defect occurs or a product variation is introduced, users must develop a new deep learning model, consuming time and resources. SuaKit overcomes such problems with a type of transfer learning, dubbed “domain adaptation,” that utilizes pre-labeled data in relevant source domains to execute new tasks. (See: http://bit.ly/VSD-DA). By utilizing resources such as image data and a model of existing products, the reduction of resources needed for defect detection in a new product is possible, Kim explained.

Figure 3: Using the “Detection” function, detection/classification of various types of objects in a single image is possible.

Implementation of deep learning software—SuaKIT or others—replaces the need for human visual inspection and allows the relocation and deployment of people to other manufacturing processes or tasks within a facility, improving productivity. For example, printed circuit boards (PCB) have a complicated circuit pattern. For the inspection of defects on PCBs, conventional machine vision inspection methods such as pattern matching can find the difference between the master image and captured product images, but deep learning techniques are more effective.

Related: Deep learning promises automotive inspection improvements

“At one facility in which SuaKIT was deployed, the identification of defected products using conventional machine vision inspection methods resulted in an accuracy rate below 40%. The over-detection rate was also high, and at least three manual inspectors were required to inspect the products, even after the automated inspection was done,” says Kim. “SuaKIT improved accuracy to 97.4% and the over detection rate was drastically reduced in that application, reducing the number of images that needed review.”

Deep learning software offers an efficient alternative over “traditional,” rule-based machine vision techniques, but there are still limitations involved. When these limitations are overcome, deep learning really offers its users or potential users promising improvements in their processes.