Fraunhofer Institute Researchers Develop AI assisted Inspection System

Researchers at the Fraunhofer Institute for Material and Beam Technology IWS (Dresden, Germany) have developed SURFinpro, a new AI-assisted automated inspection system specifically designed for roll-to-roll processing.

Roll-to-roll processing refers to a manufacturing process that involves starting with a roll of base material, such as foil, flexible glass or plastic, and continuously applying very thin multiple layers to that base material to create an output roll. Once the rolls of material have been processed, they can be cut to their appropriate finished size.  

This process is often used to manufacture products such as thin film transistors and semiconductors, as it can reduce production costs. In developing this inspection solution, the Fraunhofer team worked with a client who manufactures sheets of organic solar cells, which are very expensive to make, notes Christopher Taudt, group manager for surface metrology.

A roll-to-roll manufacturing line is typically running a roll of sheet materials at 4-6 m per second, through a large-scale industrial production environment, Taudt says. The rolls ultimately contain 200-300 m of this material. Until recently, nearly all such inspections were conducted by human inspectors, which is not only slower and more labor intensive but also less thorough, which means more material is wasted.

The  client’s production line typically uses rolls of material that are 2-3 m wide, Taudt says. The Fraunhofer team’s goal was to develop an automated inspection system that can “see” the entire surface of the material as it moves along the production line in much greater detail than a human inspector, and detect, identify, and accurately label minute defects such as wrinkles, particles, bubbles, and other such irregularities earlier in the production process—and pinpoint exactly where those defects are located, Taudt says. These are defects that a human inspector can miss or possibly cannot even see.

“The goal is to reduce the amount of material you have to discard,” Taudt says. “By effectively pinpointing defects in real time and in smaller areas, you don’t have to discard as much of the material.”

Building a solution

The team set up four IDS Imaging Development Systems GmbH (Obersulm, Germany) U3-3000SE cameras and four z-Laser (Freiburg, Germany) ZX-20 lasers at the end of the production line. Each camera is set at a 90°angle above the surface of the sheet material; each laser is positioned slightly to one side of a camera and aimed at the sheet material at a slightly steeper angle. The laser projects a beam to the material roll, which bounces off the material. The cameras then capture images, frame by frame, as the laser light hits the surface of the material.

“Imagine looking at the surface of your cell phone, for example,” Taudt explains. “If you look directly at the screen, you will see only your reflection in the dark screen. But if you tilt it slightly, allowing the light to hit it, you start to see every smudge, every fingerprint.”

Each camera and laser cover a separate section, or “slice” of the sheet material, providing four different views and transmitting the data to individual NVIDIA (Santa Clara, CA, USA) Jetson computers connected to each camera; each camera is running at 200-300 fps. A larger industrial PC serves as a main control computer, connected by cable to the smaller computers, and coordinates the data from the smaller computers to “stitch” together one continuous view of the material’s surface. If a defect is present, it will appear as a 3D image when the laser light hits it.

The central computer uses AI software, written by the team, to stitch the frames into the one continuous image, identify potential defects and aberrations, label them, outline their shape, and pinpoint their exact location on the sheet material, all in real time. And, instead of multiple human inspectors, the central computer can be monitored by one employee. The entire system is set up and operated onsite.

Future plans

SURFinpro is unlikely to become an “out-of-the-box” solution any time soon, as most of the companies who will likely need it are fairly new companies working with manufacturing environments and processes unique to their businesses. Nonetheless, while each SURFinpro system must be configured to individual customer needs, the team will continue to work to adapt SURFinpro for other industries, product materials such as silicon, and larger material rolls, Taudt says.