Hackathon winner creates portable AI inspection demo kit

Sergio Velmay, winner of the 20/20 Vision Hack hackathon sponsored by ADLINK (San Jose, CA, USA; www.adlinktech.com) and Intel (Santa Clara, CA, USA; www.intel.com), developed an industrial machine vision demo kit that uses artificial intelligence (AI) and 3D imaging to assist with parts preparation and verification of the correct steps for assembly applications, alerting the employee of errors at any stage of an assembly.

He developed the system using the ADLINK Vizi-AI kit, developed for prototyping and for deployment on ADLINK hardware. The Vizi-AI kit includes an Intel Atom-based SMARC computer module with Intel Movidius Myriad X VPU, an Intel Neural Computer Stick 2, and the ADLINK Edge software suite. The prototype for Velmay’s demo kit used an Intel RealSense D435 camera.

Velmay created and trained seven AI models, including object detection, multilabel classification, and image classification models using the Microsoft (Redmond, WA, USA; www.microsoft.com) Azure Custom Vision service and then exported the models to the OpenVINO inference engine. The Windows and Ubuntu Linux operating systems were used to test the machine vision software Velmay developed.

Related: Glass vial manufacturer's vision system allows 100% inspection

The demo kit uses a set of plastic cylinders, rectangles, gears, and other 3D printed shapes that represent different kinds of parts for demonstration purposes. The system captures “good” images of the correct number of correct parts for a specific task and uses the images for model training.

Once trained, the model draws a green bounding box on the software’s GUI to indicate correct parts and a red bounding box to indicate incorrect parts. When a task requires a specific number of the same part, the GUI also displays an “X/Y” notation over each part, where X counts the part and Y is the total number of parts required.

Correct notations appear in green, i.e. if an assembly requires three of the same type of part, the notations “1/3,” “2/3,” and “3/3” are displayed on the parts. A notation in red, for example “4/3,“ is displayed over extraneous parts of the same type in the image. To train the system to identify the proper steps in an assembly, the assembly is conducted several times and captured on video. Still images, extracted from the video, are organized and cataloged to create datasets for AI model training.

The trained system can then track the assembly in real time, marking correct completions of an assembly step on the GUI with a green checkmark and marking incorrect steps with an X.

The system can also detect whether an employee follows safety protocols during an assembly. If an application requires that an assembler wear gloves, and the images from the training dataset feature hands wearing gloves, but the system detects a bare hand when monitoring an assembly, an alert is issued on the GUI to the employee.

While the lightweight and easily transportable demo kit is designed for delivering live demonstrations of AI inspection for potential customers, Velmay’s employer, Bravent (Madrid, Spain; www.bravent.net) deployed an industrial vision solution utilizing the same principles as the demo kit at a John Deere (Moline, IL USA: www.deere.com) factory in Getafe, Madrid, Spain.