July - August 2018 snapshots: Hyperspectral imaging, beer bottle inspection, deep learning from Microsoft, autonomous vehicles from Intel.
Machine vision aids in high-speed beer bottle inspection process
Engineers at R&D Vision (Saint-Maur-des-Fossés; France; www.rd-vision.com) have developed a machine vision system called the SpeedView system, which is used to inspect Heineken (Amsterdam, The Netherlands; www.theheinekencompany.com) beer bottles as they pass through a bottling machine at a facility in Marseille, France.
Since green Heineken beer bottles pass through the machine at a rate of 22 bottles/sec (80,000 bottles/hr), process problems that may arise are invisible to the human eye without stopping the machine, which costs both money and time.
Specializing in integrated imaging, the French test and measurement company developed the SpeedView mobile analysis system to automate the inspection of the bottles. Based on a Mako G-030 camera from Allied Vision (Stadtroda, Germany; www.alliedvision.com) which features an ams Sensors Belgium (CMOSIS; Antwerp, Belgium; www.cmosis.com) CMV300 CMOS image sensor, a 0.3 MPixel global shutter sensor with a 7.4 µm pixel size and Power over Ethernet operation. At full resolution of 644 x 484, the GigE camera captures images at a rate of 309 fps, and even higher if a narrower region of interest is used.
By reducing image detail to 300 x 200, the camera is able to operate at a speed of 1000 fps. At this speed, the SpeedView system records images for up to an hour, according to Allied Vision. Individual recordings can be triggered either manually or by a programmable logic controller (PLC) signal in the camera. Using these recordings, the maintenance team can identify and remedy the cause of failure without unnecessarily interrupting the production process or incurring downtime costs in the thousands of Euros per minute range.
SpeedView fits into a transport case and can reportedly be set up within a few minutes. In addition to the Allied Vision camera, the system features a set with three different lenses, high-intensity illumination, and a mounting bracket. For recording, playback, frame-by-frame analysis, and trigger programming, HIRIS software is used. HIRIS is a modular image acquisition solution developed by R&D Vision. Additionally, the system features an integrated monitor and a control panel.
“By uncovering problems before they affect the final product, users like Heineken can increase the quality level of their production and practically achieve a 0% failure rate,” confirmed Francois-Dimitri Mennesson, Export Manager at R&D Vision in France.
Deep learning acceleration platform preview released by Microsoft
A preview of Microsoft’s (Redmond, WA, USA; www.microsoft.com) Project Brainwave, a deep learning acceleration platform built with three layers designed for real-time artificial intelligence processing, has been released on the Azure cloud platform and on the edge.
Project Brainwave, according to Microsoft, makes Azure the fastest cloud to run real-time artificial intelligence (AI) and is now fully integrated with Azure Machine Learning. Supporting Intel FPGAs and ResNet50-based neural networks, the hardware is built with three main layers:
- A distributed systemarchitecture
- A hardware deep neural network (DNN) engine synthesized ontoFPGAs
- A compiler and runtime for low-friction deployment of trainedmodels
Mark Russinovich, chief technical officer for Microsoft’s Azure cloud computing platform, said the preview of Project Brainwave marks the start of Microsoft’s efforts to bring the power of FPGAs to customers for a variety of purposes.
“I think this is a first step in making the FPGAs more of a general-purpose platform for customers,” he said.
Microsoft is working with manufacturing solutions provider Jabil to see how Project Brainwave could be used to “quickly and accurately use AI to scan images and flag false positives,” which would free people up who manually check for defects to focus on more complex cases. The preview version of Project Brainwave that is now available offers the ability for customers to perform similar “AI-based computations in real time, instead of batching it into smaller groups of separatecomputations.”
This would be done in the TensorFlow framework for AI calculations using deep neural networks from Google (Mountain View, CA, USA; www.google.com). In addition, according to Microsoft, the company is working on building the capability to support Microsoft Cognitive Toolkit, another popular framework for deep learning. Microsoft is also offering a limited preview to bring Project Brainwave to the edge, which would enable users to take advantage of the computing speed offered by the framework in their own businesses and facilities, even if their systems aren’t connected to a network or the Internet.
“We’re making real-time AI available to customers both on the cloud and on the edge,” said Doug Burger, Distinguished Engineer, Microsoft (Pictured, holding an example of the hardware used for Project Brainwave.)
The public preview of Project Brainwave comes about five years after Burger, a former academic, first began talking about the idea of using FPGAs for more efficient computer processing. As he refined his idea, the current AI revolution kicked into full gear, according to Microsoft, which created a massive need for systems that can process the large amounts of data required for AI systems to do things such as scan documents and images for information, recognize speech, and translateconversations.
Additionally, Burger says that the framework is perfect for the demands of AI computing, and that the hardware design can evolve rapidly and be remapped to the FPGA after each improvement, “keeping pace with new discoveries and staying current with the requirements of the rapidly changing AI algorithms.”
Burger wrote in a blog post in 2017 (http://bit.ly/VSD-BRW) that the system was for “real-time AI, which means the system processes requests as fast as it receives them, with ultra-low latency,” and that “Real-time AI is becoming increasingly important as cloud infrastructures process live data streams, whether they be search queries, videos, sensor streams, or interactions with users.”
Intel and Mobileye begin testing autonomous vehicle fleet
Last year when Intel (Santa Clara, CA, USA: www.intel.com) completed its acquisition of autonomous driving software company Mobileye (Jerusalem, Israel; https://www.mobileye.com), it was announced that it would soon thereafter build a fleet of fully-autonomous vehicles for testing. Now, the first of the 100-car fleet is in operation in Jerusalem, demonstrating the capabilities of its technology and safety model.
In the testing, the cars are demonstrating the power of the Mobileye software approach and technology, to prove that the Responsibility Sensitive Safety (RSS; http://bit.ly/VSD-RSS) model increases safety, and to integrate key learnings to products and customer projects, according to Professor Amnon Shashua, senior vice president at Intel Corporation and the chief executive officer and chief technology officer of Mobileye.
The key differentiator in the system launched by Intel and Mobileye, according to Shashua, is the fact that they have targeted a vehicle that “gets from point A to point B faster, smoother and less expensively than a human-driven vehicle; can operate in any geography; and achieves a verifiable, transparent 1,000 times safety improvement over a human-driven vehicle without the need for billions of miles of validation testing on public roads.”
During the initial phase, the vehicles in the fleet will feature only cameras, in a 360° configuration. Each vehicle uses 12 cameras, with eight cameras providing long-range surround view and four cameras dedicated to parking. The goal, according to Intel, is to prove that it can “create a comprehensive end-to-end solution from processing only the camera data.”
An “end-to-end” autonomous vehicle solution, according to Shashua, is a solution consisting of a “surround view sensing state capable of detecting road users, drivable paths and the semantic meaning of traffic signs/lights; the real-time creation of HD-maps as well as the ability to localize the AV with centimeter-level accuracy; path planning (i.e., driving policy); and vehicle control.”
This “camera-only” phase is the company’s strategy for achieving what it refers to as “true redundancy” of sensing, with “true redundancy” referring to a sensing system consisting of multiple independently-engineered sensing systems, each of which can support fully autonomous driving on its own. This is in contrast to fusing raw sensor data from multiple sources together early in the process, which in practice results in a single sensing system, according to Intel. For more information on the concept of “true redundancy” view: http://bit.ly/VSD-TRD.
Radar and LiDAR will be added to the vehicles in the current weeks as a second phase of development, and then the synergies among the various sensors can be used for increasing the “comfort” of driving, suggests Intel.
Mobileye’s proprietary software is designed using artificial intelligence-based reinforcement learning techniques and was trained off-line to optimize an “assertive, smooth and human-like driving style.” To enable the system to understand the boundary where assertive driving becomes unsafe, the AI system is governed by a formal safety envelope, in the form of the RSS system.
This system is a model that formalizes the “common sense principles” of what it means to drive safely into a set of mathematical formulas that a machine can understand. This includes things like safe following/merging distances, right of way, and caution around obstructed objects. If the AI-based software proposes an action that would violate one of these principles, the RSS layer rejects the decision.
“Put simply, the AI-based driving policy is how the AV gets from point A to point B; RSS is what prevents the AV from causing dangerous situations along the way,” wrote Shashua. “RSS enables safety that can be verified within the system’s design without requiring billions of miles driven by unproven vehicles on public roads.”
The current fleet implements this view of the appropriate safety envelope, but Intel and Mobileye have shared the approach publicly and look to collaborate on an industry-led standard that is technology neutral.
On the computing end, the fleet is powered by four Mobileye EyeQ4 System on Chips (SoC). Each SoC has 2.5 Terra OP/s running at 6 W of power. The next generation, EyeQ5, targets fully autonomous vehicles, and engineering samples of this product are due later this year, according to Intel.
Looking forward, Intel’s goal—in support of its automakers—is to bring this system to series production in L4/L5 vehicles (http://bit.ly/VSD-6LV) by 2021. View the full Intel blog post here: http://bit.ly/VSD-TRD.
Hyperspectral imaging aids in fish oil gel capsule inspection
Hyperspectral imaging company Perception Park (Graz, Austria; www.perception-park.com) recently performed a successful feasibility check to see if would be possible to use hyperspectral imaging technology to detect leaked fish oil on the surface of gel capsules.
In the test, which was performed free of charge with Perception Park’s own samples, two fish oil capsules were prepared as a sample. One capsule was slightly cut, causing some fish oil to leak onto the surface, while the other was undamaged and clean. To determine if hyperspectral imaging technology could be used for this detection, Perception Park utilized a “chemical color imaging setup,” which consisted of Perception Core, Perception Studio, and an FX17 hyperspectral camera from Specim (Oulu, Finland; www.specim.fi).
Perception Core software is the run-time engine that enables the live translation of camera raw data into molecular information described by machine vision information such as a color stream according to a loaded run-time job. Perception CORE also supports the interfacing and control of hyperspectral cameras and communication with a client application by standard industrial interfaces.
Perception Studio is a PC-based software suite for hyperspectral data acquisition, exploration, and modeling, and offers a plug-and-play solution for camera integration, standardization, and calibration. Perception Studio, according to the company, is used primarily for development work with the aim to model the extraction of application-relevant information.
Specim’s FX17 hyperspectral camera is an InGaAs-based camera with a spectral range of 900 – 1700 nm and free wavelength selection from 224 bands within the camera coverage. At full frame, the 150 x 85 x 75 mm camera can reach 670 fps, and with 4 bands selected, can reach 15,000 fps. The camera is available with a Camera Link or GigE Vision interface and comes with factory-loaded unified wavelength calibration that produces unit-to-unit compatible data output. The camera also has a 38° field of view, a spectral full width at half maximum (FWHM) of 8 nm, a spatial sampling of 640 pixels, integrated shutter, and a camera signal-to-noise ratio of 1000:1 (peak).
After scanning the gel capsule samples with the hyperspectral camera, the datasets were merged to one set. Representative spectra sets were selected in the above image with the blue-shaded capsules. In the image, red represents spectra of the capsule with fish oil on the surface, green represents the clean capsule, and blue represents the spectra of the background. The spectral differences between the clean capsule and the capsule with fish oil on it is quite big, and is obvious in the spectral view, noted Perception Park.
When applying its chemical color imaging methodology “Correlate,” Perception Park was able to show that it can be clearly seen that parts with the fish oil film on the surface appear in red, while the clean parts are green.
Perception Park noted that even in the green images, some red appeared despite the capsule being clean. After applying various pre-processing methods in an attempt to improve the results, the borders of the green (clean) capsules still appeared red. Perception Park says that this is based on reflections and is potentially something that could be improved with a better designed setup.
Perception Park’s Case Discrimination methodology was also applied to determine if it might be possible to identify a classifier for the parts that had oil film on the surface. Using this method, it was possible to see the oil film, but nonetheless, the reflections on the borders appeared again.
During the process, the Perception Park team also noticed an additional defect. In the capsule that was cut, some air made it inside, which appeared in some of the images. By applying pre-processing techniques, the defect became even more visible. However, by applying the “Correlate” or “Case Discrimination” methodology, the air bubble could be clearly identified.
In conclusion, Perception Park found that—using its chemical color imaging methodology—it was possible to identify fish oil on the surface of the capsule, as well as air bubbles inside of the capsule. These results—according to the company—are valid for the tested capsules, tested under the used measurement conditions, and for other capsules, this must be verified.
View more information on Perception Park’s chemical color imaging technology: http://bit.ly/VSD-PERPARK.