Bringing 20/20 Vision to the Edge with AI Inferencing

By Dana MCarty, VP Inference Sales, Marketing, and Applications for Flex Logix

It’s an exciting time to be a part of the rapidly growing AI industry where the technology capabilities are available for mainstream products at the edge. Manufacturers in a wide range of industries—from medical, industrial, robotics, security, retail, and agriculture—are learning how to add 20/20 vision to their products by using AI inferencing capabilities and pre-trained models. In this article, we’ll explain what inferencing is and then showcase some of the applications where it’s gaining the most significant traction.

What is AI inferencing?

We’ve all heard the term machine learning. This is the process of training models by using massive amounts of data. The more data used for training, the more intelligent the models become. Over time, the models even make predictions and if the predictions are correct, the model becomes even more accurate. This training can occur for quite a long period of time—even months—and the data being analyzed are often massive.

The result of all this compute-intensive training, however, is a program that can be run on a much smaller device at the edge to do inferencing, or the process by which the trained model is used to produce a predicted result.

There is demand today for edge vision solutions that bring AI capabilities to a wide range of products and services—yet many companies lack the expertise or data science know-how to develop and train models and then integrate them with existing AI accelerators. To solve this problem, many companies are turning to pre-trained models that they can easily integrate into their devices to make them smarter and given them better vision. These models have been trained to perform the most common object detection capabilities such as hard-hat detection, people counting, face mask detection, and license plate recognition. In fact, they have also been optimized to work with an inference accelerator, which is basically the chip that performs the inferencing function.

The ability to do inferencing in edge products can be transformational for both the company developing the device and the consumer benefitting from its use. Below are a few use cases:

• Hard-hat detection for plant safety applications: AI detection can ensure that employees are wearing hard hats in dangerous locations and if no hat is detected, an alert can go off or the machinery can be shut down.
• Counting customers in crowded environments: This example involves the ability to count customers in crowded settings. In the retail space, it can enable capabilities like as pose estimation, which uses computer vision technology to detect and analyze human posture. The data gained from this analysis could enable brick-and-mortar retailers to better understand human behavior and foot traffic in their stores, enabling them to set up the store in a way that maximizes their retail sales and customer satisfaction. 
• Edge servers in factories, hospitals, financial institutions, and other enterprises: For example, in the industrial space, AI can be used to help manage inventories, detect, or even predict defects before they happen.
• Cancer detection: Machine learning approaches have been demonstrated to detect cancer by processing digital X-rays. The process for doing this is to develop a machine learning model designed to process X-ray images typically using semantic segmentation algorithms trained to detect cancer. In the training phase, images of cancers, identified by expert radiologists, are used to train the network to understand what is not cancer, what is cancer, and what different types of cancers look like. The more machine learning models are trained, the better they get at maximizing correct diagnoses and minimizing incorrect diagnoses. What this means is that machine learning depends on both smart model design and, just as importantly, on a massive quantity (like hundreds of thousands to millions) of well-curated data examples in which the cancer has been expertly identified. The inferencing application would then be applied as a second set of eyes to help radiologists in reviewing X-rays and highlighting areas of interest for the radiologist to focus on.
• Smart shopping carts: Several companies are developing and deploying smart shopping systems that can recognize products, not by their UPC barcode but by the visual appearance of the package itself. This capability allows shoppers to just drop products in the cart or place them on the checkout system without needing to look for the UPC code and scanning it with the UPC laser scanner. This technology makes the shopping process more accurate, quicker, and more convenient. 
• Agriculture: AI inferencing is being used to more effectively track and monitor herds of cows or other livestock.

How Do I Speed the Development of AI Products and Services?

Companies are in a race to incorporate AI capabilities into their products because it makes them smarter, more useful, and more effective. Consumers have an insatiable demand for their devices to be “cooler” and provide new capabilities—and the products that can provide this are the ones that will win the market share. Thus, if you are a company looking at AI inferencing, there are several key things you can do to speed the development of these new intelligent devices.

• Choose the right AI inference hardware. All too often companies get to the finish line with their products only to find out that the performance is not what they expected. Good inferencing chips are now being architected so that they can move data through them very quickly, which means they have to process that data very fast and move it in and out of memory very quickly. What really matters is what throughput an inferencing engine can deliver for a model, image size, and power or inference/Watt. These are the measurements of how well a solution will perform.
• Look for turnkey solutions that make it easier to integrate AI. Bringing AI products to market is a complex task that often requires data science expertise, which many companies don't have. Incorporating pretrained models can take some of the complexity out of the process.  In addition, there are now hardware and software-ready mini-ITX x86 systems available that are designed to help customers quickly and easily customize, build and deploy edge and embedded AI systems.

The Recurring Revenue Advantage

By incorporating AI inference, not only can companies provide consumers with cutting edge new capabilities, but they can also provide the product via an everything-as-a-service (XaaS) payment model. In today’s digital world, companies that can capture recurring revenue streams are winning in the marketplace. Because AI models are always evolving and improving, manufacturers have future opportunities to provide improved solutions over time. This not only provides a recurring revenue stream, but it’s also another opportunity to continually engage with customers to strengthen that relationship.

Are your products seeing 20/20?

AI inferencing has the potential to transform everyday products around the things they need to accomplish. Whether that is a simple people counting application or advanced medical imaging that is used to help detect cancer, the opportunities are endless and exciting.