Phil Lapsley, co-founder of embedded-AI consulting firm BDTI and a Vice President of the Edge AI and Vision Alliance
It can be a tricky question, one with multiple answers—some right, some wrong! And, it’s often challenging to think through. At the Edge AI and Vision Alliance, we use the funny acronym BLERP (bandwidth, latency, economics, reliability, and privacy) to help decide where processing should take place. Following are five of the most important factors to consider when making edge-cloud tradeoffs.
Bandwidth
Obviously, vision processing in the cloud requires a network connection over which to send your images. Depending on your application, your bandwidth requirements could be a trickle (say you’re sending one small image of a dumpster to determine whether it’s full or not) or a flood (say you’re monitoring hundreds or even thousands of cameras in real time at a grocery store). Whether you can get a network connection that will accommodate your requirements is another story. In some cases, you may not have the option for an Internet connection at all (e.g., a wildlife camera out in the middle of nowhere), and in others, you might have a reasonably reliable, low-cost pipe to the cloud (think of a consumer’s WiFi-connected doorbell). Balancing your requirements and available network capacity is critical. Edge systems excel when bandwidth needs are high or available bandwidth is low (or non-existent).
Latency
Some applications require instant answers. Think about self-driving cars, for example. If your cloud-based image processing system takes several seconds to recognize an object, you might have already run over a pedestrian and be halfway down the block by the time you realize you should have stopped. But. other systems can tolerate much longer latency. A camera based system to recognize what food items you have in a refrigerator, for example, might be happy taking tens of seconds, or even minutes, to realize you’ve put a carton of milk in the fridge. The lower your latency requirements, the more pressure to do your processing at the edge.
Economics
The best things in life may be free, but, sadly, bandwidth and computing aren’t among them. If you’ve ever paid for cloud computing, you know how quickly those costs can add up. Similarly, just look at your cable or cell phone bill to get an idea of how expensive bandwidth can be. Edge processing can save you money in bandwidth charges (the more you do on device, the less you do in the cloud), but adding a more capable processor to your product costs money. For many products, a key business insight is understanding who is paying for compute and bandwidth, what their willingness to pay is, and whether they’re already paying for them in some form. For example, if you’re making a consumer appliance that uses a homeowner’s existing Internet, well, they’re already paying for that network connection; from your perspective, bandwidth is free. Conversely, if your customer is willing to pay for a more capable device (or better still already has such a device, like maybe a high-end mobile phone), that can save you money by reducing your cloud computing costs by offloading processing to the edge.
Reliability
Is it important that your system continue to function if there’s a network outage? A facial-recognition-based home door lock, for example, probably needs local processing (at least as a fallback!) if the homeowner’s WiFi network goes down. In general, the more critical reliability is, the greater the need for edge processing.
Privacy
To paraphrase the Vegas slogan, “What happens at the edge stays at the edge.” Stated differently: if you’re not sending images or video up to the cloud for processing, there’s nothing in the cloud for the bad guys to steal, nor for you to accidentally leak via a misconfigured AWS S3 bucket. Not only does this reduce your liability, it’s also a great selling point for your customers that care about privacy.
Where you do your vision processing is a balancing act and depends on a variety of factors: bandwidth, latency, economics, reliability, and privacy chief among them. Thinking through how these five factors relate to your application can help determine whether processing should take place at the edge, in the cloud, or some combination of the two. The ultimate answer depends on the specific requirements and constraints of your particular application, of course, but we hope that a BLERP analysis can help you reach an answer that’s right for you.
PHIL LAPSLEY is a co-founder of embedded-AI consulting firm BDTI and a Vice President of the Edge AI and Vision Alliance, a 100+ Member industry association dedicated to inspiring and empowering innovators to create systems that perceive and understand. He is also an organizer of the Embedded Vision Summit, which will be held in Santa Clara, California, May 22-25, 2023.