Delivering High-Resolution Machine Vision for Enterprise Drone Applications

By Cedric Marta, vice chair of the MIPI Camera Working Group

From among the tremendous scope of markets enveloped by the IoT, the enterprise drone segment presents distinct opportunities and challenges, especially concerning machine vision.

High-resolution machine vision is essential for many of the use cases in this space, such as remote inspection of power lines and telecommunications towers, monitoring crops and livestock, performing industrial site inspections, and enabling law enforcement reconnaissance.

Machine vision also helps unlock the highest-value use cases in the enterprise drone market in a much more fundamental way: It enables the operation of drones beyond the visual line of sight (BVLOS)—i.e., for drones to navigate themselves autonomously, well beyond the visual range of the drone operator, connected via wide-area cellular and satellite networks. BVLOS operation greatly expands the application horizon for enterprise drones.

Addressing Varied Challenges for BVLOS Operation

Drone experts have highlighted the development of detect, sense, and avoid (DSA) systems as critical for BVLOS drone operation. These systems reduce the risk of a mid-air collision by ensuring that drones can operate safely in a variety of environments and avoid obstacles, including other aircraft.

These solutions require integrating high-resolution cameras, LiDAR, and radar. The data from DSA sensors are fused, using cellular connectivity, with data from other unmanned aerial vehicles (UAVs) or ground-based systems so that the drone has a good understanding of its environment and can safely maneuver within it. Designing and integrating sensors and their associated components within necessary performance, size, weight, and power constraints is no small challenge, however.

This is where interface specifications from the MIPI Alliance (Piscataway, NJ, USA; www.mipi.org) can significantly enhance ongoing development of the enterprise drone market. MIPI specifications define standardized connections between the embedded electronic components such as cameras, displays, sensors, and communication modules within electronic devices, which greatly simplify the design and integration process. MIPI specifications are also predicated on key attributes that map neatly to the unique needs of machine vision and DSA solutions:

• High bandwidth—MIPI specifications detail interfaces for connecting electronic systems and components in drones and other systems that are high speed, highly scalable, and have ultra-low latency. Use of these interfaces allows for highly data-efficient modes of operation. As a result, a wide range of imaging and sensing components—from highly advanced cameras to low-speed sensors—is supported.
• Low electromagnetic interference—Drone designers are challenged to meet stringent, sector-specific electromagnetic compatibility (EMC) regulations. MIPI specifications meet these needs by reducing electromagnetic interference (EMI) through a combination of techniques, including low-voltage swings on high-speed physical layers and the support of slew rate control. This flexibility gives developers the ability to match the EMI profile of the physical layer interface to the EMI needs of the end device. An added benefit is reduced product weight, meaning less EMI shielding is needed.
• Low power consumption—MIPI specifications facilitate flexible, highly power-efficient data transmission among drone components in multiple modes of use. This means power consumption is kept low—whether a component is in an “active” mode, where data is being sent and received; in “active-standby” mode, where data transmission is low and interrupt-driven; or in a "full-standby” mode, where there is no data transmission.
• Security—Devices in the drone market must also adhere to strict functional safety standards, so security is extremely important. The MIPI Security Working Group is overseeing the creation of a security framework, starting with cameras, capable of providing security at the protocol level, thus offering full end-to-end protection across the entire data stream.
• Small size and weight—MIPI specifications include high-performance serial interfaces that rely on a minimal number of wire conductors, allowing designers to minimize pin counts. This leads to fewer interconnections on chips and across printed circuit boards. Components can be packaged more tightly, and the weight of the design, as a whole, is reduced. These are especially key benefits in the drone market.
• 5G readiness—MIPI specifications already can support 5G services, including Enhanced Mobile Broadband, mMTC and URLLC, providing drone designers with a migration path for emerging communications and technology requirements.

Powerful Features in Latest MIPI Embedded Camera and Imaging Interface

The recently updated MIPI Camera Serial Interface 2 (MIPI CSI-2) specification is especially relevant to the enterprise drone market. CSI-2 is a high-speed protocol fortransmitting still and video images from image sensors to application processors. It is suitable for single- or multi-camera implementation in many different application spaces.

The CSI-2 interface contains many attributes and features relevant to the growing needs of enterprise drones. For instance, Differential Pulse Code Modulation (DPCM) 12-10-12 compression reduces bandwidth while delivering high-quality signal-to-noise-ratio (SNR) images, free of compression artifacts. Also, CSI-2's Smart Region of Interest feature, introduced in v3.0, allows for analyzing images, inferencing algorithms, and making better deductions. This capability could enable a drone monitoring a pipeline, for example, to more quickly identify a potential break or other anomaly.

The newest version of the specification, MIPI CSI-2 v4.0, released in late 2021, facilitates development of devices with even greater capability for machine awareness. These features prove useful across many application spaces, including enterprise drones:

• Always-On Sentinel Conduit (AOSC), which allows drones and other devices to use always-on machine vision systems. Combinations of ultra-low-power image sensors and video signal processors (VSPs) can continuously monitor their surrounding environments, waking their higher-power host CPUs when something significant occurs. AOSC also allows for highly efficient streaming of image frames from a drone image sensor to a VSP over a low-power MIPI I3C bus, with scaling options to add extra I3C lanes and bandwidth as defined by the I3C specification.
• Multi-Pixel Compression (MPC), which optimizes pixel compression for state-of-the-art image sensors with multi-pixel color filter arrays (CFAs).
• RAW28 color depth pixel encoding, which supports new levels of image quality and a better signal-to-noise (SNR) ratio in the next generation of high-dynamic-range image sensors. RAW28 is an addition to previously introduced RAW 16 and 20 in CSI-2 v2.0 and RAW 24 in CSI-2 v3.0.

A Family of Relevant Specifications

CSI-2 v4.0 also supports transmission of image frames over the low-cost, low-pin-count MIPI I3C/I3C Basic two-wire intelligent serial interface. The MIPI Camera Control Interface (CCI) can work with I3C/I3C Basic to support advanced imaging performance requirements, such as auto focus and optical image stabilization (OIS), among other applications. Both of these improvements in imaging performance support drone usage for detailed inspections and monitoring.

Many other MIPI specifications also offer benefits for the field of enterprise drones:

• MIPI C-PHY and MIPI D-PHY are low-pin-count serial interfaces suitable for linking high-speed sensors in smaller drones in which line lengths are less than 50 cm.
• MIPI A-PHY provides a reliable serial interface with high noise immunity that uses coax or STP cables to connect high-speed sensors over up to 15 meters in large drones.
• MIPI UniPro and MIPI M-PHY enable JEDEC Universal Flash Storage (UFS) for storing high-resolution video locally on a drone.
• MIPI RFFE controls RF front-end components within wireless communication modules such as power amplifiers, low-noise amplifiers, filters, switches, and antenna tuners.
• MIPI CSE is a set of extensions that define functional safety and other services for image sensors and associated control interfaces in drones, aeronautic, and other use cases involving cameras.

Going Further

With the legal landscape for drone usage clarified by the European Union Aviation Safety Agency and the US Federal Aviation Administration regulatory activity, the global enterprise drone market is poised for significant growth. Analyst reports suggest that the enterprise market will generate most of that market value, with the most significant growth expected in agriculture, construction and mining, insurance, media and telecommunications, and law enforcement.

MIPI specifications can help address the wide-ranging challenges facing enterprise drone designers—from reducing complexity and weight, to complying with EMI and safety regulations—in seizing the burgeoning opportunity. For details on how MIPI specifications relate to enterprise drones and other IoT usages, see the MIPI white paper, “MIPI Alliance: Enabling the IoT Opportunity.” And for more information about MIPI CSI-2, listen to the MIPI CSI-2 v4.0 Panel Discussion with the MIPI Camera Working Group.