3D-vision-guided robots assist in COVID-19 measures in China

Social distancing, the most effective method to slow the spread of the novel coronavirus that causes the COVID-19 disease, is generally defined as staying 6 ft. away from other human beings. Maintaining social distancing policies allows countries to flatten the curve, or keep infection rates as low as possible so as not to overwhelm hospitals with COVID-19 cases.

In some environments maintaining social distance proves almost impossible, for instance in hospitals where personal protective equipment (PPE) for doctors and nurses, like masks and gowns, is in short supply. Protecting these frontline healthcare workers is imperative to success in the fight against the disease.

Robots, on the other hand, do not require PPE and can perform simple tasks. Replacing humans promotes social distancing, helping to slow the spread of the virus, while also allowing healthcare personnel to focus on more immediate tasks.

3D depth sensor manufacturer Orbbec (Troy, MI, USA.; www.orbbec3d.com) has for years championed the value of 3D sensing to the medical community for applications like fall detection and patient monitoring, alerting doctors or nurses when a patient requires assistance. According to David Chen, co-founder of Orbbec, thousands of these cameras, currently deployed in hospitals across the United States, can “decrease injury levels in senior citizens due to falls by more than 50%.”

With the outbreak of the novel coronavirus, the company has demonstrated many applications of its 3D cameras in healthcare environments on robots that perform supply transportation, custodial, and guiding tasks in hospitals in China (figure 1).

The robots deployed in Chinese hospitals use three models from Orbbec’s Astra 3D camera line: the Astra, Astra Mini, and Astra Mini S. All three cameras have RGB and depth image resolutions of 640 x 480, operating at 30 fps. The cameras also require less than 2.4 W power and can operate on embedded platforms like single-board computers, making the cameras ideal for deployment on robots.

Astra series core technologies, optical design, depth calculation algorithm (based on self-developed and open source code), application-specific integrated circuits, and software were all developed by the manufacturer. The cameras provide two different modes, according to Chen: conventional RGB camera functions for environmental monitoring and 2D algorithms for tracking and recognition, and full-field depth information without the need for scanning or point sensing.

All of Orbbec’s cameras also support Android, Linux, and Windows operating systems, particularly important in China where robot manufacturers often use Android operating platforms for service robots instead of Linux or Windows platforms, says Chen.

While the company touts the adaptability of its cameras to a myriad of applications, use in hospital robots presents specific issues such as limited field of view (FoV). Obstacle avoidance algorithms operate best with wide FoV and the tight quarters of most hospitals restrict FoV to a large degree. Most Orbbec cameras have around a 70° diagonal FoV. The company is currently developing new optical designs with a 110° diagonal, in part to tackle conditions such as hospital deployments.

Camera robustness poses another challenge. Most 3D cameras can only work continuously for around one year, says Chen. Orbbec works to increase camera life and stability for deployment in harsh conditions, which from the company’s perspective includes hospital applications.

Peanut model robots from Keenon Robotics (Pudong, Shanghai, China; www.keenonrobot.com), equipped with Astra Mini S cameras, have been deployed within the Guangdong, Hubei, Beijing, Shanghai, Hunan, Zhejiang, and Jiangsu regions since the coronavirus crisis began, in more than 50 hospitals, to deliver supplies within the buildings.

The Astra Mini S, 80 x 80 x 20 mm in size, has a range of 0.35 to 1 m and 60° H x 49.5° V x 73° D FoV. Also equipped with LIDAR, infrared, and ultrasonic sensors, the 1200 x 500 x 500 mm Peanut robot can run for under 8 hours drawing 500 W from a DC 48 V 12 Ah battery, and has a maximum speed of 1 m/s.

The robot has three trays for holding items, with the top and middle trays measuring 400 x 350 x 200 mm and the bottom tray measuring 280 mm square by 140 mm deep. All three trays can carry up to 10 kg each. In a supply room, a hospital employee loads the delivery robot with meals or medical supplies that the robot delivers to a target location. The robot navigates via a pre-defined route or using simultaneous location and mapping routines.

Frequent sanitizing, the second most important strategy in the fight against the spread of the novel coronavirus, is a task for which robots are particularly well-suited. Relieving humans from the need to scrub or spray down surfaces on which the virus may have settled removes a significant source of possible exposure.

Orbbec has provided 3D cameras for sanitizing robots since 2016, according to Chen. Today, the company supplies Astra model cameras for Iben Robot sanitizing robots (figure 3), manufactured by Xiaoben Intelligence (Beijing, China; www.ibenrobot.com). The robots spray liquid sanitizer on and clean floors in hospitals in the Shengzheng, Hubei, Beijing, and Shanghai regions, and are equipped with baseline Astra series cameras.

Astra cameras, 165 x 30 x 40 mm in size, with a 0.6 to 8 m range, have the same 60° H x 49.5° V x 73° D FoV as the Astra Mini S. The Astra provides obstacle avoidance functions while the robot sprays micron-sized beads of sanitizing liquid in a 360° circle around the robot. The robot moves between 0.1 and 0.7 m/s, can hold 16 L of sanitizing fluid, and can sanitize 1000 m² of floor within 15 minutes, according to Xiaoben Intelligence. Hospital staff set the time, route, and planning tasks for the robots.

Guide robots, robots that humans can follow from point A to point B, can eliminate the need for hospital personnel to direct patients or guests to one department or another and therefore also support social distancing. Orbbec provides Astra Mini cameras for around 200 GreetBots (figure 4), manufactured by OrionStar (Chaoyang District, Beijing, China; www.ainirobot.com), currently deployed in the Shengzheng, Beijing, and Shanghai regions.

GreetBots, designed as general-purpose service robots, run on single-board PCs with Rockchip (Fuzhou, Fujian province, China; www.rock-chips.com) 3288 or 3399 System on Chip (SoC) processors. The Astra Mini camera has a 0.6 to 5 m range and otherwise shares specifications with the Astra Mini S.

The robots track human faces, always facing the person the robot assists, according to OrionStar, and use active obstacle avoidance routines to lead the person to their selected destination. The robots currently operate on a single floor and cannot take elevators to different levels of the building, says Chen.

Measuring the efficacy of these medical robots deployed for slowing the virus’s spread remains difficult, but Orbbec offers a take on the emotional impacts of the hospital robots.

“A lot of patients can be in each individual hospital. The doctors have a high risk for exposure to the virus. Both patients and doctors are pretty anxious,” says Chen. “Our company cannot measure how much risk has been reduced, but the robots make people feel safer and more relaxed. That’s very important feedback.”

Since these robot deployments in hospitals, Orbbec has also observed a marked increase in purchases from its robotics partner companies to restaurants in China, largely for food delivery applications, as the value of these hospital deployments validates the usefulness of the technology generally.

Conducting supply delivery, floor sanitizing, and guiding operations, while less complicated functions compared to other tasks robots equipped with 3D cameras can accomplish, serves an important role in the fight against COVID-19 in China. By promoting social distancing and freeing healthcare workers for other, more important tasks, these robots can literally save lives.