Rotor Technologies Developing Giant Ag Drone

Rotor Technologies (Nashua, NH, USA) has developed a drone for agricultural applications such as crop spraying. The company, founded in 2021 by Hector Xu, a former MIT scientist who led the development and successful flight of the first ion-propelled airplane, specializes in large drone technology.

This drone, known as  Sprayhawk, is in the final stages of development before going on the market. Indeed, Sprayhawk has flown successfully several times, most recently in a demonstration at the National Agricultural Aviation Association’s Ag Aviation Expo in Fort Worth, Texas, in November.

“We were flying, we were spraying; it works,” says Nicholas Coates, vice president of partnerships with Rotor. “We are now fine tuning all these systems.”

Sprayhawk is not a fully autonomous vehicle but integrates autonomy and autopilot systems and is flown remotely by a pilot on the ground. It is built on the airframe of a Robinson R44 four-passenger helicopter. Instead of seats, however, Sprayhawk is outfitted with the equipment it needs to function. This includes four 2.9 to 5 MPixel industrial cameras from Allied Vision (Exton, PA, USA) equipped with Sony (Kanagawa, Japan) Pregius global shutter sensors, LiDAR, GPS, and an onboard computer system that utilizes Nvidia (Santa Clara, CA, USA) GPUs that allows the pilot to control the craft and all its functions, from flying and navigation to spraying crops.

Related Content: Drones, High Resolution Cameras Boost Inspection Safety and Quality

Vision Components, Multi-Source Data Stream Helps Pilot Fly Drone

Underpinning Sprayhawk’s functionality are numerous elements working in synchronization to provide the pilot with the most detailed and useful information possible, in as close to real time as possible. The drone has four cameras, three in front facing left, right and center and another mounted at the tail facing the rear. Simultaneously, the LiDAR and GPS gather and transmit topographical information of the area in which the drone is flying.

“Somebody might call that a ‘digital twin,’ where we're loading 3D maps of the environment that we're flying in, and we're using that to create sort of a backdrop for our pilot,” explains Alex Lesman, Rotor’s Director of Software Development. “Then, we're taking the images and we've calibrated our cameras to the aircraft. And so we're able to overlay the video stream onto this digital environment, to present the pilot with a cohesive view of the world. and it also gives us some kind of fall back.”

“You can imagine if something happens to a camera physically or if our bandwidth is severely degraded or something else is obstructing visibility, you have this digital twin backdrop.”

While the digital twin currently does not show dynamic obstacles in the area, it does give the pilot a general sense of the area, he says.

Communicating Useful Flight Data to a Remote Pilot

Data is collected via cameras, LiDAR, and GPS and transmitted to a small commodity computer equipped with Nvidia GPUs that is installed aboard the aircraft. The information is then streamed to the pilot over multiple mediums, Lesman says.

“Broadly speaking, we use the Nvidia GPUs and some other libraries, and code we've written in as low latency a way as possible, process the video streams, and then stream them over to the pilot over mixed links—those links could be LTE, satellite, radio, or whatever we have available. We split up the video stream, send it across those multiple links, reassemble it on the far side, do some error correction, put it back together, and play it to the pilot—and try to do that while introducing as little latency as possible.”

They use multiple streams so as not to be dependent on one medium; if a satellite link goes down, for example, the pilot can still receive the information over another link, such as radio.

“We can support multiple communications links,” Lesman says. “We can support communications by running a communications relay. Our focus has been delivering reliable low latency video to the pilot. And so that means no single wireless link is perfectly reliable but if you put enough of them together it's reliable enough.”

Related content: Norwegian Maritime Company Develops Autonomous Ferry

Challenges and Next Steps for the Drone

The team has conducted flights from different distances, from areas within sight line and/or radio range of the drone and from remote locations several miles away from where the drone is flying. The imaging/communications systems have worked as intended, Lesman and Coates say.

There were some challenges along the way, they say. When they first started building the system, they used GigE cameras. However, this added latency to the video stream and was expensive at higher levels. USB3 is significantly more cost effective and works efficiently for their purposes, but they are working on further ruggedization, they say.

And, while their basic system works as intended, they are fine tuning and improving the communication system for optimal operational ability for the pilot.

“I think the idea over time would be to do more and more fusion, to try to fuse our virtual map of the world with our real time data,” Lesman says.

Another goal is to eventually increase the autonomy of the aircraft by automating some routine functions to free up the pilot for other tasks that require attention. For example, the aircraft must constantly maintain precise altitude and speed while it’s spraying a field. To be able to automate that aspect of the flight would be beneficial for the pilot, who can then handle additional tasks associated with the mission.

Also, the aircraft is usually transported to the work site on a truck, from and upon which it takes off and lands. If a pilot is performing crop spraying tasks, then that pilot will be taking off and landing very frequently. This is not mundane, but it is routine and can be ultimately very tiring to the pilot.

“You go out, you spray, and you come back to reload the chemicals,” Coates says. “You're landing on that truck and you're taking off from that truck, over and over and over again. It's incredible to see the pilots do it -- they land on a truck that's smaller than the helicopter and they come in hot while they’re doing it.”

Coates and Lesman do note that the autonomy would be related to pilot and flight safety before anything else.

“There are ways we can have kind of – maybe you would call it collaborative autonomy— where the aircraft would assist, rather than take over,” Coates says. “You could think about it as lane keeping in a modern car—it’s not driving for you, but it helps you stay on course and safely in your lane. I think we’ll figure out where the sweet spot is between assistive autonomy and full autonomy.”

Related content: German Aerospace Center Explores Using Drones to Sniff Out Hazardous Gasses