Aerial Imaging: I Fly with My Little Eye
Digital Imaging Finds a New Home in Small Unmanned Aircraft Systems
An electrical lineswoman hunches against the storm, squinting at a tablet screen and fidgeting nervously. It’s November in northern Ontario, the sleet is blowing 75 kph sideways, and there’s a problem with the power lines. She watches intently for minutes that to her seem much longer…then exhales pent-up tension and breaks into a smile.
The images on her tablet tell the story—the closeup of the high voltage line on top of the transmission tower (across the river 500 m away) looks intact. A thermal view shows none of the heat spikes that would indicate insulator damage. The problem isn’t up there—she’s not going to have to climb the tower in this weather. Thank you, tiny flying robot.
The Value of Perspective
The point of having flying robots, aka small unmanned aircraft systems (sUASs), is as much about imaging as about flying. Amazon Prime Air may someday deliver instant shopping gratification from the skies, but for the near future the most valuable payload a tiny flying robot can carry is usually an image sensor, followed closely by a transmitter to beam images back to the ground for a remote view.
This should be no surprise—the insights we gain from high-level perspectives are profound and incredibly valuable. Whether from trees, hills, towers, or eventually balloons and aircraft, we have always sought higher vantage points. The urge to acquire them is fundamental, built into our bodies (better visibility is one theory for bipedalism) as well as our languages—think of words like “overview,” “supervise,” “surveillance,” and “understanding the big picture.” Poets have written about the gift of “see[ing] ourselves as others see us.” Perhaps the ultimate “perspective photo” is the one astronauts get looking back at earth; those lucky enough to experience it firsthand frequently describe it as life-changing. In some cases aerial views can also be life-saving…or at least day-making.
Eye in the sky applications
Examples abound of applications eagerly adopting aerial imaging: pipeline inspection, wind turbine inspection, search and rescue, forestry management, mine surveying, precision agriculture (pinpointing application of pesticides, irrigation and fertilizer), construction surveys, broadcast television, sports photography, cinematography, petrochemical flare stack inspection, railway inspection, highway maintenance surveys, bridge inspection, oil rig inspection…the list goes on. In each situation, sUASs provide views that until now have not been practical or even possible by any other means.
SUASs’ advantages are both clear and game-changing. Small, light, and unmanned, they pose a much smaller risk to people (both pilots and bystanders) and property if anything goes wrong. The minimized mass that allows them to fly also minimizes the kinetic energy of a crash impact. Compared to conventional aircraft, they are much more portable, much more easily deployed—they can be carried in a suitcase or backpack and snapped together without tools—and cost much less to own or operate. They are also more tolerant of poor weather…and operators’ aerial selfie vanity shots.
Challenges: Motion Pictures from Moving Cameras
Flight capability by battery power is a marvel of engineering in its own right, but image capture brings extra challenges for flying robots. It isn’t as simple as just strapping a “point and shoot” handycam onto the flyer. It’s true that there are rigs that will carry DSLRs and panoramic cameras aloft, but for sUASs, every choice has a tradeoff.
Perhaps most obviously for a flying application, the first issue is mass. Every extra gram affects flight time, range, altitude, balance, stability and manoeuverability. Lifting a DSLR rules out long flights or anything less than perfect conditions. Inexpensive flying robots integrate their own cameras, optimized for small size and mass. They have to depend on off-the-shelf commercial image sensor components, so mobile phone imagers are obvious choices for small size, mass, and low power. Current imagers provide HD resolution video and 14+ megapixel stills in visible spectra or VGA resolution in infrared, but there is more to imaging than resolution. Since flying robots can move rapidly in X, Y, and Z dimensions and must deal with vibration from the motors as well as buffeting winds affecting pitch, roll, and yaw, camera shake is a huge issue. Even with careful design to balance and stabilize camera modules, getting a crisp image in spite of aircraft residual motion is difficult, especially if the image sensor uses a typical rolling shutter. Sensors with top notch global shutters of the type more common in machine vision help to eliminate smear and the “jello video” effect. To further isolate the camera from the flyer motion, high-end flyers build their imager modules into carefully designed custom gimbals that also allow the camera to look one direction while the robot flies in another.
Challenges: Resolution and Image Quality
Surveying, agriculture and GIS applications among others demand resolutions higher than flyers can provide in a single shot; as a result, sUASs often take multiple overlapping images that can be processed into simple 2D mosaics. Properly planned and positioned, flyer imagery can supply detailed elevation and 3D models with ground resolutions up to 1cm/pixel.
Photographers of all kinds appreciate remote focus control and optical zooming. Sophisticated flyers provide optical zooms of 20x or more, although this must be traded off against the added mass of lenses and servos, as well as the effect of those parts’ movement on a flyer’s center of gravity. Adding infrared sensors provides new and tremendously valuable levels of insight, but this also multiplies system complexity, mechanically and in terms of data processing and transmission. Almost all flyers now integrate GPS functionality and data—the flyers need it to follow a path home if they lose communication with the remote pilot.
Challenges: Data Transmission
Beyond image capture, sUASs’ next task is to reliably transmit stills and real-time video back to the base station. The first issue is availability of spectrum for communications—commercial spectrum can be quite limited, and while law enforcement and military have their own spectrum, all depend on locale. In RF communications, there is a direct tradeoff between bandwidth, power, and transmission distance—want to transmit a higher data rate from farther away? It will cost you power, which will cost you flight time. As with any bandwidth-limited application, data compression codecs can be very useful—the flyer may choose not to transmit full resolution via real-time image down-link. In this case, there is typically also on-board storage such as with a SD card, so that in the event of packet loss or image degradation otherwise during flight, there is a way of retrieving and physically offloading full resolution images and video immediately post-flight.
With images received, the processing pipeline continues—the host system must interpret and integrate all the data streams into meaningful imagery and show it to operators…preferably on high-performance daylight displays. Smartphone apps serve low-end monitoring, but higher end systems demand larger, more robust displays and finer controls. If you have to pilot a flyer manually, the video feed is vital, although this isn’t the case for more autonomous systems—sUASs such as Aeryon’s Scout and SkyRanger eschew joysticks and follow flight paths swiped across a tablet screen with a Google map-like interface. SUASs are outstanding examples of innovation and the interdependence of hardware and software. The challenges are considerable, but the payback from the unique perspectives they enable is compelling.
Back at the power lines, the tiny flyer has swiftly returned on command, and the lineswoman has quickly broken it down and packed it into its suitcase, ready for its next eye in the sky adventure.
