Looking at Mars With the Right Kind of Eyes

How NASA is Approaching Imaging for the Mars Rover 2020 Mission

BY Geralyn Miller and Robert Groulx February 28, 2020

Hadley Green, a middle-schooler in Mississippi and finalist in NASA’s naming contest, suggested that the 2020 Mars Rover be named “Vision.” This is an excellent suggestion: the 2020 mission will pack 23 cameras, more than Curiosity’s 17 cameras, MER’s 10 cameras per rover (Spirit and Opportunity), and way ahead of the Pathfinder mission’s five (two on the lander and three on the Sojourner rover). With each new mission, the amount and quality of data returned by rovers has increased dramatically as technology has improved.

Illustrations of NASA's Curiosity and Mars 2020 rovers. While the newest rover borrows from Curiosity's design, each has its own role in the ongoing exploration of Mars and the search for ancient life. (Image credit: NASA/JPL-Caltech)

This expanded vision capability will be essential for the 2020 rover to complete its role in NASA’s Mars Exploration Program (MEP): to seek signs of life. The mission will seek signs of habitable conditions on Mars in the ancient past and search for evidence of past microbial life, particularly in special rocks known to preserve signs of life over time. In addition, it will collect core rock and soil samples, storing them for later retrieval, and test oxygen production from the CO²-rich Martian atmosphere.

To do its work, the rover will have a few categories of cameras onboard. Descent cameras will help pilot the rover’s descent to the Martian surface. Engineering cameras will allow the rover to successfully navigate the rocks and dust that cover the planet. And finally, scientific cameras will make observations and help collect samples.

Choosing the right solution: The sensor you know

Mars is a difficult place to get to. By 2019, only 25 of 55 missions there have been fully successful, with a further three only partially successful. Every time we go there, decisions of what to bring become a balance of what scientists would like, and what engineers feel will reliably work. This makes each Mars mission part of a continuing chain of innovation, with the learning from past missions informing proven technologies and innovations to future missions.

Over the last twenty years, one perfect example has been CCD sensors. It has been the go-to standard for cameras in space, including on Mars. “A CMOS sensor would probably die very rapidly as soon as it went into space,” points out Robert Groulx, Product Manager for CCD technology at Teledyne DALSA Semiconductor. He has worked in semiconductor R&D process development since 1985. Extremes of heat and cold can wear out electronics, and radiation and cosmic rays can wreak havoc on electronic integrated circuits by altering the states of the elements inside them. “CCD is old technology, but very good technology. Any camera that goes up needs to be space-qualified, which is a very rigorous process, taking a long time. With CCD, we know exactly how to do that.”

On five of NASA’s missions to Mars—in 2000, 2004, 2008, 2012, and the current Mars InSight mission—the CCD image sensors on its cameras were made by Teledyne, in collaboration with NASA’s Jet Propulsion Laboratory (JPL). The cameras on the InSight lander helped scientists and engineers choose where to place instruments on the surface of Mars; these instruments will help them study the interior of Mars. This included the Instrument Deployment Camera, which is placed under the lander and pointed at 120° with a fisheye-type lens, and the Instrument Context Camera, which is on the lander’s robotic arm, looking at a 45° angle. Besides their lenses, both cameras are identical, collecting one-megapixel images to help with the placement of other instruments. Since their placement, the sensors have been almost immediately successful, indicating that Mars is more magnetic and more tectonically active than we expected.

The Insight cameras were monochrome frame-transfer CCD models left over from the earlier MER program. “But this time JPL wanted to take color photos. Color cameras are much larger, and new models couldn’t be qualified in time for launch. We needed another solution,” says Groulx. The solution was to put color filters over the monochrome sensors. “We knew from the users of those color filters that the potential for qualification was good. We had used them on Teledyne DALSA CCD sensors for many years and really know they work. But we had never sent them to space! JPL had to make sure that those color filters would endure the trip to Mars.”

“For the color filters, we use a technology that is unique to Teledyne DALSA that we’ve been doing since 2002 or 2003,” says Groulx. “It uses color pigments that are evaporated onto the CCD. It’s more expensive but produces very good results in terms of color separation and reproduction, better than the typical dyed resist method.”

NASA's InSight spacecraft captured this panorama of its landing site on Dec. 9, 2018, the 14th Martian day, or sol, of its mission. The 290-degree perspective surveys the rim of the degraded crater InSight landed in, nicknamed "Homestead Hollow." The panorama is made up of 30 individual images that were taken by the spacecraft's Instrument Deployment Arm camera, located on its robotic arm.

Choosing the right solution: The camera you know

The four cameras on Curiosity used a different solution, the KAI-2020 (originally from Kodak, then sold to TrueSense and now belonging to OnSemi), an interline CCD sensor that shoots at 1600×1200. The cameras went into the Mars Hand Lens Imager (MAHLI), Mars Descent Imager (MARDI), and two MastCams.

Curiosity launched in 2011. Why did it only have a two-megapixel sensor, smaller than a lot of mobile phones even then? Once again, the sheer logistics and complexity of a Mars mission defined what was possible. “There’s a popular belief that projects like this are going to be very advanced but there are things that mitigate against that. These designs were proposed in 2004, and you don’t get to propose one specification and then go off and develop something else. 2MP with 8GB of flash didn’t sound too bad in 2004. But it doesn’t compare well to what you get in an iPhone today,” said Mike Ravine, project manager at Malin Space Science Systems, responsible for the cameras’ development in a 2012 interview, “And the state of CMOS sensors wasn’t credible in 2004. They’re an interesting option now, but they weren’t then.”

By using the same sensor NASA was able to minimize the effort needed for mission qualification, and more easily optimize the sensors for their operating environment, such as accounting for the effect of radiation on individual pixels. There’s also the type of photos the rover would be taking: landscapes. With nothing on Mars moving (besides the ubiquitous dust), it was possible to take multi-shot panoramas with the help of the MastCam arm, creating extremely high-resolution results.

This panorama of 817 stitched images taken by the Opportunity rover from December 2011 to May 2012. The shots were taken on Greeley Haven, on the western edge of Mars’ Endeavor Crater.

One disappointment in the Curiosity imaging system was the cancellation of 6.5-100mm zoom lenses that would allow the cameras to catch additional detail, as well as cinematic 3D images (which director James Cameron was apparently very excited about). Zoom lenses would have to wait until the next rover in 2020.

Getting around Mars in 2020: Applying everything you know

To keep mission costs and risks as low as possible, the Mars 2020 design is based on NASA’s successful Mars Science Laboratory mission architecture, including its Curiosity rover and proven landing system. Like the other missions, cameras on the 2020 Mars Rover will play several roles. They will assist in landing, be a key part of the surface navigation process, and contribute to the scientific objectives. But big improvements in the onboard cameras will expand what the 2020 rover is capable of.

Descent Cameras

When the Curiosity rover landed on Mars, it recorded the descent and landing with its Mars Descent Imager or “MARDI” camera. This camera shot full-color video of Curiosity’s journey through the atmosphere all the way down to the Martian surface. The view was extremely valuable to engineers; it helped them understand what happens during one of the riskiest parts of the mission. The footage also gave the science team and rover drivers a glimpse of the landing site to aid them in accurately identifying Curiosity’s landing spot and plan the rover’s first drives to explore Gale Crater.

The Mars 2020 will also be equipped with a Lander Vision System, which the team expects to capture better-than-ever recordings of the events that deliver the rover to the Martian surface. It will be the first spacecraft with the ability to accurately retarget its touchdown during the landing sequence. For the Mars 2020 rover, the engineering team is adding several cameras and a microphone to document entry, descent and landing in even greater detail. They will shoot full-color video of Mars 2020 throughout its descent to the Martian surface. Some of what the cameras see on the way down will help mission planners decide on the rover’s first drives.

The Mars 2020 entry, descent and landing camera suite includes:

“Up look” cameras on the descent stage to record the parachute’s inflation
“Down look” camera on the descent stage to view the rover from above during descent
“Up look” camera on the rover itself to see the descent stage in operation as it lowers the rover from the sky crane
“Down look” camera on the rover to view the ground

In addition to providing engineering data, NASA considers the cameras and microphone partly a “public engagement payload,” giving all of us onlookers a good and dramatic sense of the ride down to the surface. No one has ever seen a parachute opening in the Martian atmosphere, the rover being lowered down to the surface of Mars on a tether from its descent stage, the bridle between the two being cut, and the descent stage flying away after rover touchdown.

Still, there is a lot of competition for payload and limited space. Currently, the cameras and microphone are counted as “discretionary payload,” which means they’re assets, but not required for the mission. Perhaps as a result, NASA has not revealed the specifications of these sensors, instead simply saying that they are “assembled from easily available commercial hardware.”

Engineering cameras

Mars 2020 uses a new generation of engineering cameras that build on the capabilities of past Mars rover cameras. These “enhanced” engineering cameras give much more detailed information (20 megapixels), in color, about the terrain around the rover. Each of the nine cameras share the same camera body, but use different lenses selected for each camera’s specific task. They have various functions: they measure the ground around the rover for safe driving (HazCams), support pathfinding (NavCams), and aid sample-gathering (CacheCam).

The HazCams and NavCams will help human operators on Earth drive the rover more precisely, and better target the movements of the arm, drill and other tools that get close to their targets. A much wider field-of-view gives the cameras a much better view of the rover itself. This is important for checking on the health of various rover parts and measuring changes in the amount of dust and sand that may accumulate on rover surfaces. The new cameras are also capable of higher frame rates than previous cameras systems, allowing it to take pictures while the rover is moving.

Science cameras

This image, taken in the Spacecraft Assembly Facility's High Bay 1 at the Jet Propulsion Laboratory in Pasadena, California, on July 23, 2019, shows a close-up of the head of Mars 2020's remote sensing mast. The mast head contains the SuperCam instrument (its lens is in the large circular opening). In the gray boxes beneath mast head are the two Mastcam-Z imagers. On the exterior sides of those imagers are the rover's two navigation cameras. (Image credit: NASA/JPL-Caltech)

While the additional cameras will help, real mission success lies with the scientific payload. Here we find some of the most significant differences between Curiosity and Mars 2020. The large robotic arm on the front of the Mars 2020 rover is bigger than Curiosity’s. This will enable the 2020 rover to collect and store samples, but also manage a host of new functions and new science tools. This turret has the coring drill and two science instruments, plus a color camera for close-up surface inspection and “selfies” for rover health checkups.

Mastcam-Z, an upgraded version of Curiosity’s high-definition, two-camera Mastcam instrument, will serve as Mars 2020’s main set of eyes. With a maximum image size of 1600 by 1200 pixels, it still can resolve incredible detail, between about 150 microns per pixel (0.15 millimeters / 0.0059 inches) to 7.4 millimeters (0.3 inches) per pixel depending on distance. “Camera technology keeps improving,” said Justin Maki of JPL, Mars 2020’s imaging scientist and deputy principal investigator of the Mastcam-Z instrument. “Each successive mission is able to utilize these improvements, with better performance and lower cost.” The “Z” is for zoom, the capability that was left out of Curiosity’s Mastcam.

Mastcam-Z will be able to take more 3D images, potentially allowing mission scientists to examine geologic features in more detail and scout out promising scientific sites from large distances. “Routinely using 3D images at high resolution could pay off in a big way,” says Mastcam-Z principal investigator Jim Bell, of Arizona State University in Tempe, said in the same statement. “They’re useful for both long-range and near-field science targets.” And making James Cameron happy, we hope.

A 3D printed model of Mastcam-Z, one of the science cameras on the Mars 2020 rover. Mastcam-Z will include a 3:1 zoom lens. (Image credit: NASA/JPL-Caltech)

From here the imaging tools become much more exotic. The SuperCam was built by a team of hundreds and packs what would typically require several sizable pieces of equipment into something no bigger than a cereal box. It fires a pulsed laser beam out of the rover’s mast, or “head,” to vaporize small portions of rock from up to seven meters away at up to 18,000 degrees Fahrenheit (10,000 degrees Celsius). This combination camera and spectroscope then analyzes the chemicals released in a method called laser-induced breakdown spectroscopy (LIBS). SuperCam will use artificial intelligence to seek out rock targets worth zapping during and after drives, when humans are out of the loop. In addition, this upgraded AI lets SuperCam point very precisely at small rock features.

There is PIXL, an X-ray fluoroscopy “micro-context camera” that targets grain-size pieces of rock. The rover can check PIXL’s images against a library of “elemental composition maps” to identify chemical elements.

Then there are the pair of SHERLOC and WATSON, which are a pair of cameras focused on searching for signs of life. Despite these bacronyms* being a little old-fashioned, the cameras are very forward-looking. SHERLOC is another laser-and-spectroscope combo with a macro camera capturing extreme closeups of areas under study, to send home for mission scientists to work on how rock samples formed. WATSON will help with SHERLOC’s mission, capturing fine-scale textures of rocks and dust, and can be turned inwards to look at the rover.

Ready to go and see

A Mars 2020 Rover test model at NASA’s primary robotics research facility is the Jet Propulsion Laboratory in Pasadena, California, managed by the California Institute of Technology.

The 2020 Mars Rover is scheduled to launch between July 17^th and August 5th, 2020, with a planned landing on February 18^th, 2021. Once on the planet surface, the mission should last at least one Mars year, which is about 687 Earth days. Mars 2020 will give us all a front-row seat in a Mars landing for the first time in the history of space exploration.

The march to Mars will clearly continue. Five more missions are in the late stages of development and fabrication and will be launched between 2020 and 2021. These include the ExoMars Rosalind Franklin rover and landing platform Kazachok by Russia and ESA, NASA’s Mars 2020 rover, the 2020 Chinese Mars Mission, the Hope Mars Mission by the United Arab Emirates, and India’s Mars Orbiter Mission 2. We’ll be eagerly watching them all.

Additional Links

*A bacronym is a constructed phrase that purports to be the source of a word that is an acronym. Backronyms may be invented with either serious or humorous intent, or they may be a type of false etymology or folk etymology. The ideas of discovery and mystery fit with a theme of exploring the unknown, but these look a little like someone had too many late nights on a deadline:

SHERLOC: “Scanning Habitable Environments with Raman & Luminescence for Organics & Chemicals”
WATSON: “Wide Angle Topographic Sensor for Operations and eNgineering”