Methods for Pancam Multispectral Color Imagery

The Mars Exploration Rover Pancam

The MER pancam consists of a black and white, 1024x1024 resolution CCD and a choice of 13 color filters through which an image can be taken. 6 of these filters (L2, L3, L4, L5, L6, L7) encompass the range of human color vision, and using them the brightness and colors of the scene can be reconstructed.

Thanks to the enormous effort by NASA, JPL, and especially the Pancam team at Cornell University, the PDS release of data from the mission includes radiometrically calibrated versions of these images. Their calibration puts each pixel into a scale of absolute radiance at the wavelength appropriate to the filter in which is was taken

6 filter color

If all 6 visible light filters are used on a given target, a stepwise 3rd order polynomial fit is used to interpolate the radiance at wavelengths between the 6 points, reconstructing a full visible light spectrum. The interpolated spectrum is integrated against the CIE XYZ Color Matching Functions (2 degree CIE 1931 modified by Judd, 1951 and Vos, 1978) to translate the 6 images into a universal color space.

The use of CIE XYZ color space allows every possible visible color to be described, but it fails to tell us how best to reproduce those colors (appearance) as they would be seen in person, mostly due to an effect known as chromatic adaptation. In order to display the colors appropriately (particularly, on a computer monitor) conversion from the whitepoint of the surface illuminant is necessary to project the colors into the sRGB specification appropriate to most PC monitors. This adaptation is achieved using the CIECAM02 Image Appearance Model, available within the lcms library for Python.

The white-point converted XYZ color space is then converted to sRGB values for display purposes. This is a source of distortion, as the XYZ colorspace is larger than sRGB, therefore some color information is clipped to the limits of sRGB.

3 filter color

One of the biggest challenges inherent to the Pancam color dataset is that most targets were not imaged in all 6 visible light filters, but rather in 3. It is inherent to the time, energy, storage, and most importantly bandwidth limitations of operating these rovers that most of the large panoramas be taken using only 3 filters. This produces images which are effectively “color blind” to variation at certain wavelengths, depending on which 3 filters were used.

An interpolation, based on a methodology similar to the 6 filter interpolation, can be used to recreate the colors of a 3 filter scene, but it suffers from a lack of information about the shape of the spectrum at wavelengths not covered by the filters. In order to allow the greatest consistency between the 6 filter and 3 filter images, I chose to use the information from the 6 filter color (which is the most accurate description of the spectrum) as a training set to determine the appropriate formula for converting 3 filter data into the XYZ colorspace. Specifically, this was achieved by fitting via least squares an equation of the form i * filter1 + j * filter2 + k * filter3 = X, to compute i,j,k for an X value determined by 6 filter interpolation for all appropriate images over the first 180 sols of each rover. This is a bit of a hack, but had surprisingly consistent results (mean error is 1-3%) at determining the XYZ values using only 3 filters.

This fit is also responsible for the greatest ‘errors’ in the images, most specifically, the calibration target. The training gives greater weight to those colors which occupy the majority of the images (dust, rock, sky) and does not train the formula to correctly display colors which appear as a small fraction of the total image. One of the most blaring examples of this error would be the color chips on the calibration dial. In the two images below, you can see the difference between the correct rendering of the chips on the left, as seen in 6 filters, and the errors produced when using only L257 filters on the right.

The information contained in the L257 or L256 images allows little to no differentiation between the yellow and green chips as it does not have an image at those specific wavelengths, much like a person with color blindness would have trouble differentiating between two specific colors. Also, the blue chip now appears pink, this mostly due to the use of L2 (a near infrared filter, at which the blue chip appears bright, while in visible red light the blue chip appears dark) as the only information pertaining to the red portion of the spectrum. Since most of the other objects on Mars that appear bright in L2 are in fact red, the trained formula imparts red onto the blue chip making it pink.

The ‘majority training’ also dulls the diversity of colors of the sky , since the sky and the ground appear the same in several of the 3 filter views, but in 6 filter show variation. This ‘dulling’ is something I’m trying to work around, by training formula specific to the material (dusty ground, rock, sky) and then applying the appropriate formula to that material when it appears in a 3 filter image, but might not be ready for some time.

The filter combos which are available here are L234567 (full 6 filter, most accurate), L456, L457, L257, L256 and L247.

Brightness Scaling

When the XYZ values are converted to sRGB, a somewhat arbitrary decision is made as to how to scale the range of values on the limited range of brightness available to a monitor. Traditionally, the scale is shifted to maximize the range available in the photograph, giving the most possible contrast to that individual frame. On the other hand, if a mosaic is to be constructed containing many different frames, the scale is set to the brightness range of the collection of images. With the goal of maximizing consistency and comparability between all the images in this gallery, I have taken this a step further, and have chosen a single range to apply to almost all the images for each rover. This acts to compress individual frames into very limited ranges within a much wider scale which can include all ground and near horizon brightnesses. This makes many images look washed out and very drab, particularly when imaging a dark ground target or any other target which occupies a small range of brightnesses. I regret limiting the contrast of the images in this manner, but in the hopes of being able to convey the difference between a dark sunrise and noon time sun, I feel it is appropriate. Much more spectacular, vibrant and exciting images have been produced by NASA/JPL/Cornell and many talented amateurs, but they lack an absolute scale which can put a scene into a perspective relating to the rest of the images from other sols.

The brightness range used here for each rover is different, with Spirit being set to 0 - 0.12 Watts/m^-2/nm/sr and Opportunity set to 0 - 0.10 Watts/m^-2/nm/sr. These ranges allow almost all ground and near horizon sky images to be displayed without clipping to the limits of sRGB. The exception to this rule are images of the sun and bright sky around it, which are hundreds of times brighter than the set range allows. The sunset sequences (Spirit, sol 67, Opportunity sol 94 and 101) were set to a fixed range 3 times as large as the normal images. The rest of the bright sky images are scaled automatically, according to their maximum brightness values.

Image Sizes

The images stored onboard the rovers are sometimes reduced in dimension or subframed before being transmitted. In order to line up frames (often one filter was compressed in a manner different than another filter of the same target), each image here is restored to its original resolution as it was on the CCD. Some frames, particularily atmospheric images, were resized as small as one tenth the original resolution before being sent to Earth. Their restoration to original size shows noticeable artifacts. This resizing is done via the bicubic interpolation built into the Python Imaging Library.

MER Publications

The planetary data systems (PDS) mission data for the Mars Exploration Rovers is available through the Analyst's Notebook interface provided by Washington University in St. Louis and also through the PDS Imaging Node.

I owe a huge debt of gratitude to Bill Allen at Asteroid/Comet Connection who has written Python PDS reading software, available at http://www.3dartist.com/WP/bippy/

I would like to thank Michael Lyle at mars.lyle.org for hosting this site, as well as his full array of images.

The calibrated galleries are produced using the Calibrated to Radiance (RAD) files produced by the Pancam team at Cornell University. They are listed as science RDR's for the Pancam. More specifically, the RAD files are "Radiometrically-corrected RDRs calibrated to absolute radiance units".

More information can be found in the MER Software Interface Specification, particularly section 5.2.2.1, PANCAL method for Radiometric Calibration and MER/PANCAM Data Processing User's Guide.

My methods for colorizing from radiometrically calibrated files are a work in progress (particularly in terms of white point, brightness, and saturation scaling).

Color by Daniel Crotty ( slinted@lyle.org )

Web design by Adam Wick