What is the Space of Spectral Sensitivity Functions for Digital Color
Cameras |
|
Camera spectral sensitivity functions relate scene radiance with captured RGB
triplets. They are important for many computer vision tasks that use color
information, such as multispectral imaging, color rendering, and color constancy.
In this paper, we aim to explore the space of spectral sensitivity
functions for digital color cameras. After collecting a database of 28
cameras covering a variety of types, we find this space convex and
two-dimensional. Based on this statistical model, we propose two methods to
recover camera spectral sensitivities using regular reflective color targets
(e.g., color checker) from a single image with and without knowing the
illumination. We show the proposed model is more accurate and robust for
estimating camera spectral sensitivities than other basis functions. We also
show two applications for the recovery of camera spectral sensitivities —
simulation of color rendering for cameras and computational color
constancy. |
Publications
Jun Jiang, Dengyu Liu, Jinwei Gu and Sabine Susstrunk.
What is the Space of Spectral Sensitivity Functions for
Digital Color Cameras?. IEEE Workshop on the Applications of Computer Vision (WACV), 2013.
Jun Jiang, Dengyu Liu, Jinwei Gu and Sabine
Susstrunk.Supplementary
Document (with proof and other experimental details).
|
Images
|
|
Experimental setup to obtain the groundtruth of spectral
sensitivity:
we have measured the spectral sensitivity functions for 28
cameras, including professional DSLRs, point-and-shoot, industrial and
mobile cameras (i.e., Nokia N900), using a monochromator and a spectrometer
PR655. At each wavelength, the camera spectral sensitivity in RGB channels
is calculated by c(λ) = d(λ)/(r(λ)t(λ)), where d(λ) is the raw data
recorded by the camera, r(λ) is the illuminant radiance measured by the
spectrometer, and t(λ) is the exposure time of the camera. All other settings
(i.e., ISO and aperture) remained the same during the measurement for each
camera. The procedure is repeated across the whole visible wavelength from
400 to 720nm with an interval of 10nm.
|
|
|
The need of statistics prior of camera spectral sensitivities:
It is known that common color targets such as a colorchecker cannot
be used directly to recover camera spectral sensitivities under
conventional illumination (e.g., daylight, tungsten, fluorescent).
This is because the intrinsic dimensionality of real-world objects' reflectance is about 8, which is less than the number of unknowns in camera spectral sensitivities.
Direct inversion is not reliable, even with a small amount of
noise (1%).
|
|
|
Luther condition evaluation:
A camera satisfies the Luther condition if
its spectral sensitivity function is a linear transformation of the
CIE-1931 2-degree color matching function. The Luther condition can be
evaluated by the RMS error between C2deg and TC, C2deg are the CIE-1931
2-degree color matching functions, and C are the measured camera spectral
sensitivities. Color difference (CIEDE00) is calculated between C2deg and TC
under CIE D65 illuminant and the 1269 Munsell color chips. Ideally,
spectral RMS and color differences are zero if a camera perfectly satisfies
the Luther condition. Overall, most cameras have a deviation from the
Luther condition, especially for the two industrial cameras.
|
|
|
Principal components of camera spectral sensitivities:
The principal
components of camera spectral sensitivities. The three columns represent
the R/G/B channels, respectively. We performed PCA on Canon cameras, Nikon
cameras, and all 28 cameras. The 1st principal component accounts for over
95% of total variance for all three channels, and the first two principal
components accounts for over 97% of total variance. Thus, we model camera
spectral sensitivity functions as two-dimensional functions.
|
|
|
The recovery of camera spectral sensitivies of Canon 60D:
(a) The measured spectrum of a daylight. (b) The spectral reflectance of a
color checker DC. (c) The captured image (glossy and duplicate patches are
removed to avoid overweighting certain colors). (d) The recovered spectral
sensitivities with known daylight spectrum. By using a daylight model, we
can recover both the daylight spectrum (e) and the camera spectral
sensitivities (f). The subscripts m and e in (d) and (f) stand for the
measured and estimated camera spectral sensitivities, respectively.
|
|
|
Comparison of recovered camera spectral sensitivies using 3 basis
functions:
(a) Fourier basis, (b) polynomial basis, and (c) radial basis.
The results are worse than that of using the PCA model. The subscripts m
and e stand for the measured and estimated camera spectral sensitivities,
respectively.
|
|
|
Comparison of four types of basis functions for modeling camera spectral
sensitivity functions:
A -- PCA model, B -- Fourier basis, C -- radial basis and
D -- polynomial basis with the ground truth (E). A color checker is rendered
under D65 with camera spectral sensitiv- ities recovered using these basis
functions, and converted to sRGB. The average color difference between the
renderings (from A to D) and the ground truth (E) are 1.59, 3.54, 2.43 and
7. The gain of the imaging system remains the same for all four basis
functions.
|
|
|
Simulation of color rendering for cameras:
The images are rendered to
sRGB based on the measured (top row) and estimated (bottom row) camera
spectral sensitivities of Canon 60D. (a) face, (b) beads, and (c) peppers
are from the multispectral image database [25]. The values in the
parentheses are the average color difference (CIEDE00 [11]) between the
bottom and top images in each column. For all three examples, the color
difference is close to one, indicating a close color match.
|
|
|
Correction of images by Canon5D Mark II:
CC is put in the scene to
locate the white point. The estimated camera spectral sensitivity of
Canon5D Mark II is used to calculate T. Left column: The captured image;
Middle column: the corrected image based on T, and Right column: the corrected
image by dividing the white point (without using T). The images are
rendered in sRGB color space.
|
|
Slides
WACV 2013 Presentation (coming soon)
WACV 2013 Poster (coming soon)
|
Software
Database of camera
spectral sensitivity The database includes the spectral sensitivity functions for 28 cameras,
including professional DSLRs, point-and-shoot, industrial and mobile
cameras. The measurement starts from 400nm to 720nm in an interval of 10nm. The database is in the
form of a text file. Each entry starts with camera name and follows by
measured spectral sensitivities in red, green and blue channel.
Code for recovering camera spectral
sensitivity from a single image This demo MATLAB code shows the recovery of camera spectral
sensitivity with a regular color checker from a single picture under unknown
daylight. An example image captured by a Canon 60D (CR2 RAW format) is
included. The measured camera spectral sensitivity for Canon 60D
and measured daylight are also included for comparison.
|
Spectral Sensitivity Measurement, CVL, University of Tokyo
|
|