A Survey of Specularity Removal Methods
Abstract
In the real world, materials often show both diffuse and specular reflections. To separate these two reflection components may help applications which need consistent object surface appearance. In fact, many algorithms used in numerous tasks of computer vision, computer graphics and image processing, work under the assumption of perfect Lambertian surfaces (or perfect diffuse reflection). They consider specular pixels (or highlights) as outliers or noise. Moreover, the two reflection components can be processed separately and afterwards recombined to produce particular visual effects [MZK*06] (dichromatic editing Figure.1).Figure 1: Example of different visual effects produced by dichromatic editing [MZK*06]: (a) is the input image, (b) wetness effect, (c) skin colour change and (d) effect of make-up. Input image courtesy of S.P. Mallick. |
Acquired textures with the presence of highlights are a classical example of how details may be lost. In this case, the simulation of the changes of the light source position will be affected by the highlights and shadows generated by the light source used during the acquisition process. Figure.2 shows an image where details and colours are completely washed out in the highlights region. To recompose the details and the colour information, highlights removal techniques are required. These techniques, starting from one or more input images, extract two intrinsic properties; the diffuse and the specular images. This is a classical ill-posed problem [Wei01], because the number of unknown variables is larger than the number of equations. In fact, we have only one equation that defines the total radiance as the sum of different terms.
Figure 2: Example of an image where the presence of highlights generates the loss of details and colour information. Details and colours are completely washed out in the highlights region. |
Several specularity removal techniques are available in literature; they differ in the information they use and in how it is used. Table 1 summarises how these techniques may be classified and in which paragraph of this survey they are explained. The notation used in Table 1 for the categories is the following: CSA, colour space analysis; NA, neighbourhood analysis; POL, polarization; IS, image sequences; MFI, multiple-flash images. The notation used in Table 1 for the techniques is the following: CRM, using colour reflection model [KSK87, KSK88]; TDA, 2D diagram approach [ST95b, SK00]; BM, Bajcsy et al. method [BLL96]; USFI, use of specular-free images [TI05b, YCK06, SC09]; PDE, PDE approach [MZK*06]; IT, in-panting technique [TLQ06]; CC, use of colour information and classifier [TFA05]; TS, separation of highlight reflections on textured surfaces [TLQ06]; FR, Fresnel methods [Ang07]; HM, histogram methods [CGS06]; HFLF, high-low frequency separation [LPD07, NGR06]; MBS, multi-baseline-stereo [LB92, LYK*03, LS01]; MII, deriving intrinsic images from image sequences with illumination changes [Wei01]; CP, colour and polarisation methods [NFB97, KLHS02, MPC*07, USGG04]; MF, multi-flash methods [FRTT04, ARNL05].
|
||||||
Single
Image |
Multiple
Images |
|||||
|
|
|||||
Tech. - Sec. |
Type |
CSA |
NA |
IS |
MFI |
POL |
|
||||||
CRM - 3.1.1 |
G |
x |
- | - | - | - |
TDA - 3.1.2 |
L |
x |
- | - | - | - |
BM - 3.1.3 |
G |
x |
- | - | - | - |
USFI - 3.2.1 |
L |
- | x |
- | - | - |
PDE - 3.2.2 |
L | - | x |
- | - | - |
IT - 3.2.3 |
L | - | x |
- | - | - |
TS - 3.2.4 |
L | - | x |
- | - | - |
CC - 3.2.5 |
L | - | x |
- | - | - |
FR - 3.2.6 |
G |
- | x |
- | - | - |
HM - 4 |
G |
- | - | x |
- | - |
HFLF - 4 |
G | - | - | x |
- | - |
MBS - 4.1 |
G | - | - | x |
- | - |
MII - 4.2 |
L |
- | - | x |
- | - |
CP - 4.3 |
G |
- | - | - | - |
x |
MF - 4.4 |
L |
- | - | - | x |
- |
|
Discussion
Several factors may influence which approach is superior to another one, such as the number of images to be captured, automatic operation vs. manual help, light constraints and the reflectance model used, merits of quality, and the hardware used during the acquisition phase. Table 2 provides a comparative summary of the specular removal methods discussed in this survey.
|
||||
Technique |
Images |
User
Interaction |
Light
Requirement |
Hardware |
|
||||
Colour Space [KSK87, KSK88, ST95b, SK00, BLL96] |
1 |
MS |
IC-DM |
S |
Specular-Free Image [TI05b, YCK06, SC09] |
1 |
A |
IC-DM | S |
Inpainting [TLQ06] |
1 |
MS |
IC-DM | S |
PDE [MZK*06] |
1 |
- |
IC-DM | S |
Textured Surfaces [TLQ06] |
1 |
MS |
IC-DM | S |
Colour Classifier [TFA05] |
1 |
- |
- |
S |
Fresnel Coefficient [Ang07] |
1 |
MS |
No IC-DM | S |
Multi-Baseline Stereo [LB92, LYK*03, LS01] |
50-70 |
NS | DM |
M |
Deriving Intrinsic Images [Wei01] |
40-70 |
NS | Lighting changes |
S |
Polarization [NFB97, KLHS02, MPC*07, USGG04] |
6-10 |
NS | DM |
S* |
Histogram [CGS06] |
200 |
NS | - |
S |
High-Low Frequency Separation [LPD07, NGR06] |
4, 32 |
NS |
- |
S |
Multi-flash [FRTT04, ARNL05] |
4-8,2 |
NS |
FM |
FS |
|
Finally we have performed several experiment, to compare different specularity removal techniques, on various input images starting from single-colour, multi-colour and textured surfaces, increasing the texture complexity.
References
For the references please see the paper.Images and movies
BibTex references
@Article\{ABC11, author = "Artusi, Alessandro and Banterle, Francesco and Chetverikov, Dmitry ", title = "A Survey of Specularity Removal Methods", journal = "Computer Graphics Forum", number = "8", volume = "30", pages = "2208-\^a€“2230", month = "December", year = "2011", keywords = "specularity removal, dichromatic reflection model, colour shading, shadowing, diffuse and specular reflections, highlights", url = "http://vcg-legacy.isti.cnr.it/Publications/2011/ABC11" }