[NOTE: The live web app that encompasses this functionality can be found here: https://www.paintmap.studio. All of the underlying code for that site can be found here: https://github.com/bytebodger/color-map.]
In the previous article I took a pixelated image and matched each of the blocks to their "closest" colors in my inventory of paints. But as we saw in the finished images, the color matches were not entirely... satisfying. If we want to strive for better color-matching, the first thing to look at is the color space that we choose to use.
We started using a basic RGB model. This is intuitive to web developers because nearly all colors that we deal with follow an RGB model. But RGB is only one of many different color spaces. And each different color space will produce difference results.
For reference, here is our base image, which has already been pixelated, but has not been matched against the physical colors in my inventory of paints:
And when we did a basic RGB calculation, we ended up with this conversion:
That's not... horrible. But it's not great either. So let's look at using some other potential color spaces.
CMYK
CMYK is the color space used by printers. From this perspective, it may seem to be a good fit for our needs, because we're trying to match digital colors to those that exist in the real world (e.g., paints).
First, we'll need to update our getClosestColorInThePalette() function. The new version looks like this:
const getClosestColorInThePalette = (referenceColor = rgbModel) => {
const key = `${referenceColor.red},${referenceColor.green},${referenceColor.blue}`;
if (closestColors[key])
return closestColors[key];
let closestColor = {
blue: -1,
green: -1,
name: '',
red: -1,
};
let shortestDistance = Number.MAX_SAFE_INTEGER;
const algorithm = local.getItem('algorithm');
palette.forEach(paletteColor => {
if (shortestDistance === 0)
return;
let distance;
switch (algorithm) {
case algorithms.CMYK:
const {cyan: paletteCyan, magenta: paletteMagenta, yellow: paletteYellow, key: paletteKey} = convertRgbToCmyk(paletteColor);
const {cyan: referenceCyan, magenta: referenceMagenta, yellow: referenceYellow, key: referenceKey} = convertRgbToCmyk(referenceColor);
distance = Math.sqrt(
Math.pow(referenceCyan - paletteCyan, 2)
+ Math.pow(referenceMagenta - paletteMagenta, 2)
+ Math.pow(referenceYellow - paletteYellow, 2)
+ Math.pow(referenceKey - paletteKey, 2)
);
break;
case algorithms.RGB:
default:
distance = Math.sqrt(
Math.pow(paletteColor.red - referenceColor.red, 2)
+ Math.pow(paletteColor.green - referenceColor.green, 2)
+ Math.pow(paletteColor.blue - referenceColor.blue, 2)
);
break;
}
if (distance < shortestDistance) {
shortestDistance = distance;
closestColor = paletteColor;
closestColors[key] = paletteColor;
}
});
return closestColor;
};
I added case statements so that we can use different algorithms depending upon the color model we've chosen. Almost all of these calculations will use a root-mean-square (RMS) calculation. RMS is the formula that you use to find the distance between two different points in three-dimensional space. Note that I also converted the RGB algorithm to use RMS. I've found that this alteration makes little difference in the visual output of the RGB calculation, but using RMS will make it more consistent with the rest of the color space calculations.
Also note that we're always starting from RGB values. That's what gets passed into the function. So if we want to calculate differences based upon the CMYK color space, we'll first need to convert our RGB value to CMYK. That's why the values are first passed into convertRgbToCmyk() before we run the algorithm.
Here's what the convertRgbToCmyk() function looks like:
const convertRgbToCmyk = (rgbColor = rgbModel) => {
const {red, green, blue} = rgbColor;
let cyan = 255 - red;
let magenta = 255 - green;
let yellow = 255 - blue;
let key = Math.min(cyan, magenta, yellow);
const divider = key === 255 ? 1 : 255 - key;
cyan = ((cyan - key) / divider);
magenta = ((magenta - key) / divider);
yellow = ((yellow - key) / divider);
key = key / 255;
return {
cyan,
magenta,
yellow,
key,
};
};
And here's what the pixelated image looks like when we do our color matching based upon CMYK:
Well...that's a little different than the RGB calculation. But I'd hardly call it "better". So let's keep searching through different color spaces.
HSL
If you've done any work in graphic tools like PaintShop, you're probably familiar with HSL. It represents a conical color space based on hue, saturation, and lightness. This color space is also more intuitive for some artists.
So once again, we'll update our getClosestColorInThePalette() function. The new version looks like this:
const getClosestColorInThePalette = (referenceColor = rgbModel) => {
const key = `${referenceColor.red},${referenceColor.green},${referenceColor.blue}`;
if (closestColors[key])
return closestColors[key];
let closestColor = {
blue: -1,
green: -1,
name: '',
red: -1,
};
let shortestDistance = Number.MAX_SAFE_INTEGER;
const algorithm = local.getItem('algorithm');
palette.forEach(paletteColor => {
if (shortestDistance === 0)
return;
let distance;
switch (algorithm) {
case algorithms.CMYK:
const {cyan: paletteCyan, magenta: paletteMagenta, yellow: paletteYellow, key: paletteKey} = convertRgbToCmyk(paletteColor);
const {cyan: referenceCyan, magenta: referenceMagenta, yellow: referenceYellow, key: referenceKey} = convertRgbToCmyk(referenceColor);
distance = Math.sqrt(
Math.pow(referenceCyan - paletteCyan, 2)
+ Math.pow(referenceMagenta - paletteMagenta, 2)
+ Math.pow(referenceYellow - paletteYellow, 2)
+ Math.pow(referenceKey - paletteKey, 2)
);
break;
case algorithms.HSL:
const {hue: paletteHue, saturation: paletteSaturation, lightness: paletteLightness} = convertRgbToHsl(paletteColor);
const {hue: referenceHue, saturation: referenceSaturation, lightness: referenceLightness} = convertRgbToHsl(referenceColor);
distance = Math.sqrt(
Math.pow(referenceHue - paletteHue, 2)
+ Math.pow(referenceSaturation - paletteSaturation, 2)
+ Math.pow(referenceLightness - paletteLightness, 2)
);
break;
case algorithms.RGB:
default:
distance = Math.sqrt(
Math.pow(paletteColor.red - referenceColor.red, 2)
+ Math.pow(paletteColor.green - referenceColor.green, 2)
+ Math.pow(paletteColor.blue - referenceColor.blue, 2)
);
break;
}
if (distance < shortestDistance) {
shortestDistance = distance;
closestColor = paletteColor;
closestColors[key] = paletteColor;
}
});
return closestColor;
};
The HSL calculation is basically identical to the RGB calculation. We're still taking the RMS result from the values in the palette color versus the reference color. But rather than comparing red, green, and blue between both colors, we're comparing hue, saturation, and lightness.
Just as we need to convert RGB values to CMYK to run the CMYK matching algorithm, we'll need to convert RGB values to HSL to run our new HSL matching algorithm.
Here's what the convertRgbToHsl() function looks like:
const convertRgbToHsl = (rgbcolor = rgbModel) => {
let { red, green, blue } = rgbcolor;
red = red / 255;
green = green / 255;
blue = blue / 255;
const maximum = Math.max(red, green, blue);
const minimum = Math.min(red, green, blue);
const basis = (maximum + minimum) / 2;
let hue;
let saturation;
let lightness = basis;
if (maximum === minimum) {
hue = 0;
saturation = 0;
} else {
const difference = maximum - minimum;
saturation = lightness > 0.5 ? difference / (2 - maximum - minimum) : difference / (maximum + minimum);
switch (maximum) {
case red:
hue = (green - blue) / difference + (green < blue ? 6 : 0);
break;
case green:
hue = (blue - red) / difference + 2;
break;
case blue:
default:
hue = (red - green) / difference + 4;
break;
}
hue = hue / 6;
}
return {
hue,
saturation,
lightness,
};
}
And here's what the pixelated image looks like when we do our color matching based upon HSL:
Honestly, I prefer this transformation to the RGB or CMYK results. But she still kinda looks like she's wearing a mask. At this point, it's time to look beyond these "basic" color models.
XYZ
The XYZ color space (formally known as the CIE 1931 color space) is quite intriguing. As the name implies, it was developed back in 1931. It was the first attempt to produce a color space based on measurements of human color perception and it's the basis for almost all other color spaces.
That's what we're really going for here. We want to take two RGB colors and determine how "close" they are - based on human perception.
Again, we'll update our getClosestColorInThePalette() function. The new version looks like this:
const getClosestColorInThePalette = (referenceColor = rgbModel) => {
const key = `${referenceColor.red},${referenceColor.green},${referenceColor.blue}`;
if (closestColors[key])
return closestColors[key];
let closestColor = {
blue: -1,
green: -1,
name: '',
red: -1,
};
let shortestDistance = Number.MAX_SAFE_INTEGER;
const algorithm = local.getItem('algorithm');
palette.forEach(paletteColor => {
if (shortestDistance === 0)
return;
let distance;
switch (algorithm) {
case algorithms.CMYK:
const {cyan: paletteCyan, magenta: paletteMagenta, yellow: paletteYellow, key: paletteKey} = convertRgbToCmyk(paletteColor);
const {cyan: referenceCyan, magenta: referenceMagenta, yellow: referenceYellow, key: referenceKey} = convertRgbToCmyk(referenceColor);
distance = Math.sqrt(
Math.pow(referenceCyan - paletteCyan, 2)
+ Math.pow(referenceMagenta - paletteMagenta, 2)
+ Math.pow(referenceYellow - paletteYellow, 2)
+ Math.pow(referenceKey - paletteKey, 2)
);
break;
case algorithms.HSL:
const {hue: paletteHue, saturation: paletteSaturation, lightness: paletteLightness} = convertRgbToHsl(paletteColor);
const {hue: referenceHue, saturation: referenceSaturation, lightness: referenceLightness} = convertRgbToHsl(referenceColor);
distance = Math.sqrt(
Math.pow(referenceHue - paletteHue, 2)
+ Math.pow(referenceSaturation - paletteSaturation, 2)
+ Math.pow(referenceLightness - paletteLightness, 2)
);
break;
case algorithms.XYZ:
const {x: paletteX, y: paletteY, z: paletteZ} = convertRgbToXyz(paletteColor);
const {x: referenceX, y: referenceY, z: referenceZ} = convertRgbToXyz(referenceColor);
distance = Math.sqrt(
Math.pow(referenceX - paletteX, 2)
+ Math.pow(referenceY - paletteY, 2)
+ Math.pow(referenceZ - paletteZ, 2)
);
break;
case algorithms.RGB:
default:
distance = Math.sqrt(
Math.pow(paletteColor.red - referenceColor.red, 2)
+ Math.pow(paletteColor.green - referenceColor.green, 2)
+ Math.pow(paletteColor.blue - referenceColor.blue, 2)
);
break;
}
if (distance < shortestDistance) {
shortestDistance = distance;
closestColor = paletteColor;
closestColors[key] = paletteColor;
}
});
return closestColor;
};
Just like before, we're using a RMS calculation to determine the distance between the reference color and the palette color. But this time we're comparing the values from the XYZ color space.
Of course, that means that we need to convert the RGB values to XYZ. Here's what the convertRgbToXyz() function looks like:
const convertRgbToXyz = (rgbColor = rgbModel) => {
const convert = color => {
color = color / 255;
color = color > 0.04045 ? Math.pow(((color + 0.055) / 1.055), 2.4) : color / 12.92;
color = color * 100;
return color;
}
let {red, green, blue} = rgbColor;
red = convert(red);
green = convert(green);
blue = convert(blue);
const x = (red * 0.4124564) + (green * 0.3575761) + (blue * 0.1804375);
const y = (red * 0.2126729) + (green * 0.7151522) + (blue * 0.0721750);
const z = (red * 0.0193339) + (green * 0.1191920) + (blue * 0.9503041);
return {
x,
y,
z,
};
};
No, I certainly didn't come up with that equation on my own. But that's how you convert an RGB value to XYZ.
And here's what the pixelated image looks like when we do our color matching based upon XYZ:
Wow. To my eye at least, this particular transformation is a lot better. There's still some wonkiness there. There's some odd groupings of pinks above her eyes. But it seems to be a much closer match than anything we accomplished with RGB, CMYK, or HSL comparisons.
Delta-E 2000
The last model we're gonna experiment with is called Delta-E 2000. Delta-E 2000 isn't a color space. Rather, it's the latest in a long line of formulas that have been developed for color matching based upon the CIELAB, or L*a*b*, color space.
Here's a snapshot of the calculations that go into that formula:
Dayyumm... Well, let's drive into implementing this. First, we'll update our getClosestColorInThePalette() function again:
const getClosestColorInThePalette = (referenceColor = rgbModel) => {
const key = `${referenceColor.red},${referenceColor.green},${referenceColor.blue}`;
if (closestColors[key])
return closestColors[key];
let closestColor = {
blue: -1,
green: -1,
name: '',
red: -1,
};
let shortestDistance = Number.MAX_SAFE_INTEGER;
const algorithm = local.getItem('algorithm');
palette.forEach(paletteColor => {
if (shortestDistance === 0)
return;
let distance;
switch (algorithm) {
case algorithms.CMYK:
const {cyan: paletteCyan, magenta: paletteMagenta, yellow: paletteYellow, key: paletteKey} = convertRgbToCmyk(paletteColor);
const {cyan: referenceCyan, magenta: referenceMagenta, yellow: referenceYellow, key: referenceKey} = convertRgbToCmyk(referenceColor);
distance = Math.sqrt(
Math.pow(referenceCyan - paletteCyan, 2)
+ Math.pow(referenceMagenta - paletteMagenta, 2)
+ Math.pow(referenceYellow - paletteYellow, 2)
+ Math.pow(referenceKey - paletteKey, 2)
);
break;
case algorithms.HSL:
const {hue: paletteHue, saturation: paletteSaturation, lightness: paletteLightness} = convertRgbToHsl(paletteColor);
const {hue: referenceHue, saturation: referenceSaturation, lightness: referenceLightness} = convertRgbToHsl(referenceColor);
distance = Math.sqrt(
Math.pow(referenceHue - paletteHue, 2)
+ Math.pow(referenceSaturation - paletteSaturation, 2)
+ Math.pow(referenceLightness - paletteLightness, 2)
);
break;
case algorithms.XYZ:
const {x: paletteX, y: paletteY, z: paletteZ} = convertRgbToXyz(paletteColor);
const {x: referenceX, y: referenceY, z: referenceZ} = convertRgbToXyz(referenceColor);
distance = Math.sqrt(
Math.pow(referenceX - paletteX, 2)
+ Math.pow(referenceY - paletteY, 2)
+ Math.pow(referenceZ - paletteZ, 2)
);
break;
case algorithms.DELTA_E:
const paletteLabColor = convertRgbToLab(paletteColor);
const referenceLabColor = convertRgbToLab(referenceColor);
distance = calculateDeltaE2000(paletteLabColor, referenceLabColor);
break;
case algorithms.RGB:
default:
distance = Math.sqrt(
Math.pow(paletteColor.red - referenceColor.red, 2)
+ Math.pow(paletteColor.green - referenceColor.green, 2)
+ Math.pow(paletteColor.blue - referenceColor.blue, 2)
);
break;
}
if (distance < shortestDistance) {
shortestDistance = distance;
closestColor = paletteColor;
closestColors[key] = paletteColor;
}
});
return closestColor;
};
In this case, we're not doing the calculation directly in this function because we're not using a simple RMS algorithm. Instead, we're passing the values into our new calculateDeltaE2000() function.
But first, we need to get L*a*b* values from our RGB objects. Here's what the convertRgbToLab() function looks like:
const convertRgbToLab = (rgbColor = rgbModel) => {
const xyzColor = convertRgbToXyz(rgbColor);
return convertXyzToLab(xyzColor);
};
That's obviously pretty simple, because the L*a*b* conversion expects an XYZ color. Of course, we already wrote a convertRgbToXyz() function. Once we have the XYZ value, then we call convertXyzToLab().
Here's what the convertXyzToLab() function looks like:
const convertXyzToLab = (xyzColor = xyzModel) => {
const adjust = value => value > 0.008856 ? Math.pow(value, (1 / 3)) : (7.787 * value) + (16 / 116);
let {x, y, z} = xyzColor;
x = x / 94.811;
y = y / 100;
z = z / 107.304;
x = adjust(x);
y = adjust(y);
z = adjust(z);
const lightness = (116 * y) - 16;
const redGreen = 500 * (x - y);
const blueYellow = 200 * (y - z);
return {
lightness,
redGreen,
blueYellow,
};
};
Once we have the L*a*b* values, we can plug them into our calculateDeltaE2000() function looks like:
const calculateDeltaE2000 = (labColor1 = labModel, labColor2 = labModel) => {
const {lightness: lightness1, redGreen: redGreen1, blueYellow: blueYellow1} = labColor1;
const {lightness: lightness2, redGreen: redGreen2, blueYellow: blueYellow2} = labColor2;
Math.rad2deg = function (rad) {
return 360 * rad / (2 * Math.PI);
};
Math.deg2rad = function (deg) {
return (2 * Math.PI * deg) / 360;
};
const avgL = (lightness1 + lightness2) / 2;
const c1 = Math.sqrt(Math.pow(redGreen1, 2) + Math.pow(blueYellow1, 2));
const c2 = Math.sqrt(Math.pow(redGreen2, 2) + Math.pow(blueYellow2, 2));
const avgC = (c1 + c2) / 2;
const g = (1 - Math.sqrt(Math.pow(avgC, 7) / (Math.pow(avgC, 7) + Math.pow(25, 7)))) / 2;
const a1p = redGreen1 * (1 + g);
const a2p = redGreen2 * (1 + g);
const c1p = Math.sqrt(Math.pow(a1p, 2) + Math.pow(blueYellow1, 2));
const c2p = Math.sqrt(Math.pow(a2p, 2) + Math.pow(blueYellow2, 2));
const avgCp = (c1p + c2p) / 2;
let h1p = Math.rad2deg(Math.atan2(blueYellow1, a1p));
if (h1p < 0)
h1p = h1p + 360;
let h2p = Math.rad2deg(Math.atan2(blueYellow2, a2p));
if (h2p < 0)
h2p = h2p + 360;
const avghp = Math.abs(h1p - h2p) > 180 ? (h1p + h2p + 360) / 2 : (h1p + h2p) / 2;
const t = 1 -
0.17 * Math.cos(Math.deg2rad(avghp - 30))
+ 0.24 * Math.cos(Math.deg2rad(2 * avghp))
+ 0.32 * Math.cos(Math.deg2rad(3 * avghp + 6))
- 0.2 * Math.cos(Math.deg2rad(4 * avghp - 63));
let deltahp = h2p - h1p;
if (Math.abs(deltahp) > 180) {
if (h2p <= h1p) {
deltahp += 360;
} else {
deltahp -= 360;
}
}
const deltalp = lightness2 - lightness1;
const deltacp = c2p - c1p;
deltahp = 2 * Math.sqrt(c1p * c2p) * Math.sin(Math.deg2rad(deltahp) / 2);
const sl = 1 + ((0.015 * Math.pow(avgL - 50, 2)) / Math.sqrt(20 + Math.pow(avgL - 50, 2)));
const sc = 1 + 0.045 * avgCp;
const sh = 1 + 0.015 * avgCp * t;
const deltaro = 30 * Math.exp(-(Math.pow((avghp - 275) / 25, 2)));
const rc = 2 * Math.sqrt(Math.pow(avgCp, 7) / (Math.pow(avgCp, 7) + Math.pow(25, 7)));
const rt = -rc * Math.sin(2 * Math.deg2rad(deltaro));
const kl = 1;
const kc = 1;
const kh = 1;
return Math.sqrt(Math.pow(deltalp / (kl * sl), 2)
+ Math.pow(deltacp / (kc * sc), 2)
+ Math.pow(deltahp / (kh * sh), 2)
+ rt * (deltacp / (kc * sc)) * (deltahp / (kh * sh)));
};
You probably don't wanna spend too much time looking at that function. It'll give you a headache. For now, we can just be content to know that we have the logic quantified here in JavaScript.
And now that we can calculate Delta-E 2000 values, here's what the pixelated image looks like when we do our color matching based upon that algorithm:
To be frank, I don't like this one nearly as much as the product that we received from the XYZ calculation. But it's still useful as another reference point.
Which algorithm "won"??
Based upon the test results from this one image, it may be tempting to say that the XYZ algorithm is the "best". But that assessment would be overly-simplistic.
You see, I've learned after playing around with hundreds of images, that each algorithm (even, the simplistic RGB comparison) has its strengths and weaknesses. For this particular image, it may be true that XYZ is the "winner". But when you load other images, you may find that you get a much more satisfying result from one of the other algorithms.
For this reason, I didn't simply settle on a "winner" and then hardcode that into Paint Map Studio. Instead, I built an interactive form that allows you to experiment with many different settings (including, different algorithms) as you try to find the best possible match.
To be honest, it can sometimes be surprising to see how well a particular algorithm performs on one image - and then how poorly it performs on the next. So it's valuable to be able to flip between them and compare the results.
In the next installment...
We've implemented all the algorithms that we need to find the "best" match. So... is that it?? Hardly!
Even when we look at the XYZ-generated image above, it may feel closer to the original image than any of the other algorithms, but it still feels... off. There are pinks where they really shouldn't be. There are dark greens halo-ing her hairline. In short, we can still do better.
But if we're not gonna keep cycling through one algorithm after the next, how do we improve on the color matching?
The answer comes from our palette of colors. Even though I have more than 200 unique paints, each with their own distinct color, that's still not enough colors to faithfully portray an image like this woman's face. We need... more colors.
In the next installment I'll show how to mix paints virtually so you can determine the best palette for your image.
Top comments (2)
Dude. There are 2 things I appreciate about your post (still about to read the rest of the series). #1, I was coming to a lot of the same approaches and conclusions as you in my own project, without having read much about how others approach it, so you make me feel smart, and #2 you show that it is in fact a good idea to allow users to select from multiple algorithms / color spaces, rather than finding some single ideal approach.
Well written and thorough, thanks for this. Also I'd never heard of Delta-E 2000.
Thank you! And I'm glad this resonates with someone else as well! As for Delta-E 2000, it was a very interesting "rabbit hole" that I went down there. Initially, I thought the entire solution to my problem was just that I didn't have the "right" algorithm. In retrospect, I do think it's helpful to be able to compare different algorithms. But I also learned that there's no "magical algorithm" you can use that will solve all of your problems.