How Does Retinex Work?


This is not the proper place to go into Land's complete Retinex Theory of Color Vision. (Note: Retina of eye, plus Cortex of the brain, gave it its name.) And I'm certainly not the best person to do it. Nevertheless, after viewing the color vision pages here some folks have commented: "Yes, but how does it work?" Arrgghh! It can't be covered in a short page, which is why I was avoiding it in the first place.
But -- you can find a lot on the subject in books and magazines on the subjects of optics, opthamology, psychological and biological studies, visual physiology, medicine, and other related fields, in the library, on the web, and from book stores, both new and used. The key words to search on are obvious. If you are mainly interested in a few general observations, I'll try to provide an overview, to get you started. But being no expert, apologies if some particular important factor is either not included, or is not described below with technical precision, after all, I'm a composer...!

Land learned that our eyes and brain together evolved to detect only some of the stimulus which vision provides. We view things more "qualitatively" than "quantitatively." Our eyes don't measure photons, their wavelength and mean energy, like an expensive laboratory instrument. We have no need to do that, much as our hearing doesn't care a fig about Hertz, or acoustic energy in Watts per second, and so on, aside from units of measurement developed to match what we hear, like the decibel (dB). We need our senses in order to survive. That's far more useful to us than a list of numbers. We need to know if a possible food or poison lies among those leaves or berries we're considering eating, don't we? Well, long ago we did care, much more than at our local supermarket...! ;^)
If we saw the berries in the daytime they would not appear exactly the same as when we looked near sunset. The sunset light would be much more orange-red than a noon impression. You could measure the amount, and a camera's film or CCD does actually measure, or at least record the true wavelength stimuli. But our eyes instead "correct" for the different lighting conditions and color. They sort of subtract out the surrounding bias, and return an impression that remains quite constant under many conditions. So we ignore our incandescent and fluorescent lighting differences at home at night, versus the light out of doors during the afternoon -- who cares?
Take two lamps, side by side, shine them both on a wall. Place a red bulb one, normal white bulb -- rather like the Land red-white lighting we got started with. Now hold your hand up in front of both. You should see two shadows. One of them will look red. The other will look not gray, but cyan (blue-green)! It's the complementary shade of the red. If the bulb were green, that shadow would be the complement, or magenta; a yellow bulb, and the shadow would appear blue. The effect had been noticed earlier, and the visibly colored shadows one sees are generally called "Goethe Shadows," for the author who first described them (yeay -- wotta poet!). I'd noticed them during my childhood "experiments," and another friend of mine tell me he had, too, at a similar age. (Thanx to Pete Z. for pointing out both of these neat bits and many other ideas for these new pages.)
What's going on is that we're witnessing the way we automatically correct our perceptions for a surrounding shade. It alters our perception of the smaller object so surrounded. A red apple looks red against a white plate, but put it on a blue or green place mat, and the color will really jump out at you! Contrast effect, which enhances the differences, that's what's going on. Same principle as before.
We can't help but make these corrections, it's a part of our human Retinex way of seeing. It's something that helps us disregard irrelevancies, like conditions in the large, to focus on the details within those surroundings. And it goes on at all sizes, nesting one surround effect within another within yet another, most of the time. That banana is yellow, but in a green dish, sitting on a blue and white tablecloth, covering a reddish brown table, all under an ordinary light bulb (yellow-white) on the ceiling, a cool-white (bluish) fluorescent lamp next to the sofa, which is of a greenish gray cloth, with beige and red pillows... the possibilities are enormous. And our brains and eyes have become amazingly adept at "canceling out" all those things, modifying the perception of that solitary banana based on the total accumulative effects of everything else I just named!
So, yes, it gets complicated, but the idea is not. Retinex both aids the actual color impressions and lighting contrasts, and heightens the differences based on what color stimulus is next to what other stimulus, next to yet another, and so on like above. When we watch a Technicolor film from the 50's we probably experience more than the camera and film actually saw. Certainly we do with our color video gear and computer monitors, even the printed color page. But early theorists thought their descriptions were complete. The ideas made sense, could be tested: we do after all see an RGB image, sort of, most of the time. And that seemed to be that.
Land received quite a bit of flack when he attempted to make sense of why these severely limited color worlds gave such surprising sensations of full color, or at least much more color than the older theories could explain. So Einstein's relativistic theories supplemented Newton's, they refined rather than "proved it wrong." And Land's Retinex supplement our earlier models. That's the usual way science works out, seldom is any that's really well established later discovered to be completely off the wall. But it may be incomplete, at least in certain conditions. Then a curious person investigates, tries to make sense of, and we're off and running once again!
So don't sweat it on the Retinex. Just realize we're making instantaneous comparisons from bit to bit, and automatically adjusting the output to the brain based on the whole environment. Change the environment, and you change the details you perceive. That's why those black borders help on the Retinex views you've been looking at: they isolate us from our surrounding world of light and color, lest these interfere, or even cancel out, the impression we're trying to study.
Make sense to you? Well, that's my best shot, can't spend any more time on it. The creation of all these pages, images and texts has been a rather enormous job, and I do hope you'll look all of it through, then investigate it further on your own. Can't be your library for you, the web seems to spoil us all that way. I know how it feels: "type in a few words or phrases, push the button, hey presto!, here's all the skinny." Nope, just the highlights, just a part of the story, one point of view, at best. Like my pages are here. Then there's work to do, to research into what everyone else has done of significance on your topic of inquiry. And finally, you must take what steps you are able to make from that juncture. You can't always "pay back" in life, so you "pay forward," for the next curious soul, bless them all...!
So don't just sit there staring at this! You might want to return to the vision pages you were looking at some minutes ago, or even sit down to think this all though until most of it makes some sense.

-Wendy Carlos
© 2001-2008 Serendip LLC. No images, text, graphics or design
may be reproduced without permission. All Rights Reserved.

Close this window to return to the main text.