How does automatic HDR software work?

Question

An HDR image is the combined image of varied exposures that give good detail of the subject in the photograph. How does HDR software know what part of the images to blend?

Manually, we know by looking at the image which part needs to come from which exposure, but do these software do simply show the composite image with medium exposure from all the varied exposed images?

I ask this because I use LuminanceHDR for making HDR images and the results seem very poor. It would help me to understand what the software can do automatically and where a human eye is needed.

Answer

There are two distinct steps to producing the images that are frequently labelled "HDR":

• Exposure blending: merging multiple low dynamic range images into one image with higher dynamic range.

• Tonemapping: processing that high dynamic range image into a low dynamic range image suitable for viewing on standard [low dynamic range] equipment (such as regular computer monitors).

There are a number of ways to perform the first step but basically you need to estimate the difference in exposure for all of the images. This enables you to take a pixel value from any image and convert it to a consistent value in your final image. For example if you have two exposures that are 3 stops apart, multiplying values in the darker image by 8 will give values consistent with the brighter image (3 stops means three doublings of brightness, or 8x).

A simple strategy for exposure blending could be to take pixel values from the brightest image unless the values get close to overexposure, in which case you switch to the next brightest image, and so on until all pixels are converted. Most software will use a more sophisticated method, perhaps by using a weighted combination of values from close exposures to minimise noise and discontinuities. Note there might not be a simple multiplicative conversion possible between images (due to nonlinear response of the camera sensor, or nonlinear tonecurves) which complicates the conversion, some alignment of images may also be necessary.

The key point is that the exposure blending part is not subjective, there ought to be one correct way to blend images together. The problem is that you would need a high dynamic range monitor in order to appreciate a high dynamic range image. If you simply scaled the values down to fit a standard display you'd end up with a very flat image as there would be little contrast between nearby tones.

The image above was produced by blending three exposures and then scaling the values in a globally linear way. As you can see there is no over or under exposure in the image, even in the bright skies and the shadows of the pillars detail is retained. However the image is dull looking overall.

This is where tonemapping comes in. It squashes the dynamic range down to fit in an locally adaptive manner, taking into account image content in order to maximise the contrast locally and retain detail. This is the subjective part, as there are many ways to reduce the dynamic range. It's also the hardest part to get right as if you vary the contrast too much over too small an area you will get artifacts such as halos around edges that are derided by critics of HDR images.

Here's the same image after tonemapping, the contrast has been boosted in the darker regions in order to make the most of your monitors limited dynamic range to show detail and texture.

This image required a lot of tweaking and I'm still not happy with the output. The problem with tonemapping is that it is very easy to push it too far and end up with huge amounts of contrast locally, and very little globally, i.e. the image is very textured but has little overall structure. Most software has some sort of adjustment for the radius of the local adjustments, the lower the radius the greater the chance of producing a fake looking result. Colour saturation can suffer and should also be reigned it carefully.

note there are pieces of software such as Enfuse that blend images directly to a low dynamic range image, negating the need for two steps. I've not used any such program so I can't tell you how they work!

Check more discussion of this question.