hdrprep
ALE Benchmark
ALE is designed to improve the resolution, reduce noise and deblur images by combining multiple images of the same scene. This is a two stage process:
- Alignment of the input frames,
- Rendering of the frames to a single image.
Step 2 is essentially what happens when an exposure-bracketed sequence is stitched into an HDR image, and ALE can actually handle different image exposures. However, to fully recover the camera's response curve, algorithms such as Robertson (implemented in pfshdrcalibrate) or Debevec (implemented in hdrgen) are needed to accurately recover the luminance distribution of the scene. This is not implemented in ALE yet but might be in the future. Other approaches are possible, but will usually not give photometrically accurate results.
For this reason, hdrprep only makes use of the ALE for image registration. The problem with this approach is that ALE was never meant to only do registration without also doing the rendering. It is the rendering, however, that takes up most of the time, and we'll be looking at how this can be minimised.
This benchmarking was carried out after consultation of the ALE mailing list and is based on the feedback from this thread on the ALE mailing list. Described here are all tests I carried out. Some are more useful than others, and you MUST read to the end before jumping to conclusions!
All test were done on a pair of images of the same scene. Image dimensions were 2048x1536 px.
Transformation (--mc 10 --scale=1 --align-all) Translation 953 1.000 Euclidean 1058 1.110 Scale (--translation --mc 10 --align-all) 0.2 299 1.000 0.5 422 1.410 1 953 3.185 mc (--translation --scale=1 --align-all) 2 892 1.000 4 908 1.018 6 922 1.034 8 938 1.051 10 953 1.069 Channels (--translation --mc 10 --scale=1) Green 944 1.000 All 953 1.009 Sum 957 1.013
Discussion:
- --euclidean takes about 10% longer than --translation
- --align-green, --align-all, --align-sum have no influence on overall time
- --mc has a surprisingly small effect
- --scale is the biggie, however, the relationship is not square (number of pixels), nor even linear. Moving from --scale=.2 to --scale=1 takes only 319% of the time.
I was adviced to keep the scale to powers of two, so --scale=1, --scale=0.5 and --scale=0.25 would have been more appropriate. The required transformation of the full-scale images as indicated in the trans file would then have to be divided by the scale factor. However, this did not always give the same transformation that was derived with --scale=1, it was usually off by a few pixels. Successive test were therefore carried out with --scale=1 (the default setting).
What I was not aware of when running the above tests is the --profile option. This will output how long ALE has spend on alignment and rendering. This option is extremely handy for working out why things take so long. Unfortunately, it only outputs the real time, not user time. Make sure you system doesn't run any other heavy jobs at the same time.
$ ale --translation --mc 10 --align-all --scale=1 --exp-extend --profile Real time measurements ====================== Alignment (sampling) : 150.589868 s Alignment (checking) : 0.000000 s Incremental rendering : 292.507266 s Irani-Peleg rendering : 1609.180981 s
Surprisingly, most of the time goes into rendering which is a complete waist. We're only interested in the alignment bit and could happily drop out once the trans file is written. Somebody on the mailing list then suggested to use --no-inc and --ips 0. Which let to another quick test:
| Options | Relative time |
|---|---|
| --no-inc --ips 0 | 1.000 |
| --q0 | 1.509 |
| --q0 --no-inc --ips 0 | 1.491 |
| --q1 | 2.195 |
| --q1 --no-inc --ips 0 | 2.180 |
| --q2 | 4.099 |
| --q2 --no-inc --ips 0 | 1.004 |
| --qn | 4.236 |
| --qn --no-inc --ips 0 | 4.216 |
A quality setting of --q2 is the default in ALE which is why the result is identical to the first one where there is no quality setting. Below is the profile of the fastest run.
$ ale --translation --mc 10 --align-all --scale=1 --exp-extend --no-inc --ips 0 --profile Real time measurements ====================== Alignment (sampling) : 81.469536 s Alignment (checking) : 0.000000 s Incremental rendering : 126.000663 s Irani-Peleg rendering : 0.000000 s
So we haven't been able to completely disable rendering, but it is much faster now. Maybe a future release of ALE will implement a --no-render option for us HDR freaks.
The following tests were run with --no-inc and --ips 0 (shorter bars are better):
| Transform | --mc | --align | Alignment Time | |
|---|---|---|---|---|
| translation | 30 | all | 229 | |
| translation | 20 | all | 160 | |
| translation | 10 | all | 92 | |
| translation | 5 | all | 41 | |
| translation | 30 | green | 205 | |
| translation | 20 | green | 142 | |
| translation | 10 | green | 71 | |
| translation | 5 | green | 36 | |
| translation | 30 | sum | 209 | |
| translation | 20 | sum | 149 | |
| translation | 10 | sum | 76 | |
| translation | 5 | sum | 39 | |
| euclidean | 30 | all | 421 | |
| euclidean | 20 | all | 289 | |
| euclidean | 10 | all | 146 | |
| euclidean | 5 | all | 74 | |
| euclidean | 30 | green | 374 | |
| euclidean | 20 | green | 260 | |
| euclidean | 10 | green | 134 | |
| euclidean | 5 | green | 67 | |
| euclidean | 30 | sum | 574 | |
| euclidean | 20 | sum | 257 | |
| euclidean | 10 | sum | 133 | |
| euclidean | 5 | sum | 67 | |
Conclusions
- Keep --scale set to 1.
- If there are plenty of details in the image, reduce --mc
- Use --no-inc and --ips 0
- --euclidean alignment takes about twice as long as --translation
This benchmark looked at ALE options from a point of view of using it with HDR sequences, particulary for pfshdrcalibrate.