The final stage which is arguably the most fascinating one involves the use of the model above, as well as the correlations learned and recorded for that model. It is possible to carry out a search which is driven by the difference calculated between the model and a given target image. In practical terms this means that fitting of the existing model will slowly be improved until the model approximately covers the target16. The model state then holds (in the form of parameter values) some information about the target image and this information can be further analysed.
The search is reliant on error (or conversely similarity) measures which are repeatedly calculated after each attempted parameterisation of the model. Each such change in parameter values is primarily guided by the matrices that express the correlation between variation modes (the Similarity transformations as well as modes of appearance change) and the intensity values.
The model, as shown in Figure 4, is initially placed somewhere inside the image frame, with reasonable proximity to its target. If the model is placed too far from its target, there is a danger that it will not be able to converge to the target correctly. It will most likely get stuck in a local minimum and the outcome can be severe in a more crucial application such as medical imaging (or perhaps more drastically, computer-guided or -aided surgery). The reason why good initialisation is essential is that significantly large displacements are rarely learned off-line and the difference between the target and the model is quite meaningless unless there is at least some partial overlap or commonality.
The algorithm which is used to perform the search sensibly has a general form that resembles the following:
The technique of matching an appearance model to a target image is well-depicted by a staged simulation, a video clip or a large sequence of images as in Figure 4 above. Surprisingly, only a few dozens of iterations are required in order to get good results. This of course depends on the algorithm, the magnitude of the problem and its internal intricacies, e.g. fitting a perfectly round ball versus a human hand.