PDF version of this entire document
We have described a model-based approach to assessing the accuracy of non-rigid registration of groups of images. The most important thing about this method is that it does not require any ground truth data, but depends only on the training data itself.
Validation experiments were conducted, based on perturbing correspondence obtained through registration. These show that our method is able to detect increasing mis-registration using just the registered image data. The results obtained for different sizes of shuffle neighbourhood show that the use of shuffle distance rather than Euclidean distance improves the range of mis-registration over which we can detect significant changes in registration accuracy.
More importantly, we have shown that what is being measured by our model-based approach varies monotonically with an overlap measure based on ground truth. And not only that, we have shown that in the case considered here, the model-based measure of specificity is in fact of greater sensitivity than the overlap measure based on ground truth, hence can reliably detect smaller differences in registration performance.
Finally, we have applied our model-based measure to assessing the quality of 3 different registration algorithms. The results obtained were in agreement with the results obtained during the validation phase as regards the relative sensitivity of the two model-based measures. We were able to show a quantitative improvement in performance of groupwise registration algorithms when compared to repeated pairwise registration.
We note that the experiments were conducted in 2D, which allowed larger-scale experiments to be conducted. However, the extension to 3D or higher is trivial, the only issue being that for higher-dimensional images, the calculation of shuffle distances (if used), will considerably increase the computational load.
In the above we used linear appearance modelling in our evaluation, but in principle, any generative model-building approach could be used. This method is totally general, and can be applied to the results of any registration algorithm.
This model-based method represents a significant advance as regards the important problem of evaluating non-rigid registration algorithms. It establishes an entirely objective basis for evaluation, since it is free from the requirement of ground truth data.