PDF version of this entire document

Measuring Sensitivity

As well as showing monotonicity, a good measure of registration quality should also sensitive. That is, it should enable us to measure small deviations from the optimum. If we can evaluate the sensitivity of a measure we will be able to fully compare the merits of various options.

The size of perturbation than can be detected in the validation experiments will depend both on the slope of the graphs of measure against degree of deformation, and also on the error on the measure. To quantify this, we define the sensitivity of a measure as follows.

Suppose 81#81 is the value of the measure for some degree of deformation 82#82. We then define the measure sensitivity as:

83#83

where 84#84 is the mean error in the estimate of 85#85 over the range. 86#86 is the change in 82#82 required for 81#81 to change by one noise standard deviation, which indicates the limit of changes in misregistration 82#82 which can be detected by the measure.

We computed the sensitivity for the data shown in Figures , , & . The averaged sensitivity over the range of deformations is plotted in Figure  for the various measures.

The first point to note is that there are statistically-significant differences between the various measures. Specificity is shown to be superior both to generalisation and most importantly, superior to the ground-truth based measure of Tanimoto overlap. Furthermore, we can see that shuffle radii of 87#87 and 88#88 for specificity give the most sensitive measure of all those studied.

Figure: Sensitivity of different NRR assessment methods

89#89