PDF version of this entire document
A correlation was shown between the degree of misregistration and measures such as Specificity and Generalisation. Specificity was found to be more sensitive than Generalisation, which is unsurprising given the difference between the size of the training set and the size of the set derived from the model. As pointed out in the previous chapter, these two measures are separate but not independent. As regards the relative size of the standard error for the two measures, this too is caused by the difference in set sizes.
Another important point is that among those two measures, Specificity is more meaningful, but in practice both Specificity and Generalisation are needed in order to perform a full and valid assessment. Ways of combining these two to obtain a single figure of merit are not trivial, although it's really Specificity that one should be interested in. It is not possible to assess the quality of a registration using Specificity alone or using Generalisation alone because in very special cases, there is a chance of the method failing slightly. For a good model to be built, those two measures would need to be optimised at the same time.
Despite this separation into two measures, it should be possible to consider just Specificity, but then use Generalisation to ensure an extreme case is not leading to an inaccurate conclusion. In practical terms, it is Generalisation that is actually quite easy to achieve, whilst being specific is hard, hence Specificity is more informative. In all cases I have come across, there was a duality between those two measures, but Specificity was the more valuable among the two. Table 7.1 shows the 4 possible situations where a comparison is made between A and B. Only in cases where Specificity and Generalisation contradict one another in their assessment will there be a dilemma because a measure cannot be chosen with certainty.
Overall, this chapter demonstrated, using a series of experiments, that the method is sensitive to reasonable degrees of misregistration and is robust to noise. It remains to be seen how the method can be applied to real-world problems, which is the topic the next chapter covers.
|