PDF version of this document

Specificity and Generalisation

Our approach to the assessment of NRR relies on the close relationship between registration and statistical model building, and extends the work of Davies et al. on evaluating shape models [5]. We note that NRR of a set of images establishes the dense correspondence which is required to build a combined appearance model. Given the correct correspondence, the model provides a concise description of the training set. As the correspondence is degraded, the model also degrades in terms of its ability to reconstruct images of the same class, not in the training set (Generalisation), and its ability to only synthesise new images similar to those in the training set (Specificity). If we represent training images and those synthesised by the model as points in a high dimensional space, the clouds represented by training and synthetic images ideally overlap fully (see Fig. 2). Given a measure of the distance between images (see next section), Specificity, $S$, Generalisation, $G$, and their standard errors $\sigma_{\mathcal{S}}$ and $\sigma_{\mathcal{G}}$ can be defined as follows:

$$G=\frac{1}{n}\begin{array}{c} n\\ \sum\\ i=1\end{array}min_{j}\,\vert I_{i}-I_{j}\vert,$$ (3)

$$S=\frac{1}{m}\begin{array}{c} m\\ \sum\\ j=1\end{array}min_{i}\,\vert I_{i}-I_{j}\vert.$$ (4)

$$\mathbf{\sigma_{\mathcal{G}}}=\frac{SD(min_{\, j}\,\vert I_{i}-I_{j}\vert)}{\sqrt{n-1}},$$ (5)

$$\mathbf{\sigma_{\mathcal{S}}}=\frac{SD(min_{\, j}\,\vert I_{i}-I_{j}\vert)}{\sqrt{m-1}}.$$ (6)

where { $I_{j}:j=1..m\}$ is a large set of images sampled from the model, $\vert\cdot\vert$ is the distance between two images and SD is standard deviation.

Both values are low for a good model. Specificity measures the mean distance between images generated by the model and their closest neighbours in the training set, whilst Generalisation measures the mean distance between images in the training set and their closest neighbours in the synthesised set. The approach is illustrated diagrammatically in Fig. 3.

  

Fig. 3. The model evaluation framework.            Fig. 2. Training set and model

Each image in the training set is compared             synthesis in hyperspace

against every image generated by the model