- Given:
- Appearance model
- Training set that generated the model

- Sought:
- Measure of model quality
- Possibilities: specificity, generalisability, etc.
- These method are based on
*distance* __The question__: How can distance be measured?

- Can articulate distance in terms of parameters
- Intensity differences are problematic
- Wish to account for shape
*and*intensity variation- It is not clear how to consider both
- They are incommensurate

- The measure need to be:
- Easily/quickly computable
- The value will need to be calculated for entire training set
- Complexity is proportional to set size

- Robust to:
- 'Folding' of mappings, e.g. in shuffling (search for match within a fixed window)
- ...

- Other properties:
- Distance from A to C is greater or equal to aggregated distance from A to B and B to C.
- ...

- Easily/quickly computable

- The approach will allow to measure faithfullness w.r.t. model
- It is known what the model encapsulates: shape and intensity
- Therefore, the behaviour is known
- Is is based on the synthesis/instantiation
- Model-building procedure is well-understood

- Data lies in space of e.g. parameter, intensity

- Each model synthesis is looked at in turn

- Distance measured to training set

- In specificity, nearest distance is of interest

- Generalisability reverses the roles of model syntheses and training set

- Does so to ensure the model does not span a large volume in space

- What distance should be measured?
- How to treat a finite yet large sets efficiently?
- Some ideas follow...

- Let us take a brain and a distorted version of it (whirling filter)

- Let us assume one of them is model synthesis

- The other one is arbitrarily taken from the training data

- Let us remember that we have a
*set*of training data

- The aim is to show that the model is not far apart from the training data (at least some instances)

- The measure cannot be solely intensity-based

- Same brain, different position in space

- This is an example of translation inconsistency
- Shape has similar properties
- Example: Brain is wider/narrower

- The shape change causes great difference in intensities

- Same brain stretched so must account for shape

- Try to match point in one image to another within a boundary

- But this can produce awkward mappings

- Can measures like MI be of help here?

- Using 'general-purpose' similarity measure
- Good for registration
- Will not take advantage of all knowledge
- Does not have proper notion of shape and intensity
- However, quick to compute

- Let us look at the set again

- The distinction between model synthesis and training data instance can be neglected

- All that is needed is a metric of distance
- Takes only 2 images (or volumes) at a given time
- Distance relates to intensity and shape
- Care for efficiency

- Look at only two instances

- Model synthesis holds extra information

- Showing that model describes instances fairly thoroughly
- Model does not describe illegal instances
- Specificity and generalisability do this
- Specificity 'handles' the former condition
- Generalisability 'handles' the latter

- The metric needs to be robust to awkward instances
- Example #1: Void image
- Example #2: Reversed (flipped/mirrored) image
- Example #3: Strange shape variation in uniform areas like background

- What if subsets of training sets taken?

- Construct model of subsets and perform model comparisons?

- What if the set is taken

- What if the images are taken

- Points of correspondence to be used

- Triangulate and treat as features

- Model image segments

- Taking intensity of points of correspondence (control nodes) is unreliable
- Not enough nodes in practice
- Sampling along them might not give good matching locally, pixel-to-pixel
- By modelling, there is a more
*tolerant*measure

- Once similarity is obtained...
- ...how does one use it to measure quality of entire model
- Most reasonable to look at the problem in terms of representation in space
- Reliability and efficiency have trade-offs