PDF version of this entire document

Motivation for Ground-Truth-Free Assessment

The search for an assessment method that uses the image data alone is motivated by numerous factors:

1. The need for an assessment that is unaffected by human judgment such as annotation, where there can be considerable inter- and intra-subject variation.
2. The need to process large amounts of data, either in 2- or 3-D. Without automation, the process is hindered by human capacity.
3. The resources allotted to obtaining ground truth. Different types of data are inherently different, so methods of extracting ground truth may vary.

The method sought would potentially be resource hungry. The subsequent chapters show that a brute-force approach can be used as a substitute to an approach that uses ground truth.

The need for ground-truth-free methods of evaluation has been recognised in other areas. The STAPLE algorithm [] from Warfield et al. addresses the problem of evaluating complete segmentations of an object. The method uses an E-M algorithm to estimate the most likely ground truth, given the set of segmentations.