The first group of more systematic experiments explored the behaviour of the algorithm and determined ways in which parameters or additional methodologies affect its performance. The experiments were all managed by the graphical front end, AART, which was deployed on 10 PCs at the shared clusters of the Computer Science Department. Hardware specifications of the different machines were kept identical so as to get comparable results where experiments' durations are being measured properly. A general-purpose Nelder-Mead optimiser was used in all cases.
NRR comparisons were performed which yielded comprehensive numerical comparisons of NRR with different methods for assessing similarity across the sets of 1-D images, such as the ones shown in Figure . The optimisation, for example, was altered in terms of its granularity and subsets - as described later - were used to build a model on which the similarity measure is based.
I will show an NRR that looks at the weighting assigned to shape and intensity when building combined models of shape and intensity, a comparison which involves changing the sensitivity of the optimiser, and another comparison where the whole set is racing against a subsets approach.
To give an overview, there are three series of experiments where parameters vary systematically in order to learn their effect on the overall NRR performance. I consider, in turn:
Models built using different approaches were difficult to discern visually, unlike the quality of the resultant NRR. So, having chosen to compute the correct solution (which is known for the synthetic data), I was able to assign a figure of merit (computed from pixel differences) which was based on the ground-truth solution. This measure was computed on a pixel-by-pixel basis, where the pixel-wise difference was averaged over all the pixels in the shape component (warp). This involved a comparison between two vectors; one is the warp that registers the data and another is the optimal warp computed by applying the bump generation routine in reverse. It is well understood what the correct correspondence should be and the correct warp is piece-wise linear.
Roy Schestowitz 2010-04-05