Progress Report
August 31st, 2005
Progress and Agenda - Overview
- Perturbation framework
- randomisation corrected
- further investigation of sensitivity aspects
- Evaluation experiments
- overlap
- specificity/Generalisation
- comparison
- Thesis Discussion
Experiments - Recapitulation
- We re-registered the set of IBIM-associated brains
- We then applied a series of perturbations to get sets of mis-registered data
- We can subsequently evaluate these multiple sets
- By doing such an evaluation, we can:
- demonstrate the correlations between mis-registration and our registration assessment methods
- (more specifically) compare overlap-based measures with model-based measures
Random Knot-Point Placement
- Started with the observation that there were non-random distributions
- The generator of knot-points had tendencies to place points at random positions, but adjacent to imaginary grid lines
- This flaw is now resolved (with the cost of an efficiency penalty)
- A couple of examples are shown in the next slides
Random Placements in Warps - #1
![](directions_corrected.png)
A colour-coded directions figure, which reflects on movement of points
Random Placements in Warps - #2
![](another_direction_figure.png)
Yet another figure showing the direction of the perturbation applied at any given point
Generalisation Sensitivity
- Continued work on investigating sensitivity
- Curves were plotted which show Generalisation sensitivity
- For the sake of comparsion, Specificity sensitivity is shown too
- The aim of these plots:
- discover the effect of the number of modes
- investigate the variation of sensitivity as registration degrades
Specificity Sensitivity
![](modes_sensitivity.png)
For various number of modes, sensitivity of Specificity is shown
Generalisation Sensitivity
![](generalisation_sensitivity_small.png)
For various number of modes, sensitivity of Generalisation is shown
Overlap Measures - Progress
- Formattting issues quickly resolved (PNG, JPEG and Analyze)
- Structure and naming-conventions make overlap processing/evaluation quicker
- Estimates of variance on the overlaps can now be retrieved as output
- Implementation the Dice coefficient (
Dice = Intersection / Mean Volume
)
- Implementation the Tanimoto coefficient (
Tanimoto = Intersection / Union
)
Specificity/Generalisation Measures - Early Progress
- Stage which involved generating sets of perturbed sets (impending)
- I was offered help in finding the necessary computer power
- The scale of the experiments is no longer a peril
- Experiments can be extended
- Generated data accommodates for extensions (owing to excessive data that gets generated)
Evaluation Dip
- Looking at some preliminary results, it appears as if we have resolved (at least) one problem
- Previously, we spotted an evaluation 'dip' at the start
- Put differently, when plotting Specificity versus the degree of mis-registration, there was a slight 'anomaly' at the start
- To resolve this problem we:
- forced a very slight re-sampling error at the start
- revised the perturbation method
Earlier Evaluation Steps
- Specificity and Generalisation curves produced at an early stage
- Should be considered a sanity check
- Span a wider range than is necessary
- Explore a wider range by sampling the curve at many points
Earlier Evaluation Steps - Ctd.
- Allowing better selection of sample points to be made
- There will be 40 such curves in total - 10 instantiations x (Euclidean + 3 shuffles)
- Can generate at most 2 such curves per day, which will be enough to complete 10 instantiations and make the curves smoother, i.e. 'refine' as much as time permits
Specificity Evaluated
![](perturbation_curve_specificity.png)
Specificity - shuffle distance 5x5, 15 modes
Generalisability Evaluated
![](perturbation_curve_generalisation.png)
Generalisation - shuffle distance 5x5, 15 modes
The Subsequent Step - Shuffles Compared
- Extension of previous figures (note range of mean pixel displacement)
- Includes only the first instantiation of pertubed sets
- Can probably finish work on 10 such instantions within a month, which makes the wishful plan practical
- Difficult to grasp why the curves are not monotonic
Shuffle Distances Compared - Specificity
![](shuffles_evaluations_specificity.png)
Various shuffle distances and corresponding evaluations of Specificity
Shuffle Distances Compared - Generalisability
![](shuffles_evaluations_generalisation.png)
Various shuffle distances and corresponding evaluations of Generalisation
Shuffles Compared - Summary & Conclusions
- Just to add context, the 'recipe' so far has been:
- take a set of 37 registered brains
- mis-register them
- measure Specificity and Generalisation
- Can reliably distinguish a good registration from bad one in the range of 0-3 pixels of mean deformation
- When going beyond that extent of deformation, the brains look a little odd
Shuffles Compared - Ctd.
- It is hard to discern bad registrations from very bad registrations
- For practical application, only a sensible range is worth considering
- Would have been nicer to see a bit more linear response
- This non-linear curve requires (begs for?) a valid explanation
- More worrying is to see non-monotonic behaviour
- Curves might look better once we have more results to average over
Finer-level Technicalities
- Building of Analyze-formatted images for overlap evaluation
- Single slices treated as 3-D objects
- There are 8 main directories, each of which has the same structure as ever before, i.e.:
- /images
- /label_02_whitematter_left
- ...
Finer-level Technicalities - Ctd.
- Each parent directory was named based on how much deformation had been applied
- The range investigated for extent of overlap is 0 to about 4 pixels in terms of mean pixel deformation
- The actual extent of deformation was intentionally disguised
- Perturbation of 0 is re-sampled from the originals so that we compare like with like
- Large-sized chunks of data, even when compressed
Overlap - Initial Results
- Measures of overlap seem to decline linearly
- Visual results shown in the next few slides
- Can use the results to argue that label overlap is related to Specificity and Generalisation
- Shown are the Tanimoto and Dice overlap measures versus the average pixel displacement
- A small fluke had values reversed at times, i.e. plots should be 'mirrorred' where indicated
Dice Overlap
![](dice_overlap.png)
Dice overlap decreases as registration is degraded (numbers need reversal)
Tanimoto Overlap
![](tanimoto_overlap.png)
Tanimoto overlap decreases as registration is degraded (numbers need reversal)
Overlap - Further Explanation
- A couple of figures were produced to show overlap versus mean of pixel
displacements (rather than just obscure numbers corresponding to warp magnitude)
- Construction of a a plot to show the relationship between Tanimoto (complexity) and the shuffle distance (window size 7 by 7)
- Parameters and measures chosen arbitrarily, not aiming to get results that best suit our arguments
- There would be at least 24 plots that we can look at (Euclidean and 3 shuffles X 6+ overlap calculations)
Overlap and Specificity
![](tanimoto_complexity_shuffle_7x7_ins_1.png)
The correlation between Specificity and overlap across labels
Corrected Overlap Figure
![](tanimoto_revised.png)
Corrected curve for Tanimoto overlap - overlap going downwards as registration degrades
Registration Assessment: Monotonic Increase in Specificity
- Nearly 4 instantiations have been completed by now
- Different perturbations applied to the registered brain set
- It seems as if, on average, we will be seeing monotonic curves for Specificity, as hoped
- The first instantiation did not have a monotonic response when perturbation exceeded some high values
- 4 out of 10 instantiation will have been completed tomorrow
- The remainder may take a couple of weeks at the least
Towards Thesis
- Form 11 meeting next week
- Working on continuation report as a starting point, but changing the content completely
- Thesis template was shared among the international LyX community and now available via the LyX Wiki
- Possibly good for the publicity?
- Already shared and used along several other Ph.D. students
Thesis Planning
- Current thesis is a 'placeholder' with template and general structure in tact
- This was a first important step towards focus on just textual content
- Figures, tables, equations, etc. should not an issue
- Thesis rough draft (PDF)
- Later slides outline what lies therein (the gist)
Thesis - Possible Structure
- Included some 'skeleton structure', which ought to make a cohesive flow of arguments
- begins with a model-based/MDL objective function
- resulting models and registration
- improving efficiency including reference to subsets in construction of optimal shape models
- carries on to 3-D registration and automatic construction of appearance models using registration
Thesis - Possible Structure - Ctd.
- Skeleton (continued from last slide):
- describes the evaluation of appearance models
- assessment of registration using the method
- results of model evaluation
- description of a perturbation scheme and some results from evaluation of non-rigid registration (in the absence ground truth)
Thesis Planning - Videos
- Plenty of separable videos
- registration in 1-D
- automatically constructed models in 1-D
- automatic construction of shape models of the hand and the bump
- 3-D registration in VXL
- model animations of automatically-constructed models of the brain
- face models evaluated and perturbed
- Produce and convert videos to GIF animations (open format, no proprietary codecs involved)
- The more time we have in our hands, the more we can embellish and extend to make the work stand out
Future Directions
- ISBI Progress
- instantiations
- evaluations
- comparisons (overlap and Specificity)
- text and figures
- Thesis
- structure and content
- time and planning