Progress Report
August 31st, 2005
Progress and Agenda - Overview
- Perturbation framework
- randomisation corrected
- further investigation of sensitivity aspects
- Evaluation experiments
- overlap
- specificity/Generalisation
- comparison
- Thesis Discussion
Experiments - Recapitulation
- We re-registered the set of IBIM-associated brains
- We then applied a series of perturbations to get sets of mis-registered data
- We can subsequently evaluate these multiple sets
- By doing such an evaluation, we can:
- demonstrate the correlations between mis-registration and our registration assessment methods
- (more specifically) compare overlap-based measures with model-based measures
Random Knot-Point Placement
- Started with the observation that there were non-random distributions
- The generator of knot-points had tendencies to place points at random positions, but adjacent to imaginary grid lines
- This flaw is now resolved (with the cost of an efficiency penalty)
- A couple of examples are shown in the next slides
Random Placements in Warps - #1
A colour-coded directions figure, which reflects on movement of points
Random Placements in Warps - #2
Yet another figure showing the direction of the perturbation applied at any given point
Generalisation Sensitivity
- Continued work on investigating sensitivity
- Curves were plotted which show Generalisation sensitivity
- For the sake of comparsion, Specificity sensitivity is shown too
- The aim of these plots:
- discover the effect of the number of modes
- investigate the variation of sensitivity as registration degrades
Specificity Sensitivity
For various number of modes, sensitivity of Specificity is shown
Generalisation Sensitivity
For various number of modes, sensitivity of Generalisation is shown
Overlap Measures - Progress
- Formattting issues quickly resolved (PNG, JPEG and Analyze)
- Structure and naming-conventions make overlap processing/evaluation quicker
- Estimates of variance on the overlaps can now be retrieved as output
- Implementation the Dice coefficient (
Dice = Intersection / Mean Volume)
- Implementation the Tanimoto coefficient (
Tanimoto = Intersection / Union)
Specificity/Generalisation Measures - Early Progress
- Stage which involved generating sets of perturbed sets (impending)
- I was offered help in finding the necessary computer power
- The scale of the experiments is no longer a peril
- Experiments can be extended
- Generated data accommodates for extensions (owing to excessive data that gets generated)
Evaluation Dip
- Looking at some preliminary results, it appears as if we have resolved (at least) one problem
- Previously, we spotted an evaluation 'dip' at the start
- Put differently, when plotting Specificity versus the degree of mis-registration, there was a slight 'anomaly' at the start
- To resolve this problem we:
- forced a very slight re-sampling error at the start
- revised the perturbation method
Earlier Evaluation Steps
- Specificity and Generalisation curves produced at an early stage
- Should be considered a sanity check
- Span a wider range than is necessary
- Explore a wider range by sampling the curve at many points
Earlier Evaluation Steps - Ctd.
- Allowing better selection of sample points to be made
- There will be 40 such curves in total - 10 instantiations x (Euclidean + 3 shuffles)
- Can generate at most 2 such curves per day, which will be enough to complete 10 instantiations and make the curves smoother, i.e. 'refine' as much as time permits
Specificity Evaluated
Specificity - shuffle distance 5x5, 15 modes
Generalisability Evaluated
Generalisation - shuffle distance 5x5, 15 modes
The Subsequent Step - Shuffles Compared
- Extension of previous figures (note range of mean pixel displacement)
- Includes only the first instantiation of pertubed sets
- Can probably finish work on 10 such instantions within a month, which makes the wishful plan practical
- Difficult to grasp why the curves are not monotonic
Shuffle Distances Compared - Specificity
Various shuffle distances and corresponding evaluations of Specificity
Shuffle Distances Compared - Generalisability
Various shuffle distances and corresponding evaluations of Generalisation
Shuffles Compared - Summary & Conclusions
- Just to add context, the 'recipe' so far has been:
- take a set of 37 registered brains
- mis-register them
- measure Specificity and Generalisation
- Can reliably distinguish a good registration from bad one in the range of 0-3 pixels of mean deformation
- When going beyond that extent of deformation, the brains look a little odd
Shuffles Compared - Ctd.
- It is hard to discern bad registrations from very bad registrations
- For practical application, only a sensible range is worth considering
- Would have been nicer to see a bit more linear response
- This non-linear curve requires (begs for?) a valid explanation
- More worrying is to see non-monotonic behaviour
- Curves might look better once we have more results to average over
Finer-level Technicalities
- Building of Analyze-formatted images for overlap evaluation
- Single slices treated as 3-D objects
- There are 8 main directories, each of which has the same structure as ever before, i.e.:
- /images
- /label_02_whitematter_left
- ...
Finer-level Technicalities - Ctd.
- Each parent directory was named based on how much deformation had been applied
- The range investigated for extent of overlap is 0 to about 4 pixels in terms of mean pixel deformation
- The actual extent of deformation was intentionally disguised
- Perturbation of 0 is re-sampled from the originals so that we compare like with like
- Large-sized chunks of data, even when compressed
Overlap - Initial Results
- Measures of overlap seem to decline linearly
- Visual results shown in the next few slides
- Can use the results to argue that label overlap is related to Specificity and Generalisation
- Shown are the Tanimoto and Dice overlap measures versus the average pixel displacement
- A small fluke had values reversed at times, i.e. plots should be 'mirrorred' where indicated
Dice Overlap
Dice overlap decreases as registration is degraded (numbers need reversal)
Tanimoto Overlap
Tanimoto overlap decreases as registration is degraded (numbers need reversal)
Overlap - Further Explanation
- A couple of figures were produced to show overlap versus mean of pixel
displacements (rather than just obscure numbers corresponding to warp magnitude)
- Construction of a a plot to show the relationship between Tanimoto (complexity) and the shuffle distance (window size 7 by 7)
- Parameters and measures chosen arbitrarily, not aiming to get results that best suit our arguments
- There would be at least 24 plots that we can look at (Euclidean and 3 shuffles X 6+ overlap calculations)
Overlap and Specificity
The correlation between Specificity and overlap across labels
Corrected Overlap Figure
Corrected curve for Tanimoto overlap - overlap going downwards as registration degrades
Registration Assessment: Monotonic Increase in Specificity
- Nearly 4 instantiations have been completed by now
- Different perturbations applied to the registered brain set
- It seems as if, on average, we will be seeing monotonic curves for Specificity, as hoped
- The first instantiation did not have a monotonic response when perturbation exceeded some high values
- 4 out of 10 instantiation will have been completed tomorrow
- The remainder may take a couple of weeks at the least
Towards Thesis
- Form 11 meeting next week
- Working on continuation report as a starting point, but changing the content completely
- Thesis template was shared among the international LyX community and now available via the LyX Wiki
- Possibly good for the publicity?
- Already shared and used along several other Ph.D. students
Thesis Planning
- Current thesis is a 'placeholder' with template and general structure in tact
- This was a first important step towards focus on just textual content
- Figures, tables, equations, etc. should not an issue
- Thesis rough draft (PDF)
- Later slides outline what lies therein (the gist)
Thesis - Possible Structure
- Included some 'skeleton structure', which ought to make a cohesive flow of arguments
- begins with a model-based/MDL objective function
- resulting models and registration
- improving efficiency including reference to subsets in construction of optimal shape models
- carries on to 3-D registration and automatic construction of appearance models using registration
Thesis - Possible Structure - Ctd.
- Skeleton (continued from last slide):
- describes the evaluation of appearance models
- assessment of registration using the method
- results of model evaluation
- description of a perturbation scheme and some results from evaluation of non-rigid registration (in the absence ground truth)
Thesis Planning - Videos
- Plenty of separable videos
- registration in 1-D
- automatically constructed models in 1-D
- automatic construction of shape models of the hand and the bump
- 3-D registration in VXL
- model animations of automatically-constructed models of the brain
- face models evaluated and perturbed
- Produce and convert videos to GIF animations (open format, no proprietary codecs involved)
- The more time we have in our hands, the more we can embellish and extend to make the work stand out
Future Directions
- ISBI Progress
- instantiations
- evaluations
- comparisons (overlap and Specificity)
- text and figures
- Thesis
- structure and content
- time and planning