Progress Report

January 31^st, 2006

Method of Delivery

• Content: developments throughout the period of time which precedes January 20^th
• Listed chronologically
• Used for recapitulation of points last explored
• Ends with some dates and key steps ahead
• Might need closer involvement from various parties

General Notes on Starting Point

• We were dealing with entropy of simple Gaussian distributions
• It was obvious there was still something wrong
• When the displacement of the dynamic distribution (one remains static) is zero, the two distributions are, by definition the same
• The entropy calculation functions were over-customised to deal with the real data
• Having found two errors and fixed them, we finally got 'sane' results
• Here is the output from some fairly coarse 'toy data'

Entropy of Two Simple Distributions

Larger Experiment: A->A, A->B, Overall Entropy

• The curves look much 'cleaner' given a large enough sample
• In this particular case:
  • number_of_examples_static=50; % AKA training set
  • number_of_examples_deformed=1000; % synthetic set
  • numbers_of_dimensions=100;
  • number_of_repetitions=10;

Larger Sample for Entropy Graph

Further Progress

• These plots at last looked sensible
• Work was done on visualising the distributions

Gamma Variation and the Effect on Entropy Curves

• Parameters, as agreed at an earlier stage:
  • large number of dimensions: 1,000 (10,000 would have taken too long)
  • points in dynamic distribution (synthetic set): 1,000
  • points in static distribution (analogous to training set): 50
  • sample from dynamic distribution (for estimation of distances to self): 50
  • 10 repetitions
  • 10 levels of increasing displacement
  • Gamma = 1 , 2 , 3 , 4

Gamma Variation - A Few More Notes

• Plotted the same entropy curve
• Head-to-head, tail-to-tail - eventually plotted on the same figure
• What we expect to change: slope, error bars, constant (although subtractions annuls its effect)
• Shortly afterward: generated plots of sensitivity for these 4 curves

Gamma Variation - Unimportant Notes

• Shown are also cloud representations of the data in question
• Even the largest among the displacements which we apply is not obvious enough to catch the eye
• Difference, effect of constant goes away
• Values of A->A and A->B change, but not their difference

Gamma Variation and its Effect

Synthetic Data clouds in Space

Offset of 5 units in axes X, Y, Z for visualisation purposes

Larger Sample: Synthetic Data clouds in Space

Offset of 5 units in axes X, Y, Z for visualisation purposes

Sensitivity Curves and Gamma Variation

• Data could be re-used for experiments that exploit large data sample
• Fortunately, one can get a large (~85MB) data dump before quitting MATLAB
• Able to re-use the data generated and produce sensitivity curves and the like rather quickly

Wrong Gamma Plots

This plot is wrong as it shows the wrong parameter altered

kNN and Gamma - Effects on Entropy Sensitivity

• Realised that we had the wrong parameter changed in the sensitivity plots
• Rather than changing Gamma, the program was changing K (as in kNN), which is still intersting
• Finally produced correct sensitivity plots which demonstrate that there is no change in sensitivity, even if one goes as high as Gamma=10
• Still have some nice plots to show the effect of varying K (plots I produced by mistake)
• The functions take very many arguments, which makes them confusing to use (later to be corrected and improved)

The Variation of K - Low Values

K in the range 1-4

The Variation of K - Larger Values

K assigned the value of 2, 4, 5, 8, and 10

Early Observations

• I had a hard time getting encouraging results
• The curves that we get in practice differ from these which we saw in the simple example
• Among the causes:
  • As indicated before, distances from the synthetic to the the training set are much larger than synthetic-synthetic distances
  • The value of Alpha is verging one because of the nigh number of dimensions
  • In the equation of entropy we multiply by 1/(1-Alpha), which is going to be a huge number

Distribution Size Increase and Resultant Entropy

• Dealing with just 100 synthetic images
• Thoughts About Results
  • Notes about the results refer to both the real and mockup case
  • Distribution expands -> inner distances increase, unlike positional movement of both samples
  • Synthetic to training likewise, reasons is the same as above
  • Training images are more 'different' from synthetic set to itself, thus the decline in the overall entropy (pace of change differs)

Overall Entropy

Entropy of Synthetic Distribution to Self

Entropy of Synthetic Distribution to Training Set

The Effect of Increasing One Distribution's Size

Specificity in the Corresponding Set of Experiments

More on Entropy Calculations

• Obtained almost all the matrix data
• Hoping that nobody switches off any of the machines at any point throughout the that time
• We would be on the 'safe side' otherwise, in terms of data collection
• Sent Carole all the paper files, programs, and data (about 100 MB)
• Rechecked the experimental results that were sent beforehand and all seemed okay
• Drawn a diagram to explain this and accompany previous correspondence, which was generated in haste

Reasoning About the Behaviour of Entropy Plots

Combined Entropy Curves

• All the entropy curves with error bars, which are derived from the variation in graph length

Entropy Curves from the Real Data

Entropy and Specificity Combined Figure

• Shown in this figure are the entropy and Specificity of the real data
• No scaling of any kind has been applied

Entropy and Specificity (Real Data)

Plots and numbers are not scaled

Resources Available

• Needed brute-force for calculation that are more statistically valid
• Finally I got hold of over 30 computers, so I began running 3 more S_i's for each value of displacement
• We could end up with 4 S_i's (148 images) for each sample point

Progress on Si's, Distribution Size Variation

• Obtained a full second batch of S_i's and verified that all matrices are okay
• With our artificially-generated distribution data, I re-ran the experiments which involve distribution size. I happened to find that, as I suspected, a single apostrophe was the cause for the curves not meeting at 0. As you can see, the tails now meet. I vary the size of the dynamic distribution from %10 to 100% (identical size). By mirroring the image, you can see the graph that you otherwise expected to see. One distribution shrinks rather than grows
• This is actually similar to what we saw in initial plots, which I produced from the real data 3-4 weeks ago (the two curves did not meet though)

Varying the Size of One Among 2 Distributions

Older Plots of Entropy

These are probably ill-computed

The Effect of Varying Gamma

• Same as before, but a variety of assignments to Gamma are investigated.

3 Entropy 'Components' and Gamma Variations

Explanation of Results

• The following might be worth pointing out:
  • There is a reason why, for the real data, the entropy will never start at 0
  • You are not comparing like with like
  • The distribution which is the training set and the distribution which is the synthetics derived from that optimal model are different
  • The model is lossy after all
  • Thus, the old plots for real data remain valid
  • They also lie in agreement (logically at least) with the mockup case

Entropy in Expanding Distribution

• An interesting plot which shows the intersection of the two entropy values

Expanded Range for Distribution Size Variation

Deadlines and Milestones Ahead

• TMI submission
• ISBI Registration (and camera-ready version)
• MICCAI deadline March 10^th - suitable for introduction of entropy as a substitute for Specificity and Generalisation
• MIUA deadline: April 24^th
• January 27^th - a 15-minute Monday talk to make up for absence in Student Talks

Deadlines and Milestones Ahead - Ctd.

• February 10^th-14^th: CVPR preliminary decisions and rebuttal period, finalised in March
• Thesis progress can resume
• Further exploration of entropy, normalisation