N the previous post on this subject we looked at the masks used in a GMDS pipeline tailored for recognition purposes. The question now is, what would be a constructive way to progress from the conclusion?

Well, the goal is to beat the competition and do so with methods of a particular kind — the kind we advocate — which seems achievable but requires a lot of tinkering, seeing where and how mistakes can be resolved/avoided.

An additional experiment, taking about a day to complete, shows not much promise. Its goal is simply to compare the performance of GMDS with the new Fast Marching Methods-based measures (faster) when all parameters are kept consistent across runs. With many rings, many points, and many vertices, recongition performance is relatively poor because of the hard dataset, as demonstrated by the ROC curve. The point to note though is that in hard cases GMDS is outperformed by the other approach. One question is, are there any measurable quantifies (other than stress) resulting from GMDS and capable of assessing similarity?

GMDS multiple runs

FMM-based

There are some additional results from the last set of shallow, comparative tests. By applying the same experiment’s parameters to test an antiquated triangle-counting approach and a best fit GMDS approach (rather than average over multiple runs) we get two more ROC curves.

Triangle counting

GMDS best fit

Finally, using this same difficult set (where problematic cases are included) we get a ROC curve for the standard GMDS approach.

Simple GMDS

In GMDS one could either work with and L₂ norm (which is what we do right now) or L_infty. In fact, if one takes the log of the distances and apply GMDS, one does, in a sense Lipschitz embedding. The diffusion distances could also be used within the GMDS framework. In fact, with better interpolation properties, one can interpolate the eigenfunction before integrating the distance itself…

I spoke to a colleague about L_infty and I now attempt to compile it on GNU/Linux (not done before). It would be interesting to know if GMDS been tested where D is computed based on spectral properties (not as geodesics). There an IJVC paper with Sapiro in which this is done; in it, the authors are also finding symmetries that way.

HIS post is part of a series that explores the potential of comparing surfaces using GMDS, or generalised multidimensional scaling.

A 15-level multi-resolution approach has led to no classification errors being made, at least thus far. This may have helped prevent the convergence near local minima, but it is painfully slow, especially when the C++ implementation does not get used. I have looked into more ways to widen the separation between correct pairs (same person) and incorrect pairs. I have begun looking at the impact of two factors; one is the size of the mask used prior to GMDS (or dilation through iteration) and another is the number of multi-resolution levels. Based on an ongoing experiment, a very coarse correspondence/initialisation leads to something quite reasonable when the pairs belong to the same subject and everything is a lot quicker and a bit of a mess otherwise (see the first two images).

At 3 cumulative levels in the multi-resolution reproach, a false classification does not take long to occur, so I increased that to 15 and ran that at 3 levels of dilation from the three centres for half a day. In spite of the optimisation process taking a lot longer, performance was not good, peaking well below 90% recognition rate. Although the tested dataset is not large enough to draw conclusions from, the recent experiments seem to suggest that not that a multi-scale approach on its own cannot resolve frequent recurrence of misclassifications.

In order to better understand what weakens these measures I have taken a closer look at visual GMDS output. It seems as though the scores are heightened when real (correspondent) pairs of surfaces are not yielding the correct correspondence, even after a 15-level optimisation when the data is exactly the same except the mask size (as shown in the images).

In the past, taking the best fit among all matches was tried (and watched as secondary/surrogate in all of the recent experiments in fact), but it does not perform well as a discriminant on its own. If GMDS succeeds at finding the accurate correspondence 95% of the time in these circumstances, then in this case we need to rerun GMDS several times to exceed it in terms of recognition rates. The other FMM-based method (the one I created) achieved better recognition rates than that.

In order to test the mask type and its effect on performance I ended up setting all parameters to fixed values and running very coarse-scale experiments, first on the entire face and later on a masked subset of limited size, particularly focusing on rigid parts alone.

Results were interesting in the sense that they showed that, based on GMDS as assessment criterion, smaller mask around the fixed points do not clearly and unambiguously produce better results, at least not in the case of this dataset. The assumption we had about removing the non-rigid area may have been misplaced — inherited from other work.

In the next stage I will return to fine levels to get vastly better results.

Ideally, we should initiate these high-resolution triangulations by the result we get from the lower resolution. Currently, by default, there are 9 levels, adjusted according to m, the number of sample size (300 in the latest few experiments). It’s the number of levels in the multi-resolution hierarchy. At the finest level there are typically 15999 faces and 8101 vertices (in older experiments we had just about ~2000 and in recent ones there were ~4000). A low-to-high resolution is operated by default, but it is not sufficient for evading local minima. This should explain the decrease in performance and Initialisation with lower resolution result (iteratively) should solve part of the problems.

AD habits die hard. Good habits stay, so over time we do things more effectively. Today’s post may be relevant to post-doctoral folks whose work involves a lot of computation, including over-night or multi-day experiments.

6 or 7 years ago I wrote detailed posts about how I was using KDE to run experiments on entire clusters of Fedora boxes, over SSH. It has been a long time since then and nowadays, rather than use many dual-core boxes I just mostly use a pair of 8-core computational servers. The tricks are mostly the same and the scripts are largely reused from those that I wrote around 2004 (they are publicly available in this Web site). The procedures are mostly the same, but technically, a few things have changed and this post will detail them (aside from that, back then I did my Ph.D. and now I get paid to run these experiments).

In case of network issues, it is important to run everything but the computation locally. This means that code editing, for example, should be done locally. This reduces cursor/menu lag and prevents loss of work in case of loss of connection. It also assures that files get written to more than one place (local plus remote). Kate with KIO (in Kate) use slaves to enable editing of files over SFTP or SCP/SSH, so this functionality ought to be exploited for controlling experiments remotely. For other tasks, separate terminal windows (e.g. Konsole) should be opened, preferably with endless scrolling buffer, for each remote machine. GUIs can be created to enable quick adjustment and running of experiments. A good terminal will stay active and visible even when a remote connection gets closed, in which case, results can still be observed. In addition, it may help to have another terminal window connected to each remote machine in order to track load throughout runtime, as well as other things (one busy session may permit nothing else to be done on the same terminal). Here it is illustrated graphically, based on screenshots just taken.

Full screen view on workspace 8 (research):

Let’s break it down into the left and right monitors:

This left side of the dual-head display contains the tabbed-style view of program files that need editing. All those files are using KIO to essentially be seen as local even though editing them modifies them on the remote servers that I connect to (see terminals on the right). In addition, I used Java framework to create a GUI front end for the experiments. It is singleton for each server.

On the other screen I have this:

Shown at the top left is a window that’s invoked by the program when results are ready. By using KDE’s window-specific settings I can force all such windows to always open in workspace 8 (research), so even if I am busy with other tasks the windows will quietly show up where it belongs, sometimes along with dozen other windows that need attention later. On the right side there are terminals connected to the computational servers. One currently gets used to rsync the code across across servers and the other is tracking server loads.

So, this is pretty much how I use workspace 8. I previously explained how I use workspace 2 (there are 12 in total). KDE makes it easy to multi-task without getting distracted.

EGACY of one’s life may typically matter to a person when death is near. That’s partly because last/recent memories persist better than old ones. Legacy is also what remains in visibility after a person departs from this world, having first emerged in it through conception. But legacy need not be associated with depressing things such as being deceased. Legacy throughout one’s life can be seen as the work that’s left to have impact when one moves from one area to another, from one field of work to another.

In older terms, publications and books were seen as a form of legacy. In a digital world the importance of these becomes more questionable and long-term persistence almost dubious. Work that is done in the disciplines of science and technology may matter a lot at the time of publication/invention, but only years later that work becomes uninteresting due to irrelevance. There are of course exceptions such as key, landmark papers (Charles Darwin’s for example) and immortal series such as Cosmos, but the vast body of work will only have its 15 minutes or fame — if any — and thereafter be shelved.

Activism is different in the sense that it has broader impact due to scale of reach (like target audience). Those who fought SOPA, for example, achieved a great deal and did this not for profit but for ideology.

In my younger days as a researcher I strived to publish papers and had my name put on half a dozen of them around 2005 when I was completing practical work on my Ph.D. In early 2006 I stopped submitting papers and also ceased to attend conferences. These had low impact compared to my sites, their target audience was small (mostly departments in the same field as mine), and the sense of accomplishment was not high. It was then that I turned to activism and spent the majority of my day dedicating energy/effort to good causes, even if it comes at the expense of a paying job. There have been no regrets, except perhaps regrets that I had not started doing this sooner.

The life of an activist is a lot richer than the life of a compulsive businessman. Richness cannot be properly measured in terms of monetary currency and some people are so poor that all they have is a high bank balance. Over the long run, history teaches, activists have a memorable legacy; the latter have not.

GMDS for surface pairs comparison

S natural succession to the previous experiments, I compromise speed and increase the number of points in GMDS from 10 to 50, hoping to see performance improved noticeably. So far, based on results that are coming out, no classification mistakes are being made, but more pairs need to be tested overnight before meaningful conclusions can be drawn.

As expected, increasing the number of points in GMDS helped improve performance, at the expense of speed of course.

They will now be increased even further to see how good an improvement can be made.

I have begun running GMDS-based recognition experiments with 300 points rather than the originally/already-tested 50. So far, based on 18 pairs, the classification is perfect (albeit the process is slow). It will be interesting to see how long it runs before the first error occurs. There was a talk back in December about a new server potentially arriving, with a lot more x86 cores in it.

Following the first mistake of the classifier, the ROC curve looks like this.

The next step doubles the density of the triangulation to see how major an improvement in performance it can entail (we never tried as many as 8,000+ vertices in such experiments).

With ~8,050 vertices the results are a bit baffling. Both computational servers got rebooted midway (memory upgrade), so there is not much data — to say the least — to base these results on.

By summing up all the GMDS stress (multiple iterations) we get iffy results.

By taking the optimal fit (over all iterations) we get something better.

In summary, despite the small size of the sample, it does seem to suggest that using a triangulation too dense leads to poor results or does not lead to optimal performance. I’ll explore other route towards improvement; adding density just slows things down considerably.

OLLOWING this previous shoutout I was speaking to a colleague about a more reliable way of comparing two triangulated surfaces as it is required once the masking gets done and good alignment is attained.

I spent some time searching around the Web to see if someone made a robust and accurate way of doing so, rather than rely on ad hoc methods that I wrote on my own. The triangles already are reduced for analogous regions to overlap, but comparing on this basis (in 3-D) needs some cunning theory to be robust to slight changes in expression. MathWorks’ File Exchange hardly provides anything for this purpose. I looked through hundreds of submissions. There is a paper titled “Surface Matching: geodesic distance evolution PDEs on manifolds” and it links to http://www.cs.cmu.edu/~3dvision/meshtoolbox/executables.html. The page is no longer there, but there is a new working URL for it. “Overlap” is the tool closest to what we need (but not quite the best for the job). The comparison we do is not a topological one but one that should be sensitive to small localised anomalies, with just 4,000 (at most, for technical limitation in FMM) triangles. To get to the very high 90s in terms of matching rates (%-wise) I could use some advice/pointers. I have been reading “Geodesic Matching of Triangulated Surfaces”. It is a paper from A. Ben Hamza and Hamid Krim (North Carolina State University) and it demonstrates the use of Jensen-Shannon dissimilarity results. Wang et al. in “3D Brain surface matching based on geodesics and local geometry” seemed interesting and relevant. This paper, however, does not deal with surface-to-surface comparison of brains but with other problems. After a while it was time to explore older methods.

We experimented a lot with GMDS back in July until a few months later. It did not give sufficiently good results for two main reasons: 1) the separability in terms of scores was not good enough and 2) GMDS did not always find the correct correspondences, perhaps due to initialisation issues and other factors. GMDS gave recognition rates higher than 90%, but this baseline was surpassed by the newer implementation. As GMDS is a refinement mechanism, it would be useful to see what would happen if we iterate (i.e. re-run it after some other way of aligning the surfaces). I decided to give that a go.

The results so far look so-and-so, with more false pairs being run to smoothen the curve and draw better conclusions. I am merely presenting the results, irrespective of their quality, just the quality of the experiments used to arrive at them. What’s measured in this case is the best fit stress (best among all iterations), which means that the rest get altogether discarded. ~4000 triangles are used for FMM in this case.

Running further experiments to account for more example pairs has done little to actually contribute to optimism, so I will explore another approach in iterating over the set with GMDS (run overnight).

Taking the average GMDS score rather than the best fit (minimum) score leads to results that are not better. These results are significantly inferior to those attaind through the older FMM-based method.

Important enough to mention was the fact that in the aforementioned experiments — those with GMDS iterations — only the area around the nose was factored in because it surrounds the landmark point which can most accurately be determined. Experiments from last year, however, systematically showed superior performance when an estimation of eye locations form the basis for an expanded mask, so additional experiments will now be run with the expectation that they will show improved recognition performance (hopefully around 95% correct).

Several more experiments have been run to explore the use of GMDS in a supposedly more robust way to establish when an image pair comprises corresponding people. Starting with experiments that look at the use of 3 GMDS processes around 3 different centres, there is a best fit approach, from which the following ROC curve was obtained.

The same for average reveals similar performance.

By taking all centre points at once and performing GMDS on a mask around that we get the third ROC curve.

he same for average shows performance lagging considerably (in comparison with results from another approach).

It is a lot worse than the ~97% recognition rates we got some months ago, Time to step back and rethink perhaps.

COULD really use the advice of some people who are interested in computer graphics and vision — inverse but complementary fields, which are largely related to geometry and mathematics at large. The challenge is to compare surfaces based on their 3-D characteristics, using Euclidean, geodesic or another non-Euclidean metrics.

Here is what the surfaces look like.

The following image represents a performance ROC curve we have.

In simple terms, this means that we’re able to classify correctly about 70% of the time. Having summarised a year’s work, culminating in mostly unsuccessful experiments around diffusion (around 97^ recognition rate, the above is poor, So I return to working with geodesic distances as the Swiss army knife for measurement of distances. But the approach being explored at the moment is different and with further enhancement it can hopefully yield performance higher than 97% (matching rate). The sources of limitation are generally well understood in the sense that they can be visualised and overlaid on top of the image pairs. There is no trivial and reliable way to establish multiple landmark points around which to measure distances consistently, so I am trying another way, which at a very coarse level has a matching rate of about 80% (can be significantly improved soon).

The approach attempted at this stage involves triangle comparison post- and pre-marking, but the performance attained so far is not satisfactory.

The limitation is likely inherent in the measuring of distances in FMM and the sub-sampling that results from triangulation (see image for the size of triangles to be fully appreciated).

One particular weakness of the diffusion approach to masking is that it leads to holes in the data, which invalidates some of the measures that were used routinely beforehand. In order to fuse together both geodesic and spectral measures, we now attempt to get a more symbiotic approach that carves out surfaces based on geodesic properties and then uses spectral features on these surfaces. Since ordering does not exist (e.g. point-to-point correspondences), the histogram of images is used to describe the sub-surfaces (carved out around a known correspondence). By increasing the number of rings and bins in the histogram, the performance can be varied somewhat. I was running overnight experiment to test this and got some other appalling ROC curves, such as this:

The results from the last experiment were disappointing because they did not provide good separability. So the following morning I designed and started running an experiment that explores the potential of measuring diffusion distance between furthest points in the surface carved in accordance with geodesic boundaries (several rings). This too did not give good results. Generally speaking, diffusion distance as a measurement has not proven to be anywhere as useful as FMM so far (since December). It seems to be insensitive to small differences and it does not seem to degrade linearly, either. The next experimental design will explore another new approach, perhaps conceding the potential of diffusion being integrated into the framework’s pipeline.

So the question is this: given two surfaces that are geodetically craves around the surface, what approach would you use to compare them for similarity? We’ve tried a variety of known methods, but none seems to yield very encouraging results thus far. Thanks for any advice or pointers you may have.