fter persistence and a new type of mask put in place (with a little trick to penalise for miscorrespondence around the top) I have been getting good validation results again, with over 98% for the first 50 pairs I have tested (just one mistake some far). Some of the harder surface pairs have not been reached yet.

In an attempt to master GMDS as cross-identity discriminator I have further cut down the process, which now operates through a pipeline of reverse dilation, as shown in the first image beneath. The second shows a broader mask working upon false pairs and mostly failing, which is probably what we want. In terms of performance, as long as the true pairs are similar enough (some are far harder than others, e.g. expression variation), we are able to get decent results. Running this on the whole database again would not help beat results that we got around December, though. Several experimental modifications are hard to identify as means of reverting back to old versions. A lot of those were made to facilitate diffusion as a supported (surrogate) option.

I have reduced the number of points in GMDS to just 10 because it seemed as though it would significantly speed up the process without compromising too much in terms of performance. The experiments were then run on “hard” pairs, giving verification performance of almost 90%, depending on the method used.

In an attempt to understand how well one can do with just 10 samples in GMDS (because it is faster and insensitive to small changes) I took some of the hardest classification cases and ran repeated coarse GMDS on them, reaching a success rate of only 75% or thereabouts. This falls short because we are assuming that only for false pairs will we have improper correspondence, whereas for true pairs everything will be perfect. As shown in the images produced from the same false pair, sometimes a good correspondence is found, but usually not (symbolised by white colours in the dots, e.g. in the third example from the top, on the right hand side).

The example at the very bottom shows that even for different identities a decent correspondence can be found, which gives rather low stress values.

In order to mitigate or altogether annul the effect/artifact of flipping, I have made the two surfaces asymmetric, with centres of the eyes removed because there’s too much noise there, depending where the eyes look and how open they are (in some surfaces, the eyes are deliberately moved to challenge algorithms).

Several empirical results show that increasing the number of vertices, even doubling them, still seems not to help much in any way. So, I have isolated some difficult cases and I am trying to cut out the sources of ambiguity.

espite the fact that I cannot rollback to old versions of the algorithm — those that worked much better — I have run experiments on half of the entire Texas database. The performance was vastly inferior because of changes that I have made over the past couple of months.

Over the past few days I experimented with other smoothing schemes, as a colleague once suggested that I do. The improvements were very minor once performance was assessed with some ROC curves (few dozens of pairs). The general idea was, by removing areas or artifacts that distract from identity-related entropy, we can improve overall performance. The impediment has always been that, given too lenient and sensitive a measure, the items being compared are not identity-related but pose-related, noise-related, etc.

In my most recent experiments I have been looking into more kinds of masks, but none so far offers a magic solution with state-of-the-art performance.

Symmetric mask where eyes are removed and the nose tip too, in order to accentuate topology and delve into areas more “stable” than eye surface

Examples of correct matches where the mask is symmetric (the “classic” mode)

Examples of correct matches where the mask is cut at nose level

Examples of correct matches where the mask is intentionally asymmetric (to avoid flipping over)

I decided to tweak this further, testing GMDS for surfaces with holes.

Dealing with the existing algorithm, which performs more weakly than months ago (due to experimental changes that are hard to selectivity rollback), I have some new results. I have tested it with and without support (difference in dilation/distance for FMM cutoff), but in both cases the ROC curves were too disappointing to be worth plotting and showing. The approach does not seem suitable for good performance to be reached. With or without the most performance-promising algorithm, the general observation is that GMDS does not deal well with these holes (connected torus-like shapes).

Example of pairs of surfaces where eyes and nose tip got removed

An increasing level of geodesic dilation where eyes are cropped out but not the nose tip

ROC curves for the above

RECENTLY uploaded a report exceeding 500 pages. The report has been narrowed down to include some of the better results that we have and some of the more important points worth getting across. A lot of the text is new and a lot in the content and organisation can be improved given more time. This too will be uploaded soon.

I’ve been running further experiments while the report was being worked on (still work in progress) and some interesting results were reached when particularly fast configurations of the GMDS algorithm were attempted, giving (so far) perfect classification for 30 pairs.

Previously, in experiments shown in the report, verification performance reached through GMDS was limited and primarily impeded not only by failures at the topology level but also finer-level issues where the 3-D surfaces provided not enough information to correctly distinguish between identities. A lot of the topological failures were averted by further optimising the algorithm such that support is made broader.

The first ROC curve shows the performance where only GMDS is used and the mask omits much of the cheeks, whereas the latter mostly includes them. By altering the configurations to the best we’ve found so far, significantly better performance can be assured (with GMDS and 3-D alone), but it would be nowhere near the accuracy levels we compete against. Cross-individual entropy seems to lie somewhere in the combination of many 3-D features, but accurate segmentation is not easy on smooth surfaces.

HE task of comparing atomically-meaningful 3-D surfaces is not simple. The plethora of diffusion-based techniques have not managed to overcome the drawbacks and ever surpass the performance attained through geodesic distances. The main issue, to summarise this very briefly, is that when the differences are very small between one subject and another (no topological differences) there is not enough that can be done to distinguish by kernel-based score.

I have meanwhile returned to geodesics again. Based on literature surveys from a few years ago (I read three with great interest), for 3-D in isolation the verification rates are almost always worse than for 2-D (and of course 3-D plus 2-D), which brings up the question, what type of performance levels are expected from 3-D alone in order to make a technique publishable? With our best methods we hover around the mid-nineties. With further refinements that do not require 2-D data we can make further improvements, but there are inherent difficulties. For example, as I showed a colleague at the lab some pairs of different range image (in 3-D) she was unable to tell if there were the same or not. When the similarity is high it becomes almost about gut feeling or guesswork, even for verification as opposed to identification (one-to-many or many-to-many).

Additionally, spectral methods degrade poorly compared to geodesics, as shown in the images below (first a meaningful slicing, then improper).

Spectral examples

Spectral problems

I am currently working on a concise report about this project (which has exceeded a year in duration so far). There’s a lot more coming. I also added some list of side projects (totally unrelated) I’ve been working on recently. I hope to focus more on ERC-funded work and not whatever pays more (industry pays more than academic). But all these are a matter to be discussed another day.

COLLEAGUE has said that diffusion-based methods were never quite so suitable for the task of analysis of surfaces where differences are very subtle. However, even when poorly adjusted, performance can be somewhere at the range of 80% recognition (3-D only, we never use 2-D). It is not clear how to divide the face and one suggestion made at the lab last week (when face-to-face meetings took place) is that smoothing methods should be changed and parts that are problematic removed altogether to reduce noise-to-signal ratio. With diffusion-based methods, performance exceeding 90% should be attainable, but it still trails behind some other methods that we tested.

I have read a lot of literature (5 papers) that presents surveys with performance benchmarks. For 3-D alone, the recognition (verification) results are not so high, at least based on surveys from a few years back. I’ll further refine diffusion-based methods as more capable computational servers return online (after over a week of partial downtime).

For context, see this previous post and some example results (GMDS and ROC curve).

VER the past few days progress has been made primarily by exploring diffusion-based methods, having come to the point where geodesics cannot quite have their resultant performance improved any further. So far I got just one mistake in the GMDS-based approach and none in the other FMM-based approach, so it is looking similar to what I saw before. If I can rid myself of all errors (even in hard cases), that will be great, but it depends on the inherent limitations and strengths of the methodology, not the will.

The latest experiment showed comparable performance (w.r.t. prior experiments) but it is a lot slower, so I switched back to 2000 vertices and tried 300 points (to find correspondence for) rather than just 100 as before.

This aforementioned approach did indeed seem to improve things somewhat, but it would take longer experiments to provide good empirical evidence for it (need for harder pairs to distinguish based on many false negatives/positives). The only pair that causes trouble at the moment (among about 60 pairs that are being tested) is one that looks similar but is actually of different subjects. GMDS also detects it as being similar enough to fall near the border but at the side of true pairs rather than false ones. Let’s just take a face example (the methods are generic and can be applied to other data).

Some of my more recent modifications did not succeed at resolving the more problematic cases — ones where pairs are too similar/dissimilar to make an accurate assessment (it would be interesting to know how other methods cope with those as even a human observer would struggle, despite our brains being well wired to recognise faces). Doing 2D+3D or just 2D might in some sense be easier than 3D only, depending on the methods tested.

The other issue is, very consistently I find that GMDS is outperformed by a simpler FMM-based method that I implemented, without exceptions (the latter is a lot faster). But the mask continues to be modified in accordance with prior results and some observation of the stress (shown in different shades of grey upon points inside Voronoi cells) studied. If we hit a recognition barrier at around 97% (depending on the datasets), then perhaps using a mixture of methods would help, e.g. shape descriptors where the photometric data is encoded as geometric. I will revisit heat kernel signature and see if I can make those work better than before, then incorporate newer code like the stuff most recently published.

Now, moving on to heat kernel signatures we get some encouraging early results. With improvements to the code and to the masks, I have reimplemented the diffusion-based thresholding around landmark points and preliminary tests suggest high recognition performance. However, at this stage, the code is not stable, so it takes a lot of effort to produce a ROC curve. Performance of the diffusion-based comparator seems to have been vastly improved since last month, but there are still some issues to overcome. Heat kernels, being less dependent on the plane’s surface than geodesics calculated upon triangles, sometimes leave isolated patches that penalise and significantly increase the dissimilarity score, removing much of its signal. Maybe both distance types can be fused to resolve this in an ad hoc fashion. The first image just shows what happens if the surface if fragmented, the second shows what happens when it’s all in one unit. The third and fourth images provide examples of diffusion-based cutoffs leaving small ‘islands’ that pose a challenge.

Over the past two days I have running some more experiments that look at what can be refined for better performance. At some adjustments of the parameters it seems to hit the sweet spot for a couple dozen pairs, but then the errors start creeping in. Basically, pairs belonging to different people do not entail big enough a penalty, even when they are clearly quite different. I am getting closer though, to the extent one can given GMDS as the instrument of choice…

No classification mistakes were made after I had adjusted several parameters appropriately. Having explored high density sampling, I found that it works reasonably well at around 2000 vertices, 20 GMDS operations on each pair, and as many as 30 levels in the multi-scale approach (this does not take long with the C++ implementation). The performance achieved is quite satisfactory and after trying all kinds of masks I ended up with a new type of mask, one which is intentionally asymmetric so as to help GMDS not flip any faces over (around the Y axis, horizontally rather than vertically), thus always comparing like with like.

I have made lots of progress although much of the progress was not documented in a detailed fashion. This was done in order to make improvements more rapid and not too dependent on formality.

There is a lot more room for improvement, so these results are just a glimpse at the sort of level easily attained. Shown are the results from the latest experiments.

FMM-based method ROC curve

GMDS-based method ROC curve

More images and explanations will now follow.

A 20-layer approach before buxfixes

2 pairs after bugfixes

Mismatches after some bugfixes

Correct matches after some bugfixes

A 20-layer coarse correspondence check

Same as above but with smaller mask

Same as above but with further improvements

Currently I am looking at whether or not doubling the number of vertices again will lead to improvements. Previous experiments suggest that at a certain point it might leads to exacerbation.