PDF version of this entire document

Registration

ICP has a working implementation and is now capable of delivering back the 6 degrees of freedom which define a shift to apply to non-reference face data, typically non-neutral one (conventions may vary). What's required is a set of neutral plus non-neutral pairs, organised suitably for a data structure which treats pairs as even and odd items in the index of an array. In its current form which is extensible, the program takes pairs as input, applies the necessary pruning, and then aligns the two at the centre. Rotation and translation (no scaling) are applied and the non-neutral image is fit to the neutral one based on closest points. This process helps determine the resultant shape residuals, which it retains until the entire set of pairs is processed. As described in Mian's paper, the residuals are then vectorised; the program reduces the scale and complexity of PCA (to prevent running out of memory) based on 4 variables which define a sub-frame to sample points from. Each residual instance is uniformly serialised and then fed into a principal component analysis algorithm that yields very many eigenvectors. At this stage, with test data that deals not with real pairs but with pseudo-pairs, the output of PCA is not particularly fascinating. The IJCV paper does not show anything too spectacular either, but decomposition of the variation does capture - although roughly - movements of the mouth and then descends to less interesting properties. If a good model is built, then removing expressions by using it 'in reverse' ought to be doable, which in turn neutralises some variation that otherwise impedes identification. Implementation was sped up considerably while coding in a way that reduces spurious computations and vectorises some expressions to dodge excessive looping (which cannot be optimised on the go).