Expression Deformation

The baseline for this work is a paper from Mian's group [2]. In their paper ``An Expression Deformation Approach to Non-rigid 3D Face Recognition,'' http://www.informatik.uni-trier.de/ ley/db/indices/a-tree/a/Al=Osaimi:Faisal_R=.htmlFaisal R. Al-Osaimi, M. Bennamoun, and A. Mian explain some encouraging results from experiments that apply PCA to face images (the paper was also http://www.csse.uwa.edu.au/%7Eajmal/papers/ijcv08_faisal_auth_version.pdfpublished online for Open Access). This comprehensive paper from the group in question is 22 pages long in the raw form and about 15 in IJCV. The abstract describes an idea and quantifies some results using known benchmarks and the ``FRGC v2.0 dataset''. Then, the method is alluded to vaguely and not formalised until later. ``Most of the approaches in the literature are rigid,'' says the text in page 2, just before the overview which states: ``The main contribution of this paper is a non-rigid 3D face recognition approach. This approach robustly models the expression patterns of the human face and applies the model to morph out facial expressions from a 3D scan of a probe face before matching. Robust expression modeling and subsequent morphing gives our approach a better ability in differentiating between expression deformations and interpersonal disparities. Consequently, more interpersonal disparities are preserved for the matching stage leading to better recognition performance.''

The background section is followed by some classification of existing work, concluding with: ``Our approach also falls into this category i.e. non-rigid 3D face recognition.'' 1.1 presents a very good summary of related work and 1.2 a clear overview of the method and the ideas behind it, accompanied by a helpful diagram at the bottom of page 3 (Figure 1). The strategy is to use pairs of image of the same individuals, normalising them a bit, and then applying PCA to reduce the dimensionality that characterises expression variation.

**Figure:** Expression parameterisation in action (image from Al-Osaimi et al.)

Section 2 in page 4 starts by describing pre-processing steps that are essential yet specific to the limitation of the FRGC v.20 dataset. Page 5 starts presenting some visual examples of the approach, with some equations relating to PCA (along with more visual examples) in pages 6 and 7. The next stage will deal with registering images from particular individuals and then building a model from them. It ought to be stressed that registration gets done with just a partial face (whose most dynamic structures are mostly omitted using a binary mask), whereas modelling brings together the entire face as much of the variability is contained in the previously-occluded or masked parts (as they mostly interfere with registration by introducing ambiguities).

Section 3 begins to deal with some other experiments that are not just dealing with models in synthesis mode. The same dataset is being used (with about 5,000 3-D faces), but more data gets added to it. To quote, ``The dataset is composed of two partitions: the training partition (943 scans) and the evaluation partition (4007 scans). [...] The FRGC dataset was augmented by 3006 scans that were acquired using a Minolta vivid scanner in our laboratory.''

Parameters and set sizes (those which are included) get tested in very large-scale experiments that yield ROC curves. These curves help show how to set the different parameters and enable one to measure advantages of one algorithm over another. Page 13 has some comparisons to other methods from the literature, with numbers summarised in a chart.

This truly inspiring work is being partly reimplemented in order for something similar to be achieved in terms of results (see Figure $[*]$ for pose/expression correction). The reason for these overly specific descriptions of this paper is that it is being used as primary reference work on which to build upon, initially by mimicking an implementation.