PDF version of this entire document

Statistical Models of Appearance

Statistical models of shape and appearance (combined appearance models) were introduced by Cootes, Edwards, Lanitis and Taylor [1,2], and have since been applied extensively (eg [14,11,10]). The construction of an appearance model depends on establishing a dense correspondence across a training set of images using a set of landmark points marked consistently on each training image.

Using the notation of Cootes [2], the shape (configuration of landmark points) can be represented as a vector $\mathbf{x}$ and the texture (intensity values) in a shape-normalised frame represented as a vector $\mathbf{g}$.

The shape and texture are controlled by statistical models of the form:

$$\mathbf{x} = \mathbf{\overline{x}}+\mathbf{P}_{s}\mathbf{b}_{s}$$

$$\mathbf{g} = \overline{\mathbf{g}}+\mathbf{P}_{g}\mathbf{b}_{g}$$ (1)

where $\mathbf{b}_{s}$ are shape parameters, $\mathbf{b}_{g}$ are texture parameters, $\mathbf{\overline{x}}$ and $\overline{\mathbf{g}}$ are the mean shape and texture, and $\mathbf{P}_{s}$ and $\mathbf{P}_{g}$ are the principal modes of shape and texture variation respectively.

Since shape and texture are often correlated, this can be taken into account in a combined statistical model of the form:

$$\mathbf{x} = \bar{\mathbf{x}}+\mathbf{Q}_{s}\mathbf{c}$$

$$\mathbf{g} = \mathbf{\bar{g}}+\mathbf{Q}_{g}\mathbf{c}$$ (2)

where the model parameters $\mathbf{c}$ control the shape and texture simultaneously and $\mathbf{Q}_{s}$, $\mathbf{Q}_{g}$ are matrices describing the modes of variation derived from the training set. The effect of varying one element of $\mathbf{c}$ for a model built from a set of face images is shown in Figure 1.

Figure 1: The effect of varying the first model parameter of a facial appearance model by $\pm 2.5$ standard deviations.