PDF version of this document

Models of Shape and Appearance

Statistical models of shape and appearance, which are often referred to as combined appearance models, encapsulate variation across the set from which they are built. To construct these models, one needs to establish dense point-to-point correspondence across all images in the set, thereby highlighting analogous structures.

Fig. 1. Appearance models showing the effect of varying the first, second, and third model parameters. Each of the models is subjected to variation of at most $\pm2.5$ standard deviations.

Construction of appearance models involves a stage where variation in shape and texture (image intensities) are learned in turn. Shape can be represented as a vector $\mathbf{x}$ while texture represented as a vector $\mathbf{g}$. Both shape and texture can be directly controlled by models of the form

$$\begin{array}{cc} \mathbf{x}=\mathbf{\overline{x}}+\mathbf{P... ...=\overline{\mathbf{g}}+\mathbf{P}_{g}\mathbf{b}_{g}\end{array}.$$ (1)

In the formulation above, $\mathbf{b}_{s}$ are the shape parameters, $\mathbf{b}_{g}$ are the texture parameters, $\mathbf{\overline{x}}$ and $\overline{\mathbf{g}}$ are the mean shape and texture while $\mathbf{P}_{s}$ and $\mathbf{P}_{g}$ are the principal modes of shape and texture variation respectively. In practice, there is a tight correlation between shape and in intensity so a combined statistical model of the form

$$\begin{array}{cc} \mathbf{x}=\bar{\mathbf{x}}+\mathbf{Q}_{s}... ...\mathbf{g}=\mathbf{\bar{g}}+\mathbf{Q}_{g}\mathbf{c}\end{array}$$ (2)

appears to work even more gracefully, integrating both sources of variation. The model parameters $\mathbf{c}$ control both shape and texture simultaneously and $\mathbf{Q}_{s}$, $\mathbf{Q}_{g}$ are matrices describing the modes of variation derived from the training set.