PDF version of this entire document

Reproducibility Concerns

Mian's group does not seem to provide enough details about the 3 cases/subjects on which they train. Understanding of this requires careful reading and some extrapolation around the very dense text (it is a whole Ph.D. that's described therein). Currently, trying to get one's head around which images they actually used to do their experiments at the lab is hard. They claim to have had 3,000 images acquired at the lab, but no examples are given and the descriptions are vague at best. To a sceptic, any such thing means that obscurity implies deficiency; this strikes a nerve because if they basically test on their own data and use just three unique physical faces (with variation), there is no guarantee that the results can be generalised, so to speak. Moreover, there are repeated admissions of weakness and argument for the training of person-specific models, which seems to be a realm more capable of easy handling. At one stage it is explicitly stated that the PCA applied to 300 FRGC instances (enough to accommodate hyperspace of limited proportions) was rather useless, which led to thinking, might we expect to reproduce these results at all, despite the fact that the original researchers themselves had reported difficulties?