Massive dataset explored

Example face with holes remaining in the data

Another example

Same as above, different angle

Summary: Notes, tips, code samples, and pointers relating to FRGC (an ongoing series of posts)

HIS post provides some information of interest to those who may find themselves working with 70 GB of data and some programs [1, 2]. The latter is a FRGC Web site. The package comes with associated applications and scripts written in Java, C++, Perl, etc. The previous post about the dataset (FRGC ver2.0) offers a bit of background that is research-specific (relating to Dr. Ajmal Mian and his Ph.D. student Faisal R. Al-Osaimi), whereas the notes below are a bit more generic. This series of posts is not about statistical models of faces but only about the dataset. This recent message from Face Recognition Research Community contains MATLAB/Octave loader code for a data instance from the dataset, where each 3-D face weighs about 13 MB (compressed):

function [x, y, z, fl] = absload(fname) 
%ABSLOAD Read a UND database range image from file.
%   [X,Y,Z,FL] = ABSLOAD(FILENAME) reads the range image in FILENAME
%   into the variables X,Y,Z,FL.
%   FILENAME is a string that specifies the name of the file
%            to be openned
%   X,Y,Z are matrices representing the 3D co-ords of each point
%   FL    is the flags vector specifying if a point is valid

% open the file
fid = fopen(fname);

% read number of rows
r = fgetl(fid);
r = sscanf(r, '%d');

% read number of columns
c = fgetl(fid);
c = sscanf(c, '%d');

% read junk line
t = fgetl(fid); clear t;

% get flags
fl = fscanf(fid,'%d',[c r])';

% get x
x = fscanf(fid,'%f',[c r])';
% get y
y = fscanf(fid,'%f',[c r])';
% get z
z = fscanf(fid,'%f',[c r])';

% close the file
fclose(fid); 

This just handles one single image. There are many in the current collection:

find | grep .abs | wc
   4950    4950  257215

All of which are compressed:

find | grep .abs.gz | wc
   4950    4950  257215

To get a list of the 3-D faces:

find | grep .abs.gz | awk '/{print $1}' 1>~/files_list.txt

It yields something like the following:

(more…)

hroughout my career I’ve always had many eggs in the basket. I’ve usually had multiple jobs and I was never fired; I always succeeded in job interviews (since 2003), except the ones with Google, which came to me three time (I never approached them regarding a job). One thing I’ve learned over the years is that one must choose a job one enjoys, otherwise it’s a chore. I never accepted a job that I disliked. I have been working in two jobs simultaneously several times (simultaneously as in overlapping months/years), sometimes on top of already being a full-time Ph.D. candidate/student. I still work two jobs and I very much enjoy both; it’s like leisure as there is a sense of achievement. Besides all of this, as a hobby I maintain some sites that promote freedom; I was never paid for this. This is part of my reading of material; it’s like a learning experience which also proved beneficial to many others — those who share interests with mine. Being enthusiastic about freedom comes very naturally.

After many years wanting to be running an independent business on the side I’ve decided to start creating a professional site. The original idea was to come up with a new name (and domain), but after much consideration I came to the conclusion that giving visibility to a new name and new site would be a lot of work. As this new blog post from Forbes correctly indicates, reputation matters a lot when seeking business. That’s why I decided to stick with my surname and in the coming days/weeks there will be a formal announcement regarding my third job, in which the work capacity cannot be guaranteed (depends on clients). The focus is affordable scientific computing solutions that put the client in control. In essence, it is about spreading free/open source software and charging for the scarcity, which is skill and (wo)man hours. There is nothing unethical about it.

Together with some friends (I shall add people to the appropriate pages), a new logo, CMS theme, and a soon-to-be redirection (dupe of index.htm will ensure all the older pages remain accessible), schestowitz.com will soon have a sort of relaunch. The site no longer attracts about 3,000 visitors per days like it used to (back in the days when it was regularly updated), but we shall see if it takes off not just as a personal workspace with a lot of informal pages. I remain very much committed to all my jobs; starting something as my own ‘boss’ will just be something on the side.

From Al-Osaimi et al., IJCV 2008

Summary: My attempt to reproduce some of the results of F. Al-Osaimi et al. and furthermore improve them using other methods and different datasets (with a 3-D scanner at our disposal)

HIS post provides some background about my next (current) research project, which deals with non-medical applications. The previous project dealt with cardiac imaging and I’ve packaged that code and published it along with other data that may be useful.

The group of A. Mian has done some fantastic work recently on 3-D face recognition and I shall attempt to reproduce some results with a NIST database. In their paper “An Expression Deformation Approach to Non-rigid 3D Face Recognition,” F. Al-Osaimi, M. Bennamoun, and A. Mian explain some good results from experiements that apply PCA to face images (paper published online in September 2008 by a leading computer vision journal, but access is restricted, so there is no link, either… unless one uses this copy).

Since I have extensive experience with NRR, PCA, and statistical models in general, this project suits me better than some previous ones. I have done limited work on analysis applied to sets of face images that are only rigidly or affinely registered.

The paper from the group in question is 22 pages long in the raw form and about 15 in IJCV. The abstract describes an idea and quantifies some results using known benchmarks and the “FRGC v2.0 dataset”. Then, the method is alluded to vaguely and not formalised until later. The phrasing could be improved somewhat to avoid repetition, e.g. in the following paragraph containing parts like: “2D face recognition has been extensively researched in the last two decades. However, unlike 3D face recognition its accuracy is adversely affected by many factors such as illumination and scale variations. In addition, 2D images undergoes affine transformations during acquisition. Moreover, handling pose variations in 3D scans is more feasible than 2D images. It is believed that 3D face recognition has the potential for more accuracy than 2D face recognition (Bowyer et al. 2006). On the other hand, the acquisition of 2D images is less intrusive than 3D acquisition. However, 3D acquisition technologies are continuously becoming cheaper and less intrusive (The International Conference on 3D Digital Imaging and Modeling, 1997–2007).”

“Most of the approaches in the literature are rigid,” says the text in page 2, just before the overview which states: “The main contribution of this paper is a non-rigid 3D face recognition approach. This approach robustly models the expression patterns of the human face and applies the model to morph out facial expressions from a 3D scan of a probe face before matching. Robust expression modeling and subsequent morphing gives our approach a better ability in differentiating between expression deformations and interpersonal disparities. Consequently, more interpersonal disparities are preserved for the matching stage leading to better recognition performance.”

The background section is followed by some classification of existing work, concluding with: “Our approach also falls into this category i.e. non-rigid 3D face recognition.”

1.1 presents a very good summary of related work and 1.2 a clear overview of the method and the ideas behind it, accompanied by a helpful diagram at the bottom of page 3 (Figure 1). The strategy is to use pairs of image of the same individuals, normalising them a bit, and then applying PCA to reduce the dimensionality that characterises expression variation.

Section 2 in page 4 starts by describing pre-processing steps that are essential yet specific to the limitation of the FRGC v.20 dataset. Page 5 starts presenting some visual examples of the approach, with some equations relating to PCA (along with more visual examples) in pages 6 and 7.

Section 3 begins to deal with some other experiments that are not just dealing with models in synthesis mode. The same dataset is being used (with about 5,000 3-D faces), but more data gets added to it. To quote, “The dataset is composed of two partitions: the training partition (943 scans) and the evaluation partition (4007 scans). [..] The FRGC dataset was augmented by 3006 scans that were acquired using a Minolta vivid scanner in our laboratory.”

Parameters and set sizes (those which are included) get tested in very large-scale experiments that yield ROC curves. These curves help show how to set the different parameters and enable one to measure advantages of one algorithm over another. Page 13 has some comparisons to other methods from the literature, with numbers summarised in a chart.

This is truly inspiring work and I shall spend the next few weeks learning from it as well as implementing something similar.

ODAY I moved on to my next research project, which I will mention very briefly in the next post. As part of this move I’ve also made some distro changes. The servers are running Ubuntu, so for the sake of consistency I wanted to explore the latest and greatest of Ubuntu and its flavours/derivatives. I gave a go to 3 and spent some time working from them (not long enough an experience as yet).

Will it be 10 out of 10 for the 10.10 releases? What about Mint 10 (codename “Julia”)? I decided to try today, hopefully seeing how it all compares to Fedora 14, which I have used since it was released around the end of October. The change is motivated by needs of consistency; I also like to try different distributions and see what they are all about over time, for the sake of comparison. I might stop using Fedora for a while. It is still installed on a PC, but I moved all my important work files out of there, which means they’ll go out of sync over time.

A report about use will be posted at a later date (this is not a review but a post with quick impressions). The important points are that today I tried Linux Mint (very latest version, which is GNOME based) and latest Ubuntu+Kubuntu shortly afterwards. Linux Mint has this nice new feature (to me at least) that has it start the installation while the user enters installation options such as user details, location, etc. The menus and graphics (including icons and window decorators) are stunning. Setting up dual-head in Ubuntu without proprietary drivers was easy, in Kubuntu it’s still not as easy (getting twinview). Overall, since both share the very same base (even same packages for the most part, except those which are preinstalled), comparison in this case ought to rely on what’s above the hood, mostly user experience.

ODAY I AM releasing the final code from my current project. The following is the accompanying README file:

Homepage: http://schestowitz.com/Research/Cardiac/Cardiomat
Index of related documents: http://schestowitz.com/Research/Progress/2010-2011.htm

INTRODUCTION

Cardiomat is the name given to an Octave/MATLAB program whose intended use is heart tracking, with various different bits that help probe cardiac structure and perform basic measurements.

FILES

Contained in the package are the following M-files and sub-directories:

./Experiments
./Experiments/fiesta_simulation-old-experiments.m
./create_synthetic_set.m
./GUI
./GUI/cardiomat.png
./GUI/splash.m
./GUI/splash.fig
./GUI/get_parameters_from_user.m
./GUI/getoptionsgui.m
./GUI/cardiomat-title.jpg
./GUI/cardiomat.m
./GUI/cardiomat.jpg
./getdicomimages.m
./loadtaggedsequence.m
./README (this file)
./Imported
./Imported/findit
./Imported/findit/findit.m
./Imported/arrow
./Imported/arrow/arrow.m
./Imported/gradient
./Imported/gradient/gaussgradient.zip
./Imported/gradient/gaussgradient
./Imported/gradient/gaussgradient/gaussgradient.m
./Imported/gradient/gaussgradient/README.txt.txt
./Imported/gradient/gaussgradient/testgaussgradient.m
./Hybrid
./Hybrid/cardsim_frame_shortaxis.m
./Hybrid/shuffle_transform.m
./fiesta_simulation.m

In addition, the program depends on at least one function from biosig4octmat-2.50, whose homepage is located at http://biosig.sourceforge.net. That function helps find the centre and the radius of a best fit circle/sphere for a group of points.

The directory named “Hybrid” contains functions that were written jointly and “Imported” contains functions which were brought from the outside. The core functions in the package are GPLv3-licensed and the rest are most likely to be BSD-licensed (check the sources to verify). Some of the imported functions may work only in MATLAB and not in GNU Octave, which is a free/libre substitute for MATLAB that lacks some graphical features for the most part.

HOW TO RUN THE PROGRAM

The program was written to be compatible with both Octave and MATLAB. There are some features in it that may require a UNIX/Linux system (e.g. animated videos) and some that may require MATLAB. Additionally, in order to reduce dependency on MATLAB’s GUI utilities, the program can be run in GUI and CLI modes separately, with only the use of figures being the exception to this (figures are output rather than means of interaction).

Key functions reside in the main directory of the program (one function, getdicomimages.m, is deprecated) and a directory named “GUI” contains some of the newer parts that add a GUI.

An important interface function is called fiesta_simulation.m. The first line of code in this function is a boolean flag that says whether the program should be run in GUI mode or not. If use_gui is 0, then the program will simply run the list of tasks specified below. It is possible to give several jobs for the program to complete this way, e.g. in order to run many experiments overnight. If use_gui is non-zero (probably 1 but not necessarily), then the function is assumed to have been called from the file GUI/cardiomat.m, which is a wrapper that invokes a splash screen and some dialogues that serve as a graphical interface. In turn, parameters fed to these dialogues will be passed to fiesta_simulation.m. It might be necessary to modify some paths inside the files in case the functions and the files are located in different places than the ones specified in parameters. Such paths are named in fiesta_simulation.m, loadtaggedsequence.m (specifies output directories onto which to save figures), and splash.m (looks for program images). For a new user, the only things require modifications are likely to be directory paths and maybe the use_gui boolean, depending on this user’s preference.

To run the program in GUI mode, MATLAB is needed. There is also a parameter to specify in fiesta_simulation.m in order to inform loadtaggedsequence.m whether MATLAB or Octave gets used. It is the boolean which is also the last argument in the function call, being 0 for MATLAB and 1 for Octave. The GUI front end will always pass 0 under the assumption that it is run under MATLAB.

DATASETS

The specified directories include a path from which to open images, optionally with a wildcard to filter bits contained within a given directory, e.g. selecting a subset of images that are a range of numbers. Due to privacy/data anonymity constraints, no raw data accompanies this program, even though it can definitely be obtained from many places (see for example https://schestowitz.com/Weblog/archives/2010/11/14/segmented-and-tagged-datasets/).

COMPATIBILITY

There are separate levels of compatibility dilemmas. One is the program level and another is the operating system level. Although the program has been tested in both MATLAB and Octave, on both GNU/Linux and Windows, there may be trivial code changes to make in case errors show up. Most will be path related, so it is abundantly clear that emphasis should be put on making all functions reachable from within paths definition (e.g. “addpath(genpath(YOUR_PROGRAM_ROOT_DIR_HERE))”) and booleans should be set to do the right thing (GUI/CLI and MATLAB/Octave). The goal is to ensure the program can be run under as many platforms as possible, with either free/libre or proprietary software.

IN CASE OF DIFFICULTIES

If issues running the program are experienced, please contact the author (address above). Bugs are expected to exist (it is a research purposes program, so polish is not a priority) and input is not validated or sanitised to provide virtual ‘rails’ on which the user operates the program (sane values need to be given). Sample function calls are given in fiesta_simulation.m and /Experiments/fiesta_simulation-old-experiments.m. Some of them may use the old interface and thus lack some input arguments, but most of them show the type of experiments which were run using the program.

Writing assignment by anonymous contributor

he nanorobot is an artificial tiny machine that can move on its own. The nanorobot consists of a central unit which can operate inside the human organisms, and of a group of additional units that back the central unit. Its task is to implement certain medical operations on separate organs and tissues. With a successful technology today, the future nanorobots will be able to perform many actions and solve many problems which modern medicine cannot solve. In addition, the nanorobots can cause various atoms and molecules to connect and create new material which can be use as a substitute for important substances in the human body, and the role of certain organs.

HE previous post about validation with rotation put focus on large regions outside the heart, where there was greater separation between landmark points. In this case, initially I tested a general case where 120 points were put around a circular area in and outside the heart, showing more or less what got detected as the expected translation at each frame. I look at only one point at a time here, on purpose — meaning that for each point only its neighbours in the next frame are being compared and there is room for improvement if the whole spacial patch (e.g. 3×3 window) gets used in the current frame. This simplification was intentional here.

(more…)