Nonlinear Shape Registration without Correspondences

This is the sample implementation and benchmark dataset of the nonlinear registration of 2D shapes described in the following papers:

1. Csaba Domokos, Jozsef Nemeth, and Zoltan Kato. Nonlinear Shape Registration without Correspondences. IEEE Transactions on Pattern Analysis and Machine Intelligence, in press, 2011.

Note that the current demo program implements only planar homography deformations. Other deformations can be easily implemented based on the demo code. The main code has been written in Java by Zoltan Kornel Torok with some minor contributions from Jozsef Nemeth. The source code of the GUI has been adopted from our Affine Registration of Planar Shapes demo. The code is released under the GNU General Public License.

The program uses the following external sources:

• JMatIO is used when saving the results into Matlab .mat file.
• Japng (Java APNG Library) is used when generating animated PNG containing the iteration steps of equation solver. (Note: this image format is supported by e.g. GIMP or ImageJ).
• JEP (Java Math Expression Parser) is used to parse expressions for the user defined omega functions.
• JMF (Java Media Framework) is used when displaying the animation of iteration steps.

The planar homography benchmark dataset has been created by Csaba Domokos and Jozsef Nemeth. It contains 37 different template images of size approx. 256 X 256 and 1480 observations generated synthetically by random planar homographies. Please cite the above publications whenever you use the dataset.

Installation

The program is platform independent.

• You need Java SE Runtime Environment (JRE 6 Update 11 or later) to run the application.
• You need Java SE Development Kit (JDK 6 Update 11 or later) and Apache Ant (1.7.1 or later) to build the application.
  (Note that Java SE Runtime Environment is included in Java SE Development Kit.)

To run the application:

1. Change to the NonlinearShapeRegistrationWithoutCorrespondences\dist directory.
2. Double-click NonlinearShapeRegistrationWithoutCorrespondences.jar in File Manager.

   Hint: specifying the maximum heap size of the Java Virtual Machine is strongly recommended when working with large images. To specify the heap size, start the application from the command line by typing:

   java -jar -Xmx512m NonlinearShapeRegistrationWithoutCorrespondences.jar

   The heap size is specified after the -Xmx parameter without whitespaces.

To build the application:

1. Set JAVA_HOME environment variable properly. Under MS Windows do not use quotes when setting JAVA_HOME.
   e.g. SET JAVA_HOME=C:\Program Files\Java\jdk1.6.0_11
   (Apache Ant bin directory must also be added to PATH.)
2. Change to NonlinearShapeRegistrationWithoutCorrespondences folder, and type in the command line
   

   ant jar

   Binaries are made into the dist directory.

Usage Notes

Only binary images can be opened. Some test images are provided under the images subdirectory. The program GUI should be intuitive. Main steps:

1. Open the template image.

   The input image format is PNG, where 0 and 1 is assigned to background and foreground area, respectively.

2. Open the observation image or create one synthetically from the template.

   You can synthetically create a new observation by pressing Create/update button. By pressing Make grid image one can generate an image to check the effect of the given transformation applying to uniform grid.

3. Press Registrate.

   In this step the values of Func number and tolerance are the parameters for Levenberg-Marquardt algorithm (please refer the above papers). The set of omega functions could be modified by pressing Setup button

4. Save registration results.

   After the registartion prosess, one can generate an animation from the iteration steps of the equation solver (i.e. Levenberg-Marquardt algorithm) by pressing the Generate animation button. One can also generate an image to check the effect of the estimated transformation applied to a uniform grid by pressing the Make a grid image button.

The red lines in the image windows show the coordinate axes, the origin is in the upper left corner. The y-axis is vertical, with values increasing from top to bottom. The x-axis is horizontal with values increasing from left to right. When displaying images, 'Location of image' tag tells us where the upper-left corner of the transformed image's bounding box is mapped.

The choosen interpolation method affects the creation of a synthetic observation from the template as well as the creation of the registered image.

Command-line options

If you start the application from the command-line, some options are available:

java -jar NonlinearShapeRegistrationWithoutCorrespondences.jar options

Available options:

• (no parameter)

  Starts the graphical user interface.

• templatePath observationPath

  Registration of the template image to the observation image without user interaction. By default, the output files will be:

  info.txt
matrix.mat
registered.png
xor.png
red-green.png
grid.png
animation.png

• templatePath observationPath infoFilePath matlabFilePath registeredPath xorPath redGreenPath gridImagePath animationPath

  Registration of the template image to the observation image without user interaction. The last seven parameters specify the output filenames. If you do not wish to save one of them use a hyphen (-) instead of that filename. Note that this option is available for batch processing only.

A short description of output files

• Informations file

  A text file containing all information about the registration containing the following fields:

  Time	CPU time (in sec.) of the registration algorithm.
  Delta	Error measure for evaluating the registration. It means the absolute difference of the template and registered image (please refer the above papers)
  Epsilon	Error measure for evaluating the registration. Note that it can only be evaluated when the applied transformation is known (e.g. in the synthetic case). It measures the distance between the true and the estimated transformation. Intuitively, it shows the average transformation error per pixel (please refer the above papers).
  a11 a12 a13 a21 a22 a23 a31 a32 a33	the estimated transformation matrix (row by row)

• MATLAB file

  The transformation matrix is saved to a .mat file that can be loaded in MATLAB. The name of the MATLAB variable is R.

• Registered image

  The registered template is displayed in the common coordinate system. First the common bounding box of the aligned images is computed, then the template image is transormed into this area by the recovered transformation.

• XOR image
  

  The result of a bitwise XOR operator on the observation and the registered template.

• RedGreen image

  The registered template is assigned to red channel while the observation is assigned to the green channel.

• Grid image

  This image shows the transformation grid.

• Animated (PNG)

  This is the animation showing the intermediate results of the equation solver.

Benchmark dataset

This is a synthetic binary image dataset for testing registration methods for recovering planar homographies. The images are binary where the 0 and 1 represents the background and foreground, respectively. All the images are in PNG format. There are 37 different template images around size 256 X 256. The 1480 observations were generated synthetically by applying randomly choosen projective transformations composed by

• 0.5, ..., 1.5 scalings;
• -pi/4, ..., pi/4 along the three axes;
• -1, ..., 1 translations along both x and y axis and 0.5, ..., 2.5 along the z axis;
• 0.5, ..., 1.5 focal length

of the normalized template (i.e. within [-0.5, 0.5]).

Templates directory contains the 37 template images in PNG file format. Naming convention: image????.png (image0001.png, image0002.png, ..., image0037.png)

Observations directory contains all observations for each template. Naming convention: observation????_????.png, where the first four digits are the template number, and last four digits count the number of the observation for the given template. Each observation has an associated data file with the same name and extension .dat.

Download

• Full source and precompiled binaries are available here
• Benchmark dataset