Research Group on Visual Computation

Nonlinear Shape Registration without Correspondences

Icon of project Related projects: Recovering Diffeomorfic Shape Deformations without Correspondences

Description

This is the sample implementation and benchmark dataset of the nonlinear registration of 2D shapes described in the following papers:

  1. Csaba Domokos, Jozsef Nemeth, and Zoltan Kato. Nonlinear Shape Registration without Correspondences. IEEE Transactions on Pattern Analysis and Machine Intelligence, in press, 2011.

Note that the current demo program implements only planar homography deformations. Other deformations can be easily implemented based on the demo code. The main code has been written in Java by Zoltan Kornel Torok with some minor contributions from Jozsef Nemeth. The source code of the GUI has been adopted from our Affine Registration of Planar Shapes demo. The code is released under the GNU General Public License.

The program uses the following external sources:

The planar homography benchmark dataset has been created by Csaba Domokos and Jozsef Nemeth. It contains 37 different template images of size approx. 256 X 256 and 1480 observations generated synthetically by random planar homographies. Please cite the above publications whenever you use the dataset.

Installation

The program is platform independent.

To run the application:

  1. Change to the NonlinearShapeRegistrationWithoutCorrespondences\dist directory.
  2. Double-click NonlinearShapeRegistrationWithoutCorrespondences.jar in File Manager.

    Hint: specifying the maximum heap size of the Java Virtual Machine is strongly recommended when working with large images. To specify the heap size, start the application from the command line by typing:

    java -jar -Xmx512m NonlinearShapeRegistrationWithoutCorrespondences.jar

    The heap size is specified after the -Xmx parameter without whitespaces.

To build the application:

  1. Set JAVA_HOME environment variable properly. Under MS Windows do not use quotes when setting JAVA_HOME.
    e.g. SET JAVA_HOME=C:\Program Files\Java\jdk1.6.0_11
    (Apache Ant bin directory must also be added to PATH.)
  2. Change to NonlinearShapeRegistrationWithoutCorrespondences folder, and type in the command line

    ant jar

    Binaries are made into the dist directory.

Usage Notes

Only binary images can be opened. Some test images are provided under the images subdirectory. The program GUI should be intuitive. Main steps:

  1. Open the template image.

    The input image format is PNG, where 0 and 1 is assigned to background and foreground area, respectively.

  2. Open the observation image or create one synthetically from the template.

    You can synthetically create a new observation by pressing Create/update button. By pressing Make grid image one can generate an image to check the effect of the given transformation applying to uniform grid.

  3. Press Registrate.

    In this step the values of Func number and tolerance are the parameters for Levenberg-Marquardt algorithm (please refer the above papers). The set of omega functions could be modified by pressing Setup button

  4. Save registration results.

    After the registartion prosess, one can generate an animation from the iteration steps of the equation solver (i.e. Levenberg-Marquardt algorithm) by pressing the Generate animation button. One can also generate an image to check the effect of the estimated transformation applied to a uniform grid by pressing the Make a grid image button.

The red lines in the image windows show the coordinate axes, the origin is in the upper left corner. The y-axis is vertical, with values increasing from top to bottom. The x-axis is horizontal with values increasing from left to right. When displaying images, 'Location of image' tag tells us where the upper-left corner of the transformed image's bounding box is mapped.

The choosen interpolation method affects the creation of a synthetic observation from the template as well as the creation of the registered image.

Command-line options

If you start the application from the command-line, some options are available:

java -jar NonlinearShapeRegistrationWithoutCorrespondences.jar options

Available options:

  • (no parameter)

    Starts the graphical user interface.

  • templatePath observationPath

    Registration of the template image to the observation image without user interaction. By default, the output files will be:

    info.txt
    matrix.mat
    registered.png
    xor.png
    red-green.png
    grid.png
    animation.png

  • templatePath observationPath infoFilePath matlabFilePath registeredPath xorPath redGreenPath gridImagePath animationPath

    Registration of the template image to the observation image without user interaction. The last seven parameters specify the output filenames. If you do not wish to save one of them use a hyphen (-) instead of that filename. Note that this option is available for batch processing only.

A short description of output files

  • Informations file

    A text file containing all information about the registration containing the following fields:

    Time CPU time (in sec.) of the registration algorithm.
    Delta Error measure for evaluating the registration. It means the absolute difference of the template and registered image (please refer the above papers)
    Epsilon Error measure for evaluating the registration. Note that it can only be evaluated when the applied transformation is known (e.g. in the synthetic case). It measures the distance between the true and the estimated transformation. Intuitively, it shows the average transformation error per pixel (please refer the above papers).
    a11
    a12
    a13
    a21
    a22
    a23
    a31
    a32
    a33
    the estimated transformation matrix (row by row)
  • MATLAB file

    The transformation matrix is saved to a .mat file that can be loaded in MATLAB. The name of the MATLAB variable is R.

  • Registered image

    The registered template is displayed in the common coordinate system. First the common bounding box of the aligned images is computed, then the template image is transormed into this area by the recovered transformation.

  • XOR image

    The result of a bitwise XOR operator on the observation and the registered template.

  • RedGreen image

    The registered template is assigned to red channel while the observation is assigned to the green channel.

  • Grid image

    This image shows the transformation grid.

  • Animated (PNG)

    This is the animation showing the intermediate results of the equation solver.

Benchmark dataset

This is a synthetic binary image dataset for testing registration methods for recovering planar homographies. The images are binary where the 0 and 1 represents the background and foreground, respectively. All the images are in PNG format. There are 37 different template images around size 256 X 256. The 1480 observations were generated synthetically by applying randomly choosen projective transformations composed by

  • 0.5, ..., 1.5 scalings;
  • -pi/4, ..., pi/4 along the three axes;
  • -1, ..., 1 translations along both x and y axis and 0.5, ..., 2.5 along the z axis;
  • 0.5, ..., 1.5 focal length

of the normalized template (i.e. within [-0.5, 0.5]).

Templates directory contains the 37 template images in PNG file format. Naming convention: image????.png (image0001.png, image0002.png, ..., image0037.png)

Observations directory contains all observations for each template. Naming convention: observation????_????.png, where the first four digits are the template number, and last four digits count the number of the observation for the given template. Each observation has an associated data file with the same name and extension .dat.

Fields of a datafile
TemplateFileName filename of the corresponding template image
TemplateWidth width and
TemplateHeight height of the template image
TemplatePixel the number of the foreground pixels on the template
ObservationFileName filename of the observation (Note that: the observation has the same name as the corresponding datafile with extension .png
ObservationWidth width and
ObservationHeight height of the observations image
ObservationPixel the number of the foreground pixels on the observation
a11
a12
a13
a21
a22
a23
a31
a32
a33
the applied transformation matrix (row by row)
minX
minY
the coordinates of the upper-left corner of the observation in the coordinate-system of the template image

Download

License

The code is released under the GNU General Public License.

Publications to cite:
  1. Csaba Domokos, Jozsef Nemeth, Zoltan Kato, Nonlinear Shape Registration without Correspondences, In IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 5, pp. 943-958, 2012. [bibtex] [doi]
  2. Jozsef Nemeth, Csaba Domokos, Zoltan Kato, Recovering Planar Homographies between 2D Shapes, In Proceedings of International Conference on Computer Vision, IEEE, Kyoto, Japan, pp. 2170-2176, 2009. [bibtex]

Hichem Abdellali has been awarded the Doctor of Philosophy (PhD.) degree...

2022-04-30


Hichem Abdellali has been awarded the KÉPAF Kuba Attila prize...

2021-06-24