Research Group on Visual Computation

Description

Android demo implementation is available.

Reconstruction of the surrounding 3D world is of particular interest either for mapping, civil applications or for entertainment. We proposed a pipeline that utilizes an ad-hoc network of camera equipped smartphones in a collaborative manner. Data, computations and results can be shared among the participant devices without centralization. The reconstruction is based on segmented planar regions which is especially suitable for scenes having large (near) planar parts. Once the calibration parameters are known, the perspective distortion of a planar region in two views makes it possible to compute the normal and distance of the region w.r.t the world coordinate system.

The pipeline is based on the algorithms developed previously.

• Initialization
  • Image acquisition

    Photos of building facades were taken by using different camera equipped smartphones.

  • Camera calibration (intrinsic parameters)

    The intrinsic camera parameters were determined by classical image processing methods. We used the Matlab Camera Calibration Toolbox by Jean-Yves Bouguet. Since the smartphone cameras have fix focus lens, this process is to be performed only once.

  • Calibration of an ad-hoc camera network

    The algorithm computes which pairs of photos have a common view using the vision graphs approach.

• Piecewise reconstruction
  • Correspondence between planar patches

    Polygonal borders of planar image regions (patches) are segmented interactively by the user. The images are rectified and the 3 DOF constrained affine geometric transformation is determined. The result is then unrectified to form the homographies between the corresponding regions.

  • Planar patch reconstruction

    By knowing the intrinsic camera calibration parameters and the essential matrix of the camera pair, the perspective distortion of a planar region in two views makes it possible to compute the normal and distance of the region w.r.t the world coordinate system.

• Uncertainty verification (optional)

  • Reject patches of possible misregistration.

• Visualization
  • 3D display and interaction

    Model textured polygons in a 3D scene. Let the user interactively change display parameters (e.g., camera position and orientation).

  • Optimal texture selection

    The texture information is selected from the image which provides the more direct visibility based on the angle between the normal vector of the plane and the projection direction of the camera.

Fig.1. Top row: The two overlapping input images. The border of the polygonal patches selections are shown in red. Bottom: Animation of the 3D reconstructed result.

Fig.2. Top row: The two overlapping input images. The border of the polygonal patches selections are shown in red. Bottom: Animation of the 3D reconstructed result.