Terrain and city modeling from aerial images

Aerial images can be acquired relative quickly and with ground resolutions of down to a few cm per pixel. Usually, cameras are pre-calibrated and the position and orientation at each imaging position is measured during the flight by differential GPS and IMU sensors.

For several applications, it is useful or required to reconstruct the scene and integrate all individual images into one consistent, large image.

Method

All images are typically bundle adjusted such that the geometry of all cameras is known with high precision (i.e. less than one pixel error). Aerial cameras include full frame cameras as well as pushbroom cameras. While the former can be modeled using a pinhole model, the latter need a more complex model (Hirschmueller et al., 2005). The most critical step requires matching of all pixels with corresponding pixels of neighboring images using the Semi-Global Matching stereo algorithm.

The geometrical and correspondence information is used for reconstructing all points and mapping them into an equidistant grid. For aerial images, it is typically sufficient to store one height (i.e. the highest) in each grid cell, which results in a 2.5D model. Thus, the individual perspective projections are transformed and fused into an orthographic projection using a 2.5D model (Hirschmueller, 2008).

A true orthographic (single) image is created from all individual images by projecting the height of each grid cell back into all images and searching the image that best "sees" this point. Some geometrical and radiometrical based heuristics are used for finding unoccluded projections and for combining their values. Similarly, tilted (i.e. side) textures for houses, etc. can be generated by using tilted ortho projections (Hirschmueller, 2008).

For visualization, it is often necessary to transform the textured 2.5D model, which is represented as an equidistant grid of height and color values, into a textured mesh, that can be rendered fast on modern graphics hardware. The automatic process performs six steps:  tilling with overlap, mesh generation, mesh simplifying, mesh cutting, mesh merging and texture mapping. The result is saved as a group of simplified meshes with corresponding textures in VRML 2.0 format (Liu et al., 2007).

Results

Publications

Heiko Hirschmüller (2008), Stereo Processing by Semi-Global Matching and Mutual Information, in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 30(2), February 2008, pp. 328-341.

R. Liu, D. Burschka, G. Hirzinger and B. Strackenbrock (2007), Real Time Fully Automatic 3D-Modeling of HRSC Landscape Data, in Proc. of Urban Remote Sensing Joint Event (6th URS/4th Urban), Paris, 2007.

Heiko Hirschmüller, Helmut Mayer, G. Neukum and the HRSC-CoI Team (2006), Stereo Processing of HRSC Mars Express Images by Semi-Global Matching, International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVI, Part 4, 27-30 September 2006, Goa, India.