Figure 1: Image orientation of 208 images of a free-standing building. The grey pyramids show the position and rotation of the camera during the image acquisition. The 3d-points of the building and environment are estimated during the image orientation.
Figure 2: 3d-reconstructed surface model of a building (triangulated point cloud, mesh) consisting of more than 25 millions points and more than 50 millions triangles. The model contains very small details as the railing along the stairs in front of the building entrance.
An object can automatically be reconstructed from images obtaining a triangulated 3d-point cloud, if it is viewed from different view points.
First, image orientations have to be estimated. Therefore, point correspondences are detected in overlapping images. Then, the optimal positions of these points in space and the optimal pose (position and rotation) of the camera for each image are estimated employing bundle adjustment, cf. Figure 1. This very precise approach [4] was developed at the research group for visual computing (Prof. Helmut Mayer, Institute of Applied Computer Science) of the Bundeswehr University Munich. The DLR Institute for Robotics and Mechatronics cooperates with this research group.
Second, depth images are derived using semiglobal matching [2]. For each pixel, we try to determine the distance from the object to the camera. The depth images are merged to 3d-points [3], which are triangulated in order to reconstruct the object's surfaces [1]. It is possible to obtain reconstructed settlements with small details at single buildings, if the images have a high resolution and largely overlap each other, cf. the railing of the stairs in Figure 2.
Third, the 3d-models can be simplified for their integration into simulation environments. This can be done after recognising and vectorising buildings and their parts, such as windows and doors [5,6]. Urban modeling can be used for cultural heritage and for process analysis in cities, e.g., traffic simulations.
Selected Publications
[1] Bodenmüller, T. (2009): Streaming Surface Reconstruction from Real Time 3D Measurements. PhD Thesis, Technische Universität München.
[2] Hirschmüller, H. (2008): Stereo Processing by Semi-Global Matching and Mutu- al Information. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2): 328–341.
[3] Kuhn, A., Mayer, H., Hirschmüller, H. & Scharstein, D. (2014): A TV Prior for High-Quality Local Multi-View Stereo Reconstruction. International Conference on 3D Vision, pp. 65–72.
[4] Mayer, H., Bartelsen, J., Hirschmüller, H. & Kuhn, A. (2012): Dense 3D Reconstruction from Wide Baseline Image Sets. In: Dellaert et al. (Eds.): Outdoor and Large-Scale Real-World Scene Analysis, 15th International Workshop on Theoretical Foundations of Computer Vision, Lectures Notes in Computer Science (LNCS), Vol. 7474, pp. 285–304.
[5] Nguatem, W., Drauschke, M. & Mayer, H. (2014): Localization of Windows and Doors in 3D Point Clouds of Facades. International Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume II-3, pp. 87–94.
[6] Nguatem, W., Drauschke, M. & Mayer, H. (2016): Automatic Generation of Building Models with Levels of Detail 1–3. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLI-B3, pp. 649–654.
