3D Computer vision efficient methods and applications

【实例简介】
计算机图形学英文原版书籍。 This work provides an introduction to the foundations of three-dimensional computer vision and describes recent contributions to the field, which are of methodical and application-specific nature. Each chapter of this work provides an extensive overview of the corresponding state of th
Christian wohler 3D Computer Vision Efficient Methods and applications ringer Dr Christian wohler Daimler AG, Group Research and advanced engineering P.O. Box 2360 D-89013Ulm christianwoehler@daimler.com ISSN1612-1449 ISBN978-3-64201731-5 ISBN978-3-642-01732-2 DOI10.1007/978-3-642-017322 Springer Dordrecht Heidelberg London New York Library of Congress Control Number: 2009929715 O Springer-Verlag Berlin Heidelberg 2009 This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, eproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965 in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law The use of general descriptive names, registered names, trademarks, etc. in this publication does not mply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use Cover design: KunkelLopka GmbH Printed on acid-free paper SpringerispartofSpringerScience+businessMedia(www.springer.com) To Katja, Alexander, and sebastian Preface This work provides an introduction to the foundations of three-dimensional com- puter vision and describes recent contributions to the field, which are of methodical and application-Specific nature. Each chapter of this work provides an extensive overview of the corresponding state of the art, into which a detailed description of new methods or evaluation results in application-specific systems is embedded. Geometric approaches to three-dimensional scene reconstruction(cf. Chapter 1) are primarily based on the concept of bundle adjustment, which has been developed more than 100 years ago in the domain of photogrammetry The three-dimensional scene structure and the intrinsic and extrinsic camera parameters are determined such that the Euclidean backprojection error in the image plane is minimised, usu ally relying on a nonlinear optimisation procedure. In the field of computer vision an alternative framework based on projective geometry has emerged during the last two decades, which allows to use linear algebra techniques for three-dimensional scene reconstruction and camera calibration purposes. With special emphasis on the problems of stereo image analysis and camera calibration, these fairly different ap proaches are related to each other in the presented work, and their advantages and drawbacks are stated. In this context, various state-of-the-art camera calibration and self-calibration methods as well as recent contributions towards automated camera calibration systems are described. An overview of classical and new feature-based correlation-based, dense, and spatio-temporal methods for establishing point cor- respondences between pairs of stereo images is given. Furthermore, an analysis of traditional and newly introduced methods for the segmentation of point clouds and for the three-dimensional detection and pose estimation of rigid articulated and fexible objects in the scene is provided a different class of three-dimensional scene reconstruction methods is made up by photometric approaches(cf. Chapter 2), which evaluate the intensity distribution in the image to infer the three-dimensional scene structure Basically, these methods can be divided into shape from shadow, photoclinometry and shape from shading, photometric stereo, and shape from polarisation. As long as sufficient information about the illumination conditions and the surface reflectance properties is available these methods may provide dense depth maps of object surfaces Preface In a third, fundamentally different class of approaches the behaviour of the point spread function of the optical system used for image acquisition is exploited in or der to derive depth information about the scene (cf. Chapter 3). Depth from focus methods use as a reference the distance between the camera and the scene at which a minimum width of the point spread function is observed, relying on an appro- priate calibration procedure. Depth from defocus methods determine the position- dependent point spread function, which in turn yields absolute depth values for the scene points. A semi-empirical framework for establishing a relation between the depth of a scene point and the observed width of the point spread function is intro uce These three classes of approaches to three-dimensional scene reconstruction are characterised by complementary properties, such that it is favourable to integrate them into unified frameworks that yield more accurate and robust results than each of the approaches alone(cf. Chapter 4) bundle adjustment and depth from defocus are combined to determine the absolute scale factor of the scene reconstruction re sult, which cannot be obtained by bundle adjustment alone if no a-priori information is available. Shading and shadow features are integrated into a self-consistent frame- work to reduce the inherent ambiguity and large-scale inaccuracy of the shape from shading technique by introducing regularisation terms that rely on depth differences inferred from shadow analysis. Another integrated approach combines photometric polarimetric, and sparse depth information, yielding a three-dimensional reconstruc tion result which is equally accurate on large and on small scales. An extension of this method provides a framework for stereo image analysis of non-Lambertian sur faces where traditional stereo methods tend to fail. In the context of monocular three-dimensional pose estimation, the integration of geometric, photopolarimetric, and defocus cues is demonstrated to behave more robustly and is shown to provide significantly more accurate results than techniques exclusively relying on geometric nformation The developed three-dimensional scene reconstruction methods are examined in different application scenarios a comparison to state-of-the-art systems is provided where possible. In the context of industrial quality inspection(cf. Chapter 5), the performance of pose estimation is evaluated for rigid objects(plastic caps, electric lugs) as well as flexible objects(tubes, cables). The integrated surface reconstruc tion methods are applied to the inspection of different kinds of metallic surfaces where the achieved accuracies are found to be comparable to those of general purpose active scanning devices which, however, require a much higher instrumental effort The developed techniques for object detection and tracking in three-dimensional point clouds and for pose estimation of articulated objects are evaluated in the con- text of partially automated industrial production scenarios requiring a safe interac tion between humans and industrial robots(cf. Chapter 6). An overview of existing vision-based robotic safety systems is given, and it is worked out how the developed three-dimensional detection and pose estimation techniques are related to state-of- the-art gesture recognition methods in human-robot interaction scenarios Preface The third addressed application scenario is completely different and regards re mote sensing of the lunar surface by preparing elevation maps(cf. Chapter 7) While the spatial scales taken into account differ by many orders of magnitude from those encountered in the industrial quality inspection domain, the underly ing physical processes are fairly similar. An introductory outline of state-of-the-art geometric, photometric, and combined approaches to topographic mapping of solar system bodies is given. Especially the estimation of impact crater depths and shapes is an issue of high geological relevance. generally, such measurements are based on the determination of shadow lengths and do not yield detailed elevation maps. It is demonstrated for lunar craters that three-dimensional surface reconstruction based on shadow, reflectance, and geometric information yields topographic maps of higl resolution, which are useful for a reliable crater classification. another geologically relevant field is the three-dimensional reconstruction of lunar volcanic edifices es pecially lunar domes. These structures are so low that most of them do not appear in the existing lunar topographic maps based on the described photometric three dimensional reconstruction methods, the first catalogue to date containing heights and edifice volumes for a statistically significant number of lunar domes has been prepared. It is outlined briefly why the determined three-dimensional morphometric data are essential for deriving basic geophysical parameters of lunar domes such as lava viscosity and effusion rate, and how they may help to reveal their origin and mode of formation Finally (cf. Chapter 8), the main results of the presented work and the most im- portant conclusions are summarised, and possible directions of future research are outlined Heroldstatt, May 2009 Christian wohler Acknowledgements First of all, i wish to express my gratitude to my wife Khadija Katja and my sons Adnan Alexander Emile and sebastian Marc amin for their patience and continuous encouragement I am grateful to Prof. Dr. Gerhard Sagerer(Technical Faculty, Bielefeld Uni- versity ), Prof Dr Reinhard Klette( Computer Science Department, University of Auckland), and Prof. Dr. Rainer Ott (Faculty of Computer Science, Electrical En gineering, and Information Technology, Stuttgart University) for providing the re- views for this work Moreover, I wish to thank Prof. Dr. Gerhard Sagerer Prof. Dr. franz Kummer, Joachim Schmidt, and Niklas beuter from Bielefeld University for the fruitful col- laboration. I gratefully acknowledge to be given the opportunity to become a vis iting lecturer at the Technical Faculty and thus to stay in touch with the university environment i also wish to thank prof, dr horst -Michael grob from the technical University of ILmenau for the long-lasting cooperation Special thanks go to my colleagues in the Environment Perception department at Daimler Group research and Advanced Engineering in Ulm for providing a lively and inspiring scientific environment, especially to Dr. Lars Kruger(to whom I am extraordinarily indebted for his critical reading of the manuscript), Prof. Dr. Rainer Ott, Dr. Ulrich KreBel, Frank Lindner, and Kia hafez, to our(former and current) PhD students Dr. Pablo d'Angelo, Dr. Marc Ellenrieder, Bjorn Barrois, Markus Hahn, and Christoph Hermes, and Diplom students Annika Kuhl, Tobias Govert and Melanie KrauB. i also wish to thank Claus lorcher and his team colleagues Werner Progscha, Dr Rolf Finkele, and Mike Bopple for their continuous support Furthermore, I am grateful to the members of the Geologic Lunar Research Group, especially Dr. Raffaello Lena, Dr. Charles A. Wood, Paolo Lazzarotti, Dr Jim phillips, michael Wirths, K. C. Pau, Maria Teresa bregante and richard Evans, for sharing their experience in many projects concerning lunar observation and geology My thanks are extended to the Springer editorial staff, especially Hermann En- gesser, Dorothea Glaunsinger, and Gabi Fischer, for their advice and cooperation Contents Part I Methods of 3D computer vision 1 Geometric Approaches to Three-dimensional Scene reconstruction 1.1 The Pinhole camera model 1.2 Bundle Adjustment methods 3379 1.3 Geometric Aspects of Stereo Image Analysis 1.3. 1 Euclidean Formulation of Stereo Image Analysis 1.3.2 Stereo Image analysis in Terms of Projective geometry .. 12 1. 4 Geometric Calibration of Single and Multiple Cameras...... 17 1. 4.1 Methods for Intrinsic Camera Calibration 1.4.2 The Direct Linear Transform(DLt) Method 1.4.3 The Camera Calibration Method by tsai (1987) 21 1.4.4 The Camera Calibration Method by Zhang g(19 25 1. 4.5 The Camera Calibration Method by Bouguet(2007) 27 1.4.6 Self-calibration of Camera Systems from Multiple views of a static scene 1. 4.7 Semi-automatic Calibration of Multiocular Camera Systems 41 1.4.8 Accurate Localisation of Chequerboard Corners 1.5 Stereo Image Analysis in Standard Geometry 62 1.5.1 Image Rectification According to Standard geometry 6 1.5.2 The Determination of Corresponding Points 6 1.6 Three-dimensional pose estimation and segmentation Methods .. 87 1.6. 1 Pose Estimation of rigid Objects 88 1.6.2 Pose Estimation of Non-rigid and Articulated Objects 95 1.6.3 Point Cloud Segmentation Approaches 113 2 Photometric Approaches to Three-dimensional Scene Reconstruction 127 2. 1 Shape from Shadow 127 2.1.1 Extraction of Shadows from Image pairs 128 2.1.2 Shadow-based Surface reconstruction from Dense sets of Images 130 【实例截图】
【核心代码】