Reconstruction from
                                                                               Multiple Views
    Chapter 6                                                                  Prof. Daniel Cremers
    Reconstruction from Multiple Views
                                                                               FromTwoViewsto
    Multiple View Geometry                                                     Multiple Views
                                                                               Preimage & Coimage
    Summer2022                                                                 from Multiple Views
                                                                               FromPreimagesto
                                                                               RankConstraints
                                                                               Geometric
                                                                               Interpretation
                                                                               TheMultiple-view
                                                                               Matrix
                                                                               Relation to Epipolar
                                                                               Constraints
                                                                               Multiple-View
                                                                               Reconstruction
                                                                               Algorithms
                                                                               Multiple-View
                                                                               Reconstruction of
                                                   Prof. Daniel Cremers        Lines
                  Chair of Computer Vision and Artificial Intelligence
                           Departments of Informatics & Mathematics
                                         Technical University of Munich
                                                                               updatedApril23,2022 1/43
   Overview                                                                    Reconstruction from
                                                                               Multiple Views
                                                                               Prof. Daniel Cremers
   1 FromTwoViewstoMultipleViews
   2 Preimage&CoimagefromMultipleViews                                         FromTwoViewsto
                                                                               Multiple Views
                                                                               Preimage & Coimage
   3 FromPreimagestoRankConstraints                                            from Multiple Views
                                                                               FromPreimagesto
                                                                               RankConstraints
   4 GeometricInterpretation                                                   Geometric
                                                                               Interpretation
                                                                               TheMultiple-view
   5 TheMultiple-view Matrix                                                   Matrix
                                                                               Relation to Epipolar
                                                                               Constraints
   6 Relation to Epipolar Constraints                                          Multiple-View
                                                                               Reconstruction
                                                                               Algorithms
   7 Multiple-View Reconstruction Algorithms                                   Multiple-View
                                                                               Reconstruction of
                                                                               Lines
   8 Multiple-View Reconstruction of Lines
                                                                               updatedApril23,2022 2/43
   Multiple-View Geometry                                                      Reconstruction from
                                                                               Multiple Views
    In this section, we deal with the problem of 3D reconstruction             Prof. Daniel Cremers
    given multiple views of a static scene, either obtained
    simultaneously, or sequentially from a moving camera.
    Thekeyideaisthat the three-view scenario allows to obtain                  FromTwoViewsto
    moremeasurementstoinfer the same number of 3D                              Multiple Views
    coordinates. For example, given two views of a single 3D point,            Preimage & Coimage
                                                                               from Multiple Views
    wehavefourmeasurements(x-andy-coordinate in each                           FromPreimagesto
    view), while the three-view case provides 6 measurements per               RankConstraints
                                                                               Geometric
    point correspondence. As a consequence, the estimation of                  Interpretation
    motion and structure will generally be more constrained when               TheMultiple-view
    reverting to additional views.                                             Matrix
                                                                               Relation to Epipolar
    Thethree-view case has traditionally been addressed by the                 Constraints
                                                                               Multiple-View
    so-called trifocal tensor [Hartley ’95, Vieville ’93] which                Reconstruction
                                                                               Algorithms
    generalizes the fundamental matrix. This tensor – as the                   Multiple-View
    fundamental matrix – does not depend on the scene structure                Reconstruction of
                                                                               Lines
    but rather on the inter-frame camera motion. It captures a
    trilinear relationship between three views of the same 3D point
    or line [Liu, Huang ’86, Spetsakis, Aloimonos ’87].
                                                                               updatedApril23,2022 3/43
   Trifocal Tensor versus Multiview Matrices                                   Reconstruction from
                                                                               Multiple Views
                                                                               Prof. Daniel Cremers
    Traditionally the trilinear relations were captured by
    generalizing the concept of the Fundamental Matrix to that of a
    Trifocal Tensor. It was developed among others by [Liu and
    Huang’86], [Spetsakis, Aloimonos ’87]. The use of tensors
    waspromotedby[Vieville ’93] and [Hartley ’95]. Bilinear,                   FromTwoViewsto
                                                                               Multiple Views
    trilinear and quadrilinear constraints were formulated in [Triggs          Preimage & Coimage
    ’95]. This line of work is summarized in the books:                        from Multiple Views
                                                                               FromPreimagesto
    Faugeras and Luong, “The Geometry of Multiple Views”, 2001                 RankConstraints
                                                                               Geometric
    and                                                                        Interpretation
                                                                               TheMultiple-view
    Hartley and Zisserman, “Multiple View Geometry”, 2001, 2003.               Matrix
                                                                               Relation to Epipolar
    In the following, however, we stick with a matrix notation for the         Constraints
                                                                               Multiple-View
    multiview scenario. This approach makes use of matrices and                Reconstruction
    rank constraints on these matrices to impose the constraints               Algorithms
                                                                               Multiple-View
    from multiple views. Such rank constraints were used by many               Reconstruction of
    authors, among others in [Triggs ’95] and in [Heyden, Åström               Lines
    ’97]. This line of work is summarized in the book
    Ma, Soatto, Kosecka, Sastry, “An Invitation to 3D Vision”, 2004.
                                                                               updatedApril23,2022 4/43