Abstract
The problem of estimating multiple parametric models that fit data corrupted by noise and outliers is an ubiquitous task in Pattern Recognition and is at the core of many Computer Vision applications. Typical examples can be found in 3D reconstruction, where multi-model fitting is employed either to estimate multiple rigid moving objects, or to produce intermediate geometric interpretations of reconstructed 3D point clouds. Other scenarios in which the estimation of multiple geometric models plays a primary role include face clustering, body-pose estimation and motion segmentation, just to name a few. In the last years, the need for compact and abstract representations of low level data has inspired research efforts towards the design of automatic methods that can aggregate visual content in higher level structures and multi-model fitting has attracted increasing interest. In this tutorial, we present an overview on the problem of multi-model fitting from complementary perspectives. Specifically, a general formulation of the multi-model fitting problem that fits with many Computer Vision applications is introduced. This provides a common context within which three main state-of-the-art approaches are presented. In particular, the tutorial covers the problem of multi model fitting from:
- a perspective based on the optimization of an energy: the multi-model fitting problem is interpreted as a labeling energy minimization procedure assigning data points to model instances. The minimized energy usually consists of terms penalizing the point-to-model assignment, the complexity and, often, some additional geometric considerations, e.g., spatial coherence;
- a perspective based on preference analysis: the estimation of multiple structure is addressed in a procedural way leveraging on simple to implement clustering techniques;
- a perspective based on hypergraph: the relationships between data points and model hypotheses are represented by a hypergraph model and the estimation of multiple structures is formulated as a hypergraph partitioning problem.
Short Bio
Luca Magri graduated in Mathematics at the University of Milan (IT) in 2012. In 2015, he received the PhD from the University of Milan with a thesis on robust multiple model fitting for Computer Vision applications. From 2015 to 2018, he has been a post-doc researcher first at the University of Verona (Dept. of Computer Science) and then at the University of Udine (DPIA), in these periods he collaborated with 3DFlow srl on 3D reconstruction themes. In 2018-2019 he joined the R&D group of FARO Technologies (Rezzato, IT) where he worked on acquisition and registration techniques for structured light scanners. Currently he is at Dipartimento di Elettronica, Informazione e Bioingegneria of the Politecnico di Milano (DEIB) as a postdoctoral researcher.
Daniel Barath was born in 1989 in Budapest. He graduated at Eötvös Loránd University in 2014. In 2020, he defended his thesis and received his Ph.D. degree. He is a researcher of the Visual Recognition Group, FEE, Czech Technical University, Prague, Czech Republic and the Machine Perception Research Laboratory at the Institute for Computer Science and Control (MTA SZTAKI), Budapest, Hungary. His research interests are robust model and multi-model estimation and minimal methods in computer vision. He was an organizer of the “RANSAC in 2020” full-day tutorial at CVPR 2020.
Guobao Xiao received the B.S. degree in information and computing science from Fujian Normal University, China, in 2013 and the Ph.D. degree in Computer Science and Technology from Xiamen University, China, in 2016. From 2016-2018, he was a Postdoctoral Fellow in the School of Aerospace Engineering at Xiamen University, China. He is currently a Full Professor at Minjiang University, China. He has published over 30 papers in the international journals and conferences including IEEE Transactions on Pattern Analysis and Machine Intelligence, International Journal of Computer Vision, IEEE Transactions on Image Processing, Pattern Recognition, IEEE Transactions on Intelligent Transportation Systems, ICCV, ECCV, ACCV, AAAI, ICIP, ICARCV, etc. His research interests include machine learning, computer vision, pattern recognition and bioinformatics. He has been awarded the best PhD thesis in Fujian Province and the best PhD thesis award in China Society of Image and Graphics (a total of ten winners in China). He also served on the program committee (PC) of CVPR, ICCV, ECCV, AAAI, IJCAI, etc. He was the General Chair for IEEE BDCLOUD 2019.
Eleonora Maset received the M.Sc. degree in environmental engineering and the Ph.D. degree in industrial and information engineering from the University of Udine, Italy, in 2015 and 2019, respectively. In 2019 she was a post-doc researcher at the University of Udine; in this period she collaborated with Helica srl to the development of LiDAR point cloud classification methods. She currently collaborates with the Polytechnic Department of Engineering and Architecture (DPIA), University of Udine. She received the Young Author’s Award at the XXIV ISPRS Congress for the work “Bundle Block Adjustment with Constrained Relative Orientations”. She served as reviewer for CVPR, ECCV, ICCV and 3DV. Her research interests include image orientation, point cloud processing and indoor modelling.
Andrea Fusiello received his Laurea (Master) degree in Computer Science from the University of Udine in 1994 and the Dottorato di Ricerca (PhD) in Computer Engineering from the University of Trieste in 1999. After graduating, he has been EPSRC Visiting Research Fellow at Heriot-Watt University, Edinburgh in 1999 (for 6 months). From 2000 to 2011 he was with the Department of Computer Science, University of Verona, where he taught Computer Graphics and Computer Vision, as Ricercatore (Assistant Professor) first and Associate Professor since 2005. In 2012 he moved to the University of Udine, at the Department of Management, Electrical, and Mechanical Engineering, where he teaches Fundamentals of Computer Science (undergraduate) and Computer Vision (graduate). In 2013 he was awarded the national scientific qualification as Full Professor. Andrea Fusiello was Associate Editor of “Pattern Recognition” and “IET Computer Vision” for three years each, and he regularly serves as reviewer for CVPR, ICCV, and ECCV. He was the workshop chair at ECCV12 and ECCV20 and general chair of 3DV18. In 2016 he was appointed as co-chair of WG II/1 of the ISPRS. His research focuses on various topics in Computer Vision, Photogrammetry and Image Analysis: stereo matching, epipolar rectification, tracking, motion segmentation, mosaics, view synthesis, auto-calibration, 3-D modelling, model acquisition. Andrea Fusiello has published more than 160 papers, of which 39 in international journals.