Map Synchronization: from Object Correspondences to Neural Networks
Tutorial at CVPR 2019
Sunday, June 16th, 9:00 am  12:30 pm
201B, Long Beach Convention Center, Long Beach, CA
TUTORIAL SUMMARY 
Establishing maps (e.g. pointwise correspondences) across object collections is a fundamental problem spanning many scientific domains. Highquality maps facilitating information propagation and transformation are key to applications ranging from multiview structure from motion, 3D reconstruction with partial scans, datadriven geometry completion and reconstruction, texture transfer, to comparative biology, joint dataanalysis, and data exploration and organization. In the deep learning era, the concept of maps naturally extends to neural networks between different domains.
High quality object maps are generally difficult to compute. Prior work on map computation focused on optimizing maps between pairs of objects. Despite the significant progress, stateoftheart techniques tends to hit a barrier on the quality of maps that are computed in a pairwise manner. Building upon the availability of bigdata, a recent line of research considered computing many pairwise maps jointly among a collection of objects (or map synchronization). The promise of these approaches hinges upon the observation that one way to obtain a high quality pairwise map between dissimilar objects is to choose a path connecting these objects but consisting of consecutive similar shapes: maps between similar objects are typically of higher quality, and so is the resulted composed map. From a regularization perspective, joint map computation leverages the generic consistency of a network of maps among multiple objects, in which composition of maps along cycles are expected to be close to the identity map.
In this tutorial, we cover different map synchronization techniques, including the ones that are based on graph theory, the ones that are based on combinatorial optimization, and the ones that are based on modern optimization techniques such as spectral decomposition, convex optimization, nonconvex optimization and MAP inference. We also cover recent techniques that jointly optimize neural networks across multiple domains. Besides optimization techniques, we will also discuss the applications of map synchronization in multiview based geometry reconstruction (RGB images or RGBD images), jointly analysis of image collections, and 3D reconstruction and understanding across multiple domains.

TARGET AUDIENCE 
The intended audience are academicians, graduate students and industrial researchers who are interested in the stateoftheart techniques for multiview structurefrommotion, geometry reconstruction from depthscans, dense image flows, unsupervised object discovery and joint learning of neural networks. The tutorial is selfcontained.

SPEAKER BIOS 
Qixing Huang
is an assistant professor at the University of Texas at Austin. He obtained his PhD in Computer Science from Stanford University and his MS and BS in Computer Science from Tsinghua University. He was a research assistant professor at Toyota Technological Institute at Chicago before joining UT Austin. He has also worked at Adobe Research and Google Research. Dr. Huang's research spans the fields of computer vision, computer graphics, and machine learning. In particular, he is interested in designing new algorithms that process and analyze big geometric data (e.g., 3D shapes/scenes). He is also interested in statistical data analysis, compressive sensing, lowrank matrix recovery, and largescale optimization, which provides theoretical foundation for his research. He also received the best paper award at the Symposium on Geometry Processing 2013. He is an area chair of CVPR 2019 and ICCV 2019.
Xiaowei Zhou is a Research Professor in Computer Science at Zhejiang University, China. He was a postdoctoral researcher in the Computer and Information Science Department at University of Pennsylvania. His research interests are on 3D vision problems including object pose estimation, shape reconstruction, human pose estimation and image matching. His current work attempts to combine geometry, optimization and learning methods to extract both semantic and geometric information of 3D object and scene from visual data. He coorganized the Workshops on Geometry Meets Deep Learning at ECCV 2016, ICCV 2017 and ECCV 2018 respectively, and the Tutorial on 3D Object Geometry from Single Image at 3DV 2016. He also served in the program committees or as a reviewer for many top conferences and journals such as CVPR, ICCV, ECCV, PAMI and IJCV.
Junyan Zhu is a postdoctoral researcher at MIT CSAIL. He obtained his Ph.D. in EECS from UC Berkeley in 2017 after spending five years at CMU and UC Berkeley. He received his B.E from Tsinghua University. His research interests are in computer vision, computer graphics, and machine learning, with the goal of building machines capable of understanding and recreating our visual world. He is a recipient of Facebook Fellowship, ACM SIGGRAPH Outstanding Doctoral Dissertation Award, and David J. Sakrison Memorial Prize for outstanding doctoral research from the Berkeley EECS. He has served on Technical Paper Committee at SIGGRAPH Asia 2018 and as a guest editor of International Journal of Computer Vision. See more details at http://people.csail.mit.edu/junyanz/.
Tinghui Zhou is a PhD student in the EECS department at UC Berkeley. He obtained my Masters degree in Robotics from Carnegie Mellon University, and Bachelor's degree in Computer Science from University of Minnesota. His research interests are in computer vision, computer graphics, and machine learning. He has published at all major journals and conferences of graphics and vision including four orals papers at CVPR and ECCV.

References (Partial List) 
6D Poses and Dense Correspondences 
[Arrigoni et al., 2016]
F. Arrigoni, B. Rossi, A. Fusiello. Spectral synchronization of multiple views in SE(3), SIAM Journal on Imaging Sciences 9 (4), 19631990, 2016.
[Bernard et al., 2015]
F. Bernard, J. Thunberg, P. Gemmar, F. Hertel, A. Husch, J. Goncalves. A Solution for MultiAlignment by Transformation Synchronisation. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
.
[Bernard et al., 2018]
F. Bernard, J. Thunberg, J. Goncalves, C. Theobalt. Synchronisation of Partial MultiMatchings via Nonnegative Factorisations, Pattern Recognition, 2019.
.
[Chatterjee & Govindu, 2013]
A. Chatterjee and V. M. Govindu. Efficient and robust largescale rotation averaging. In ICCV, pages 521–528. IEEE Computer Society, 2013.
.
[Crandall et al., 2011]
D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher. Discretecontinuous optimization for largescale structure from motion. In CVPR ’11, pages 3001–3008, Washington, DC, USA, 2011. IEEE Computer Society.
.
[Eriksson et al., 2018]
A. Eriksson, C. Olsson, F. Kahl, T. J. Chin. Rotation averaging and strong duality, CVPR, 2018.
.
[Hartley et al., 2013]
R. Hartley, J. Trumpf, Y. Dai and H. Li. Rotation averaging. International Journal of Computer Vision, 2013.
.
[Wei et al. 2016]
L. Wei, Q. Huang, D. Ceylan, E. Vouga and H. Li. Dense Human Body Correspondences Using Convolutional Networks. 15441553. 2016.
.
[Huber & Hebert, 2001]
D. F. Huber and M. Hebert. Fully automatic registration of multiple 3d data sets. Image and Vision Computing, 21:637–650, 2001.
.
[Leonardos et al., 2017]
S. Leonardos, X. Zhou, and K. Daniilidis. Distributed consistent data association via permutation synchronization. In ICRA, pages 2645–2652. IEEE, 2017.
.
[Nguyen et al., 2011]
A. Nguyen, M. BenChen, K. Welnicka, Y. Ye, and L. J. Guibas. An optimization approach to improving collections of shape maps. Comput. Graph. Forum, 30(5):1481–1491, 2011.
.
[Pachauri et al., 2013]
D. Pachauri, R. Kondor, and V. Singh. Solving the multiway matching problem by permutation synchronization. In Neural Information Processing Systems 26, pages 1860–1868, 2013.
.
[Thunberg et al. 2017]
J. Thunberg, F. Bernard, J. Goncalves, Distributed methods for synchronization of orthogonal matrices over graphs, Automatica, 80. pp. 243252, 2017.
.
[Zach et al., 2010]
C. Zach, M. Klopschitz, and M. Pollefeys. Disambiguating visual relations using loop constraints. In CVPR, pages 1426–1433. IEEE Computer Society, 2010.
.
[Zhou et al., 2015]
X. Zhou, M. Zhu, and K. Daniilidis. Multiimage matching via fast alternating minimization. In ICCV,pages 4032–4040, Santiago, Chile, 2015. IEEE Computer Society.
.

DataDriven Object/Part Discovery 
[Huang et al., 2014]
Q. Huang, F. Wang, and L. Guibas. Functional map networks for analyzing and exploring large shape collections. ACM Trans. Graph., 33(4):36:1–36:11, July 2014.
.
[Kim et al., 2012]
V. Kim, W. Li, N. Mitra, S. DiVerdi, and T. Funkhouser. Exploring collections of 3d models using fuzzy correspondences. ACM Trans. Graph., 31(4):54:1–54:11, July 2012.
.
[Ovsjanikov et al., 2012]
M. Ovsjanikov, M. BenChen, J. Solomon, A. Butscher, and L. J. Guibas. Functional maps: a flexible representation of maps between shapes. ACM Trans. Graph., 31(4):30:1–30:11, 2012.
.
[Wang et al., 2013]
F. Wang, Q. Huang, and L. J. Guibas. Image cosegmentation via consistent functional maps. In ICCV, pages 849–856, 2013.
.
[Wang et al., 2014]
F. Wang, Q. Huang, M. Ovsjanikov, and L. J. Guibas. Unsupervised multiclass joint image segmentation. In CVPR, pages 3142–3149. IEEE Computer Society, 2014.
.
[Zhou et al., 2015]
T. Zhou, Y. J. Lee, S. X. Yu, and A. A. Efros. Flowweb: Joint image set alignment by weaving consistent, pixelwise correspondences. In CVPR, pages 1191–1200. IEEE Computer Society, 2015.
.

CrossDomain Matching 
[Yi et al., 2017]
Z. Yi, H. Zhang, P. Tan, and M. Gong. Dualgan: Unsupervised dual learning for imagetoimage translation. CoRR, abs/1704.02510, 2017.
.
[Zhou et al., 2016]
T. Zhou, P. Krähenbühl, M. Aubry, Q. Huang, and A. A. Efros. Learning dense correspondence via 3dguided cycle consistency. CoRR, abs/1604.05383, 2016.
.
[Zhu et al., 2017]
J. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired imagetoimage translation using cycleconsistent adversarial networks. In ICCV, IEEE Computer Society, 2017.
.
[Hoffman et al., 2018]
Judy Hoffman, E. Tzeng, T. Park, J. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell. CyCADA: Cycleconsistent adversarial domain adaptation. In ICML, 2018.
.
[Zhang et al., 2019]
Z. Zhang, Z. Liang, L. Wu, X. Zhou, and Q. Huang. Pathinvariant map networks. In CVPR, IEEE Computer Society, 2019.
.

Theorectical Aspects 
[Huang et al., 2013]
Q. Huang and L. J. Guibas. Consistent shape maps via semidefinite programming. Comput. Graph. Forum, 32(5):177–186, 2013.
.
[Wang & Singer, 2013]
L. Wang and A. Singer. Exact and stable recovery of rotations for robust synchronization. Information and Inference: A Journal of the IMA, 2:145â193, December 2013.
.
[Shen et al., 2016]
Y. Shen, Q. Huang, N. Srebro, and S. Sanghavi. Normalized spectral map synchronization. In NIPS, pages 4925–4933, 2016.
.
[Huang et al., 2017]
X. Huang, Z. Liang, C. Bajaj, and Q. Huang. Translation synchronization via truncated least squares.In NIPS, 2017.
.
[Chen et al., 2014]
Y. Chen, L. J. Guibas, and Q. Huang. Nearoptimal joint object matching via convex relaxation. In ICML, pages 100–108, 2014.
.
[Arrigoni et al., 2018]
F. Arrigoni, A. Fusiello, B. Rossi, and P. Fragneto. Robust rotation synchronization via lowrank and sparse matrix decomposition. Computer Vision and Image Understanding, 2018. 95113.
.
[Zhang et al., 2019]
Z. Zhang, Z. Liang, L. Wu, X. Zhou, and Q. Huang. Pathinvariant map networks. In CVPR, IEEE Computer Society, 2019.
.
[Birdal et al., 2019]
T. Birdal, U. Simsekli. Probabilistic Permutation Synchronization using the Riemannian Structure of the Birkhoff Polytope. https://arxiv.org/pdf/1904.05814.pdf, 2019.
[Birdal et al., 2018]
T. Birdal, U. Simsekli, M. Eken, S. Ilic. Bayesian Pose Graph Optimization via Bingham Distributions and Tempered Geodesic MCMC. In NIPS 2018.


