UTCS Colloquia - Yaser Sheikh, Assistant Research Professor, Robotics Institute/Carnegie Mellon University, "The Panoptic Studio: Perceiving Subtle Behavior in Social Interactions using Hundreds of Cameras"

Contact Name: 
Chandrajit Bajaj
POB 6.304
Feb 19, 2014 11:00am - 12:00pm

Signup Schedule: http://apps.cs.utexas.edu/talkschedules/cgi/list_events.cgi

Talk Audience: UTCS Faculty, Grads, Undergrads, Other Interested Parties

Host:  Chandrajit Bajaj

Talk Abstract: In this talk, I will present recent advances in the measurement and modeling of subtle human motion. In particular, I will describe a massively multi-view system designed to capture the motion produced by polyads, i.e., multiple people engaged in a social interaction. Polyadic interactions present three challenges to motion capture methods: (1) subtle details that encode crucial information about the interaction need to be resolved over a volume large enough to house several people, (2) severe occlusions frequently occur in any small subset of views, and (3) no dominant direction of observation exists. 

Our system uses 500 synchronized cameras of hybrid resolutions mounted over the surface of a geodesic dome to capture detailed motion of approximately fifty thousands of points over volumes sufficient for polyadic interactions of ten people. Using a dataset with participants involved in polyadic games (to evoke natural interactions), we establish the performance of this large-scale multi-view system by demonstrating robustness to occlusion, increased capture density, and agnosticism to the orientation of activity. 

I will show raw 3D reconstruction of unprecedented motion detail over an order of magnitude longer durations than have been previously recorded, and describe our progress in modeling the motion data.

Speaker Bio: Yaser Sheikh is an Assistant Research Professor at the Robotics Institute, Carnegie Mellon University, with appointments in the Mechanical Engineering Department and the Quality of Life Technology Center. His research interests span computer vision, computer graphics, and robotics, and are focused on the machine perception of social behavior. He has won the Honda Initiation Award (2010), best paper awards at various conferences and workshops (SAP (2012), WACV (2012), SCA (2010), and ICCV THEMIS (2009)), and the Hillman Fellowship for Excellence in Computer Science Research (2004). He has over 50 publications in leading conferences in computer vision, computer graphics, and machine learning, and hold four patents on his research. Yaser chairs leading conferences in computer vision, computer graphics, and robotics including ICCP (2011), SIGGRAPH (2013, 2014), CVPR (2014), and ICRA (2014). His research is sponsored by various government research offices, including NSF and DARPA, and several industrial partners including the Intel corporation, the Walt Disney Company, Nissan, Honda, and Samsung Group. Yaser's research has been covered in various media outlets including New Scientist, BBC, slashdot, and WIRED.