Ruohan Gao

Ph.D. Candidate
Department of Computer Science
The University of Texas at Austin
Email: rhgao[AT]cs[DOT]utexas[DOT]edu
Office: GDC 4.728F

CV     Google Scholar     GitHub

Short Bio

I am a final-year Ph.D. candidate under the supervision of Prof. Kristen Grauman in the Department of Computer Science at The University of Texas at Austin. I received my B.Eng. degree from the Department of Information Engineering at The Chinese University of Hong Kong (CUHK) in 2015 with First Class Honours. Previously, I have also spent two summers at Facebook AI Research (FAIR) in 2018 and 2019.

My research interests are in computer vision and machine learning. Particularly, I am interested in multimodal learning from videos and embodied learning with multiple modalities. My research has been supported by 2019 Google PhD Fellowship and 2019 Adobe Research Fellowship.

News

  • I co-organized Sight and Sound Workshop at CVPR 2020.

    • Publications


    Learning to Set Waypoints for Audio-Visual Navigation

    Changan Chen, Sagnik Majumder, Ziad Al-Halah, Ruohan Gao, Santhosh K. Ramakrishnan, Kristen Grauman.
    arXiv preprint, 2020.
    arXiv Project Page





    VisualEchoes: Spatial Image Representation Learning through Echolocation

    Ruohan Gao, Changan Chen, Ziad Al-Halah, Carl Schissler, Kristen Grauman.
    European Conference on Computer Vision (ECCV), 2020.
    PDF Supp Data Project Page






    Listen to Look: Action Recognition by Previewing Audio

    Ruohan Gao, Tae-Hyun Oh, Kristen Grauman, Lorenzo Torresani.
    Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

    PDF Supp Poster Project Page Code






    Co-Separating Sounds of Visual Objects

    Ruohan Gao and Kristen Grauman.
    International Conference on Computer Vision (ICCV), 2019.

    PDF Supp Poster Project Page Code








    2.5D Visual Sound

    Ruohan Gao and Kristen Grauman.
    Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
    (Oral Presentation) [Best Paper Award Finalist]
    PDF Project Page Dataset Code Media Coverage Oral Video




    Learning to Separate Object Sounds by Watching Unlabeled Video

    Ruohan Gao, Rogerio Feris, Kristen Grauman.
    European Conference on Computer Vision (ECCV), 2018.
    (Oral Presentation)
    PDF Supp Poster Project Page Code Oral Video




    ShapeCodes: Self-Supervised Feature Learning by Lifting Views to Viewgrids

    Dinesh Jayaraman, Ruohan Gao, Kristen Grauman.
    European Conference on Computer Vision (ECCV), 2018.
    PDF Supp




    Im2Flow: Motion Hallucination from Static Images for Action Recognition

    Ruohan Gao, Bo Xiong, Kristen Grauman.
    Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
    (Oral Presentation)
    PDF Supp Poster Project Page Code Oral Video








    On-Demand Learning for Deep Image Restoration

    Ruohan Gao and Kristen Grauman.
    International Conference on Computer Vision (ICCV), 2017.

    PDF Supp Poster Project Page Code









    Object-Centric Representation Learning from Unlabeled Videos

    Ruohan Gao, Dinesh Jayaraman, Kristen Grauman.
    Asian Conference on Computer Vision (ACCV), 2016.

    PDF Poster Project Page





    Talks

  • Invited Talk at the UTSA AI Consortium Seminar Series, April 2020, "Look to Listen and Listen to Look: Audio-Visual Learning from Video" (PDF, PPT)

  • Invited Talk at the MIT Vision Seminar Series, Sept. 2019, "Learning to See and Hear with Unlabeled Video" (PDF, PPT)

  • Invited Talk at the Sight and Sound Workshop, CVPR'19, "Learning to See and Hear with Unlabeled Video" (PDF, PPT)

  • CVPR'19 Oral, Long Beach, "2.5D Visual Sound" (Video, PDF, PPT)

  • ECCV'18 Oral, Munich, Germany, "Learning to Separate Object Sounds by Watching Unlabeled Video" (Video, PDF, PPT)

  • CVPR'18 Oral, Salt Lake City, "Im2Flow: Motion Hallucination from Static Images for Action Recognition" (Video, PDF)

    • Media Coverage

  • MIT Technology Review: Deep learning turns mono recordings into immersive sound.

  • Two Minute Papers: This AI produces binaural (2.5D) audio.

  • Facebook AI Blog: Creating 2.5D visual sound for an immersive audio experience.

    • Undergraduate Research Publications

    Ruohan Gao, Huanle Xu, Pili Hu, Wing Cheong Lau, “Accelerating Graph Mining Algorithms via Uniform Random Edge Sampling”, IEEE ICC, 2016. [PDF]

    Ruohan Gao, Pili Hu, Wing Cheong Lau, “Graph Property Preservation under Community-Based Sampling”, IEEE Globecom, 2015. [PDF]

    Ruohan Gao, Huanle Xu, Pili Hu, Wing Cheong Lau, “Accelerating Graph Mining Algorithms via Uniform Random Edge Sampling (Poster)”, ACM Conference on Online Social Networks (COSN), 2015.