NSF Project CNS #2032125

Human Sound Localization and Analytics

Synopsis

COVID-19 is spreading at an unprecedented rate resulting in the death of so many people all over the world. This project proposes to develop techniques and mobile systems that localize human sound such as cough and voice and alarm a user when someone is within the social distance. The goal of the proposed research includes:

- Design algorithms and systems to localize uncontrolled and unknown human sound. The multi-resolution analysis will be performed on low-frequency voice signals to further enhance accuracy.

- Develop applications that can leverage the sound source location.

- Incorporate the research outcome in curriculum and outreach.

Personnel

1. Prof. Lili Qiu

2. Mei Wang

3. Wei Sun

Publication

- Localizing Human Voice. Mei Wang, Wei Sun, Lili Qiu. In Proc. of NSDI 2021.
Abstract: The ability for a smart speaker to localize a user based on his/her voice opens the door to many new applications. In this paper, we present a novel system, MAVL, to localize human voice. It consists of three major components: (i) We first develop a novel multi-resolution analysis to estimate the AoA of time-varying low-frequency coherent voice signals coming from multiple propagation paths; (ii) We then automatically estimate the room structure by emitting acoustic signals and developing an improved 3D MUSIC algorithm; (iii) We finally re-trace the paths using the estimated AoA and room structure to localize the voice. We implement a prototype system using a single speaker and a uniform circular microphone array. Our results show that it achieves median errors of 1.49o and 3.33o for the top two AoAs estimation and achieves median localization errors of 0.31m in line-of-sight (LoS) cases and 0.47m in non-line-of-sight (NLoS) cases.

Outreach

- Talk at IoT Seminar at MIT Fall 2020

- Keynote speech at Comsnets 2022 January 2022

- Keynote speech at ChinaSys 2022 May 2022