NetQuest: Scalable and Flexible Internet Measurement
The Internet connects over 200 million hosts and nearly a billion users
worldwide. How to better understand and thereby control this enormous,
decentralized, and constantly evolving infrastructure is a major
challenge faced by today's researchers, engineers, and network operators. As an
essential means of achieving such better understanding, network measurement is
becoming crucial to a variety of existing and emerging network applications,
such as IP performance management, fault diagnosis, traffic engineering, content
distribution, overlay routing, multihoming, and peer-to-peer applications. These
applications often require the ability to continuously monitor the current
network state and react to changes in a timely fashion. Despite considerable recent progress in network measurement, there are still
several gaps towards fulfilling such requirements.
First, existing network monitoring techniques often focus on accurately measuring the properties of individual links or paths and therefore lack scalability. They rarely scale beyond 100 nodes. In contrast, large-scale network management applications often require the ability to monitor the whole network which may contain orders of magnitude more nodes. Second, robustness issues have not been sufficiently investigated in the context of large-scale network monitoring and inference. Large-scale Internet measurements are inherently noisy due to failures, measurement artifacts, missing data, and even malicious behaviors. How to draw meaningful conclusions based on such noisy data remains a challenging problem that must be addressed in order to make large-scale network monitoring useful in practice. Third, existing techniques are often tailored to monitor specific types of network properties, and thus lack the flexibility to accommodate applications with even slightly different requirements and different performance metrics.
Our research aims to address the above challenges, and develop innovative techniques to design measurement experiments, conduct measurement, and analyze the data. We build our research on top of solid theoretical foundation in statistics, information theory, and optimization, and also leverage considerable practical engineering experience in measuring and managing large operational IP networks.