Effective Network Management via System-Wide Coordination and Optimization

As networked systems grow and traffic patterns evolve, management applications are increasing in complexity and functionality. To address the requirements of these management applications, equipment vendors and administrators today depend on incremental solutions that increase the complexity of network elements and deployment costs for operators. Despite this increased complexity and cost, the incremental nature of these solutions still leaves a significant gap between the policy objectives of system administrators and todays mechanisms. These challenges arise in several application contexts in different networking domains ISPs, enterprise settings, and data centers. Much of this disconnect arises from the narrow device-centric view of current solutions. Such piecemeal solutions are inefficient network elements duplicate tasks and some locations become overloaded. Worse still, administrators struggle to retrofit their high-level goals within device-centric configurations. This dissertation argues for a clean-slate system-wide approach for resource management in large-scale networked systems based on three high level principles 1 systematic selection and placement of device-level primitives 2 lightweight coordination mechanisms that enable different network elements to effectively complement one another, and 3 practical optimization models that capture operating constraints and policy objectives. This dissertation demonstrates the benefits of this system-wide approach in three application contexts 1 meeting fine-grained coverage and accuracy requirements in traffic monitoring, 2 implementing a redundancy elimination service to improve network performance, and 3 managing the deployment of intrusion detection and prevention systems.