Large Scale Self-Organizing Information Distribution System

This project investigates issues in large-scale networks. Here large-scale refers to networks with large number of high capacity nodes and transmission links, and shared by a large number of users. It is almost impossible to simulate such networks and we aim to develop mathematical tools to uncover their structure, illuminate issues, and design solutions. In the three year project period, we have developed a mathematical theory to understand large scale networks under end-to-end control such as the Internet. We have used the theory to explain the deficiencies of the current TOP congestion control algorithm and design new ones. We have implemented the new design as a new protocol FAST TOP. We have worked with a large number of collaborators around the world to test, and refine, the protocol on production networks. The software prototype that we developed has been used by the high energy physics community to break world records on data transfer in the 2003, 2004, and 2005. We have just started a company to transfer the technology to the industry.