Fast and Flexible Application-level Networking on Exokernel Systems

Application-level networking is a promising software organization for improving performance and functionality for important network services. The xokExOS exokernel system includes application-level support for standard network services, while at the same time allowing application writers to specialize networking services. This paper describes how xokExOSs kernel mechanisms and library operating system organization achieves this flexibility, and shares our experiences and lessons learned both positive and negative. It also describes how we have used this flexibility to build and specialize three network data services the Cheetah HTTP server, the webswamp web benchmarking tool, and an application-level TCP forwarder. Overall measurements show large performance improvements relative to similar services built on conventional interfaces in each case reaching the maximum possible end-to-end performance for the experimental platform. For example, Cheetah provides factor of 2-4 increases in throughput compared to highly-tuned socket-based implementations and 3-8 compared to conventional systems. Webswamp can offer loads that are 2-8 compared heavier. The TCP forwarder provides 50-300 higher throughput while also providing end-to-end TCP semantics that cannot be achieved with POSIX sockets. With more detailed measurements and profiling, these overall performance improvements are also broken down and attributed to the specific specializations described, providing server writers with insights into where to focus their optimization efforts.