AppMon: Application Monitors for Not-Yet-Trusted Software

Report developed under STTR contract for topic OSD06-SP2. AppMon represents a novel approach to monitoring the behavior of not-yet-trusted applications that avoids the disadvantages of current approaches. It is based on a self-customizing monitor that constrains the applications use of computer resources. A self-customizing monitor learns how the application normally uses computer resources and does not interfere with normal use. However, when the application uses resources in an unusual way, AppMon prevents potentially harmful accesses. Self-customizing monitors satisfy three important requirements on application security monitors. First, the application can be run immediately without testing or training. Second, customization is automatic, so only minimal demands are made on the user and system administrator. Finally, the self-customizing monitors are applicable to a wide variety of applications, including those that read and write files, read and write registry keys, invoke other processes, and use the Internet.