Evaluation of Visualization Tools for Computer Network Defense Analysts: Display Design, Methods, and Results for a User Study
Technical Report,01 Jan 2013,30 Sep 2015
ARMY RESEARCH LAB ABERDEEN PROVING GROUND MD ABERDEEN PROVING GROUND United States
Pagination or Media Count:
Computer network defense CND analysts serve an increasingly vital role in the defense of our nations computing infrastructure. An important component of their work is the monitoring of suspicious activity identified by an intrusion detection system IDS. While analysts are trained to quickly recognize abnormal patterns in textual log files, humans are generally not well suited for such processing in any large quantity. Many authors have proposed the use of visualization techniques to aid the cyber security analysts search activities however, such techniques are not widely used by analysts. This report describes an evaluation of 2 graphical displays a parallel coordinates display and a node-link display compared against a traditional tabular arrangement with the goal of better understanding analyst performance and obtaining subjective feedback on the graphical alternatives. Both expert analysts and novices students participated in the study. Results show that analysts generally preferred familiar tools but were able to use some graphical alternatives node-link to achieve similar performance in less time. Students were not found to be effective surrogates for experienced analysts for researchvalidation of techniques. This report describes the development and design of the displays and the experiment, and provides insight into analyst needs and evidence on effective methods for validating cyber defense visualization tools based on results obtained.
- Computer Systems Management and Standards