On Measuring the Effectiveness of Large-Scale Training Simulations
INSTITUTE FOR DEFENSE ANALYSES ALEXANDRIA VA
Pagination or Media Count:
This paper identifies and evaluates methods for measuring the benefit or utility of large-scale training simulations LSTSs. The intended audience is those people who are directly responsible for assessing the training effectiveness of LSTSs. We examine literature related to the measurement of LSTS training effectiveness INCLUDING GUIDANCE PROVIDED BY THE INDIVIDUAL MILITARY Services and the Office of the Secretary of Defense OSD, the body of research on performance measurement, and research practices related to experimental design and analysis. We then synthesize the findings from these literature sources to provide seven recommendations for performing training effectiveness analyses of LSTSs 1 identify specific measurement issues, 2 devise a measurement plan that specifies performance measurement and research design, 3 use valid and reliable performance measures, 4 obtain the most valid measure of proficiency possible within the constraints of the evaluation, 5 to the extent possible and reasonable, impose experimental control on the research situation, 6 use analytic models in early stages and continue using these models throughout system development, 7 and include user reactions as an adjunct to performance data and analysis.
- Administration and Management
- Personnel Management and Labor Relations
- Military Forces and Organizations