Development of an Automated Scoring System to Support Soldier Assessment with the Consequences Test

Measures of creativity, and particularly divergent thinking, such as the Consequences Test (Guilford and Guilford, 1980), have been shown to be good predictors of important aspects of career performance in the Army, including continuance and progression. These scales describe unique situations and require examinees to list implications that might arise from those situations. Until recently the requirement to have expert humans score the responses has made this test impractical to use on a large scale. Following the development of automated scoring models that can score these responses as well as expert humans, this test of creativity is being used operationally by the U.S. Army. In the latest two-year effort, we developed new scoring models for seven additional test items, increasing the available number of test items in order to improve test security. We also developed new scoring tools that incorporates the five original test items, whose scoring models were developed in an earlier project in 2013-2018, with the new seven items. The scoring models perform well, and the new scoring tool provides an alternative to human scoring that can be integrated with other assessment programs.