Validation of the Information/Communications Technology Literacy Test

The United States Army Research Institute for the Behavioral and Social Sciences, supported by the Human Resources Research Organization, conducted the current research effort to validate a measure of cyber aptitude, the Information/Communications Technology Literacy Test (ICTL), in predicting trainee performance in Information Systems Operator-Analyst (25B) and Nodal Network Systems Operator-Maintainer (25N) MOS. This report documents technical procedures and results of the research effort. Results suggest that the ICTL test has potential as a valid and highly efficient predictor of valued outcomes in Signal school MOS. Not only is the ICTL test a valid predictor of job knowledge and performance related criteria such as course grades, but is also a valid predictor of perceived MOS fit. ICTL scores are significantly related to final AIT course grades and perceptions of MOS fit in the 25N MOS. The ICTL test provides appreciable incremental validity beyond ASVAB-based predictors in the 25B MOS. Indices of fairness (e.g., sub-group differences and differential prediction) suggest that the ICTL test generally demonstrates evidence of smaller disparities that those observed in ASVAB-based predictors.