Reweighting AT-SAT to Mitigate Group Score Differences
FEDERAL AVIATION ADMINISTRATION OKLAHOMA CITY OK CIVIL AEROMEDICAL INST
Pagination or Media Count:
The Air Traffic Selection and Training AT-SAT test battery is the selection tool for applicants for Air Traffic Control Specialist ATCS positions within the FAA who have not previously been employed as an air traffic controller. AT-SAT is an aptitude test developed to predict the likelihood of successfully learning ATCS skills. Before operational use, however, concerns were raised about the low passing rate of incumbent ATCS personnel who participated in the initial research and score differences between groups, which could result in possible unfair discrimination. To address these concerns, the subscores of AT-SAT were reweighted, and the additive constant was changed to yield a new total score. This study compares the original and new scoring methods using data from 724 developmental ATCSs who volunteered to take the AT-SAT. An average increase of 4.86 points was found with the new scoring method the notional passing rate achieving a score 70 changed from 58.8 to 80. American IndianAlaskan Native, Hispanic, and black participants showed the greatest average increase in overall scores i.e., 6.97, 6.98, and 7.02, respectively. The increase in scores of Hispanic and black participants was significantly higher than the increase in scores for white participants F4,689 6.186, p .001. However, a chi square analysis showed no differences between groups for the participants whose failing score with the original scoring method changed to a passing score with the new method. A Spearman rank correlation coefficient of .85 was found between the two scoring methods, indicating that the ranking of individual participants did not change significantly. No differences were found between groups in rank ordering of the two scoring methods, and no significant gender differences were found between the scoring methods. The study found that the new weighting formula has benefited all groups and is likely to reduce the potential of adverse impact.
- Commercial and General Aviation
- Personnel Management and Labor Relations
- Numerical Mathematics