A Comparison of the Fairness of Adaptive and Conventional Testing Strategies.
MINNESOTA UNIV MINNEAPOLIS DEPT OF PSYCHOLOGY
Pagination or Media Count:
This report examines how selection fairness is influenced by the characteristics of a selection instrument in terms of its distribution of item difficulties, level of item discrimination, degree of item bias, and testing strategy. Computer simulation was used in the administration of either a conventional or Bayesian adaptive ability test to a hypothetical target population consisting of a minority and majority subgroup. Fairness was evaluated by three indices which reflect the degree of differential validity, errors in prediction Clearys model, and proportion of applicants exceeding a selection cutoff Thorndikes model. Major findings are 1 when used in conjunction with either the Bayesian or conventional test, differential prediction increased fairness and facilitated the interpretation of the fairness indices 2 the Bayesian adaptive tests were consistently fairer than the conventional tests for all item pools above the alpha.7 discrimination level for tests of more than 30 items 3 the differential prediction version of the Bayesian adaptive test produced almost perfectly fair performance on all fairness indices at high discrimination levels and 4 the placement of subgroup prior distribution in the Bayesian adaptive testing procedure can affect test fairness. Author