Accession Number:

ADA277761

Title:

Reducing Test Length with Polychotomous Scoring

Descriptive Note:

Final rept. Oct 1988-Sep 1991

Corporate Author:

NAVY PERSONNEL RESEARCH AND DEVELOPMENT CENTER SAN DIEGO CA

Personal Author(s):

Report Date:

1993-10-01

Pagination or Media Count:

28.0

Abstract:

This study compared two new types of test scoring polyweighting and dichotomous Item Response Theory IRT scoring to traditional number-correct NC scoring. The study used data collected from applicants for military enlistment who had taken the Armed Services Vocational Aptitude Battery ASVAB. The objective was to determine the impact of these two new scoring methods on alternate-form reliabilities. Content areas studied were ASVAB Mathematics Knowledge MK and General Science GS. Examinees in one part of the study completed one of five experimental MK tests and one of six operational MK tests. Other examinees in the study completed one of four experimental GS tests and one of six operational MK tests. Other examinees in the study completed one of four experimental GS tests and one of six operational GS tests. Experimental and operational tests were analyzed separately. For each type of test, at each possible test length, median alternate-form reliability was determined for each scoring method. The authors reached five conclusions 1 Polyweighting is superior to NC scoring 2 dichotomous IRT scoring is usually superior to NC scoring, but not always 3 when dichotomous IRT scoring works well, polyweighting and dichotomous IRT scoring allow similar reductions in test length 4 polyweighting and IRT scoring both provide greater benefits when item difficulties are more variable and 5 polyweighting and IRT scoring use different complementary mechanisms for increasing measurement precision

Subject Categories:

  • Psychology

Distribution Statement:

APPROVED FOR PUBLIC RELEASE