Performance Envelopes and Optimal Appropriateness Measurement.
ILLINOIS UNIV AT URBANA MODEL BASED MEASUREMENT LAB
Pagination or Media Count:
The test-taking behavior of some examinees may be so idiosyncratic that their test scores are not comparable to the scores of more typical examinees, Appropriateness indices provide quantitative measures of response-pattern atypicality, An appropriateness index can be viewed as a test statistic for testing a null hypothesis of normal test-taking behavior against an alternative hypothesis of atypical test-taking behavior. In this paper performance curves and the performance envelope are introduced as devices for obtaining a least upper bound for the power of the most powerful statistical tests for aberrance. The performance envelope of a set of tests is the function on 0,1 whose value at t is the least upper bound of the hit rates of the tests when their false positive rate is t. The performance curve of an appropriateness is the performance envelope of the tests for aberrance based on the index. For some types of testing anomalies it is possible to determine the performance envelope for the set of all statistical tests for aberrance and to identify a test whose performance curve is identical to this performance envelope. An algorithm for computing some of these optimal tests is described, and an example of its use is presented. Additional keywords Latent trait theory Item response theory Multiple choice tests Cheating and Copying.