Rater Accuracy Study.
Final rept. 9 Mar 76-30 Sep 78,
AIR FORCE HUMAN RESOURCES LAB BROOKS AFB TEX
Pagination or Media Count:
Eight hundred eighty-two airmen were divided into more accurate and less accurate rating groups on the basis of their ability to estimate scores of their peers on a vocabulary test. To test whether the method actually did separate more accurate from less accurate raters, correlations were calculated within the more and less accurate groups between ratings of carefulness and scores on carefulness tests and between ratings of decisiveness and scores on decisiveness tests. The analysis consisted of counting the number of times the correlations between test scores and ratings in the more accurate group were larger than the analogous correlations in the less accurate group and computing the probability that this number of differences in the predicted direction might be expected by chance. It appears that this method of identifying accurate raters does work reasonably well. Several auxiliary questions concerning the best qualities to test in the estimating part of the study and the generalizability of rater accuracy to different personality characteristics were asked and answered. Author