Measuring Troubleshooting Skills Using Hardware-Free Simulation
Final rept. 1 Mar 1977-3 Jul 1978
APPLIED SCIENCE ASSOCIATES INC VALENCIA PA
Pagination or Media Count:
Increased emphasis on performance-oriented training in the Air Force has created a need for more valid and reliable feedback on task performance. Traditional multiple-choice tests, while reliable and easy to administer, when related to job entry performance may not possess an acceptable level of validity. The use of actual equipment for job performance testing is expensive from the investment point of view, as well as costly in terms of test administration time and its general low availability to individual students. A possible alternative is a Symbolic Performance Test SPT. Earlier attempted SPTs have generally been part-task analogues. Performance on these earlier tests has been dissimilar to actual troubleshooting performance. The study began with a review of past SPT efforts to determine what was done and to evaluate the strengths and weaknesses of each approach. The troubleshooting task was then analyzed to determine discrete behavioral steps and the information requirements associated with each step. The SPT concept was developed around the results of these analyses. Analysis of the results indicated similar performance on both JST and SPT forms. The accuracy scores for all subjects on all tests produced a positive correlation of .384 which is significant at the .025 level. Time to completion produced a positive correlation of .588 which was significant at the .0005 level. Steps to completion produced a positive correlation of .356 which was significant at the .025 level. Analysis of the check sequences and locations produced very high positive correlations between performance on JST and SPT forms. The primary difficulty encountered was highly variable troubleshooting performance regardless of test form.
- Humanities and History