A Comparison of Domain Sampling Procedures for Test Construction

A key issue in test development is how to sample the content domain to ensure that the test best represents the target of measurement. How one approaches the selection of test content reflects a test content theory. This study was an effort to investigate the effects of different test content theories, as operationalized by the domain sampling procedures used, on test scores and the inferences that can be made from those scores. Sample tests were constructed and compared to each other in terms of test reliability, content validity, and construct validity to assess the relative efficacy of different domain sampling strategies across varying item sample sizes. It was hypothesized that tests constructed using a strong test construct theory approach would yield greater evidence of reliability, content validity and construct validity and that the efficacy of strong test content theory test construction would increase as test length decreased. Neither hypothesis was supported. Implications of these results and suggestions for future research are discussed. Content validity, Test validation, Domain sampling, Test construction.