The Relationship Between Reliability and Validity in a Bayesian World.

While the relationship between internal consistency and validity in traditional tests is well established, little is available for tests that predict dichotomous diagnostic outcomes. An understanding of such a relationship is important in cases such as the prediction of clinician diagnoses by psychological tests. Tests such as the MCMI are validated against such clinical diagnoses. The limiting effects on positive predictive powers given levels of Kappa are developed. The current work provides the relationship between Kappa and Positive Predictive Power for use with tests and applies it specifically to a test of psychopathology as an example. This may be applied to any situation where judgements are predicted by tests such as in mental health, medicine, or selection and training.