Report on a Query Generation Technique for Measuring Comprehension of Statistical Graphics

In our information-driven society, there is increasing use of statistical graphics to convey information in a variety of settings, including industry, mass media, government operations, and health care. Current methods for assessing a readers ability to comprehend statistical graphics are custom-written, not widely accepted, usable only once, andor reliant on subjective interpretations and inferences. We have developed a method for generating queries suitable for evaluating graph comprehension capability. Our method is based on the Sentence Verification Technique SVT, an empirically validated framework for measuring an individuals comprehension of prose material. Compared to ad hoc methods for testing graph comprehension, our technique is less subjective, requires less manual effort and subject matter expertise, and addresses the essential features of a given graph values and relationships depicted, frames of reference, and style attributes. The SVT and our derived method combat superficial comprehension by testing what the reader has encoded, as opposed to testing the readers ability at visual recall or ability to look up data without reaching real comprehension. We motivate and describe our query generation method and report on a pilot study using queries generated with it.