Navy Promotion Testing Research: Item Exposure Analyses
Technical Report,10 May 2018,31 Mar 2020
PDRI Arlington United States
Pagination or Media Count:
This report describes work conducted by PDRI to assist the Navy in studying item exposure and other exam development policies related to Navy-Wide Advancement Examinations NWAE, one of several factors used in the Navy Enlisted Advancement System to determine advancement to ranks E-4 through E-7. We discuss current practices and recommendations regarding ways to ensure that item and exam development processes are fair, valid, and credible. The work consisted of three components 1 A focused literature review on item exposure 2 analyses of archival data to assess how item exposure across multiple administrations affected item performance and 3 a description of best practices and recommendations on exam development policies concerning item exposure, random and randomized equivalent exams, parallel items, SJTs and other measures. This project underscores the need to carefully, consistently analyze item performance and ensure that trends in data do not change in unexplainable ways. If shifts are identified, tools are available to investigate what caused those changes, along with a variety of solutions. However, the application of appropriate tools is not always straightforward, and may have unanticipated secondary effects. Actions taken must be considered in the context of the overall program to ensure fair, valid and accurate testing.
- Personnel Management and Labor Relations
- Military Forces and Organizations