Analysis of Data from DEOCS Survey: Weighting by Decision Trees, Evaluation of Weights, and Non-Response Bias Analysis
Technical Report,01 Oct 2017,31 Mar 2018
DEFENSE EQUAL OPPORTUNITY MANAGEMENT INST PATRICK AFB FL Patrick AFB United States
Pagination or Media Count:
The dataset of this study is DEOCS survey for October 2017 to March 2018. Decision Tree method has been adopted to compute weights for respondents. Three types of weights have been computed for SAPR, DSPO, and ODMEO, separately. According to the guideline from OPA for evaluating Non-Response Bias NRB, we use these three types of weights to conduct NRB analysis by 1 comparing the composition of the sample compared with survey respondents by key demographics 2 comparing estimates from the NRB follow-up survey. Furthermore, by the technical recommendations from OPA, we evaluate the validity and reliability of the final weights by 1 checking whether all eligible respondents have final weights, and whether all other sample members do not have final weights 2 checking whether the summation of final weights equals the population totals 3 checking the trend line of the key survey weighted estimate overall and in key subpopulations to see whether the changes across waves are reasonable 4 checking the unequal weighting effect UWE of the final weights and the design effects of the weighted estimates to see whether there is large weight variability and whether any extremely large weights have an impact on the weighted estimates. Five algorithms have been developed with built-in Python program in SPSS to conduct NRB analysis and evaluation of weights. The major results are 1 no extreme weights 2 the NRB ratios for all surveys are minor 3 the difference of effects between un- and normalized weights is also little.
- Test Facilities, Equipment and Methods