A Comparison of Variable Selection Criteria for Multiple Linear Regression: A Simulation Study

The purpose of this thesis was to identify three promising least squares selection procedures discussed in the literature during the previous decade and then test these using simulation. The three criteria chosen for this study were minimum mean square error Min MSE, minimum Sp, and minimum Cp. Most of the previous simulations in this area are limited to investigating the usefulness of variable selection criteria when all relevant regressors and some noise variables are available. It is questionable whether all relevant variables will be included. This research has examined the effects of not including a significant variable in the variable pool. In examining each criterion, emphasis was placed on the techniques performance under varying amounts of multicollinearity, variable variation, number of variables, and sample size. Response Surface Methodology was used to determine the effects of varying these factors. A comparison was then made using the results from the Response Surface. To supplement the simulation research a comprehensive literature review of the most current journal articles dealing with several least squares criteria has been provided. This review includes a discussion of each techniques strengths and weaknesses. Since many of the least squares variable selection criteria are addressed, this thesis serves as a useful starting place for various regression questions. Keywords Fortran, Statistical tests.