Calibration and Validation of the Sage Software Cost/Schedule Estimating System to United States Air Force Databases
AIR FORCE INST OF TECH WRIGHT-PATTERSON AFB OH SCHOOL OF LOGISTICS AND ACQUISITION MANAGEMENT
Pagination or Media Count:
This research entailed calibration and validation of the SAGE Software CostSchedule Estimating System, Version 1.7 as a means to improve estimating accuracy for DoD software-intensive systems, and thereby introduce stability into software system development. SAGE calibration consisted of using historical data from completed projects at the Space and Missile Systems Center SMC and the Electronic Systems Center ESC to derive average performance factors i.e., calibration factors for pre-defined categories of projects. A project was categorized for calibration by either its primary application or by the contractor that developed it. The intent was to determine the more appropriate categorization for calibration. SAGE validation consisted of using the derived calibration factors to predict completed efforts, not used in deriving the factors. Statistical resampling employing Monte Carlo simulation was used to calibrate and validate the model on each possible combination of a categorys projects. Three statistical measures were employed to measure model performance in default and calibrated estimating modes. SAGE generally did not meet pre-established criteria for estimating accuracy, although the model demonstrated some improvement with calibration. Calibration of projects categorized by contractor resulted in better calibrated model performance than calibration of projects categorized by application. This categorization is suggested for future consideration.
- Operations Research
- Computer Programming and Software