Experiments in Knowledge Refinement for a Large Rule-Based System

Knowledge-refinement is a central problem in the field of expert systems. For rule-based systems. refinement implies the addition, deletion and modification of rules in the system so as to improve the systems overall performance. The goal of this research effort is to understand the methodology for refining large rule-based systems, as well as to develop tools that will be useful in refining such systems. The vehicle for our investigation is SPAM, a production system rule-based system for the interpretation of aerial imagery. Complex and compute-intensive systems like SPAM impose some unique constraints on knowledge refinement. More specifically, the creditblame assignment problem for locating pieces of knowledge to refine becomes difficult. Given that constraint, we approach the problem in a bottom-up fashion, i.e., begin by refining portions of SPAMs knowledge base, and then attempt to understand the interactions between them. We begin by identifying gaps andor faults in the knowledge base by comparing SPAMs intermediate output to that of an expert, then modifying the knowledge base so that the systems output more accurately matches the experts output. While this approach leads to some improvements, it also raises some interesting issues concerning the evaluation of refined knowledge at intermediate levels and of interaction between the refinements. This paper presents our initial efforts toward addressing these issues. Diagnostic systems, Knowledge refinement, Rule-based systems, Computer vision