200 likes | 503 Views
Knowledge Engineering and Acquisition Chapter 6 Supplement. Knowledge Acquisition. Knowledge acquisition is the extraction of knowledge from sources of expertise and its transfer to the knowledge base and sometimes to the inference engine.
E N D
Knowledge Acquisition • Knowledge acquisition is the extraction of knowledge from sources of expertise and its transfer to the knowledge base and sometimes to the inference engine
What are some of the Difficulties in Knowledge Acquisition • Expressing the knowledge: • Human knowledge exists in a compiled format. A human doesn’t remember all the intermediate steps used to in transferring and processing knowledge – representation mismatch • Number of participants • Structuring the knowledge: • We must elicit not only the knowledge but also its structure; rules • “Knowers” lack time and unwilling to help • Testing and refining knowledge is hard • Collect knowledge from one source but relevant knowledge is dispersed • Important knowledge may be mixed up with irrelevant information • Incomplete knowledge (use one source only) • “Knowers” may change their behavior when observed • Problematic interpersonal factors
Knowledge Engineering Process Activities • Knowledge Acquisition • Acquisition of knowledge from human experts, books, documents, or computer files • Knowledge Validation • Knowledge is validated and verified (using test cases) until the quality is acceptable • Knowledge Representation • Organized knowledge; creation of a knowledge map and the encoding of knowledge into a knowledge base • Inferencing • Design of software to enable the software to make inferences based on the knowledge and the specifics of the a problem • Explanation and Justification • The design and programming of an explanation capability. Why is this piece of information needed? How was a certain conclusion derived.
Knowledge Engineering Process Knowledge validation (test cases) Sources of knowledge (experts, others) Knowledge Acquisition Encoding Knowledge base Knowledge Representation Explanation justification Inferencing
Knowledge Sources • Documented (books, manuals, etc.) • Undocumented (in people's minds) • From people, from machines • Knowledge Acquisition from Databases • Knowledge Acquisition Via the Internet
Knowledge Acquisition Methods: An Overview • Manual :the knowledge engineer interacts directly with the experts • Interviews, tracking the reasoning process (protocol analysis), observing, brainstorming, conceptual graphs and models • Semiautomatic (Expert-driven): the expert encodes his or her expertise directly into the computer system or the developer uses technology to facilitate the knowledge acquistion • Expert’s self reports, computer aided approaches (visual modeling); graphical development environment where the initial knowledge domain can be modeled and manipulated (decision trees based on business process logic) ex. REFINER+ patient manager • Automatic (Computer Aided - Induction driven) • Minimize or eliminate the role of the KE and/or the expert • inference engines extract the knowledge from a set of examples
Manual Methods of Knowledge Acquisition Experts Elicitation Coding Knowledge engineer Knowledge base Documented knowledge
Expert-Driven Knowledge Acquisition Computer-aided (interactive) interviewing Coding Expert Knowledge base Knowledge engineer
Induction-Driven Knowledge Acquisition Case histories and examples Induction system Knowledge base
Manual Acquisition Techniques • Interviewing: two common types are unstructured (conversational) and structured (interrogation/using a script) • Verbal Protocol Analysis: • Most of the information necessary to model knowledge is found in the cognitive process the knower uses to solve a problem/do a task • Document the step-by-step information processing and decision making behavior by the knower • Concurrent: Think aloud or verbalize thoughts while doing task • Repertory Grid Method: • Maybe manual or computerized
Expert Driven/Computer Aided • Reparatory Grid Analysis • May also be employed by the KE • Developed by Kelly (1955) who conceived humans as ”personal scientist” each with their own model of the world. • the expert compares successive groups of three objects and tells why two differ from the third • Also used to infer similarities in construct beliefs held by multiple experts • Knowledge and perceptions about the world are classified and categorized by each individual as a personal, perceptual model.
Machine learning/Automated Rule Induction • Training set: example of a problem for which the outcome is known • After given enough examples, the rule induction system can create rules that fit the example cases. • The rules can be used to assess new cases for which the outcome is not known. • For Example: Loan Officer’s tasks: Requests for loans include information about the applicants such as income, assets, age and number of dependents
From this case, it is easy to derive the following three rules: • If Income is $70,000 or more approve the loan • If income is $30,000 or more, age is at least 40, assets are above $249,000 and there are no dependents approve the loan • If income is between $30,000 and $50,000 and assets are at least $100,000, approve the loan
Multisource Knowledge Acquisition • It is likely that multiple sources will be needed to fully acquire the knowledge for a problem and conflicting views and opinions often arise. • Brainstorming/Electronic Brainstorming • Goal is to come up with creative solutions. Idea generation and evaluation • Consensus Decision • NGT • Delphi Method • Concept Mapping • Blackboarding
Validation and Verification of the Knowledge Base • Quality Control • Evaluation • Validation • Verification
Evaluation • Assess an expert system's overall value • Analyze whether the system would be usable, efficient and cost-effective • Validation • Deals with the performance of the system (compared to the expert's) • Was the “right” system built (acceptable level of accuracy?) • Verification • Was the system built "right"? • Was the system correctly implemented to specifications?
To Validate an ES • Test • 1. The extent to which the system and the expert decisions agree • 2. The inputs and processes used by an expert compared to the machine • 3. The difference between expert and novice decisions
Accuracy Adaptability Adequacy Breadth Depth Face Validity Generality Precision Realism Reliability Robustness Usefulness Some validation measures