Knowledge Acquisition and Modelling

1. Knowledge Acquisition and Modelling Knowledge Acquisition and Elicitation Ref: Knowledge Acquisition in Practice: A step by step guide, Milton, Springer-Verlag

2. Knowledge Engineering • Transfer View • Human knowledge transferred to knowledge base • =>knowledge exists and is accessible • Typically interviews and task execution and observation used for KA • End result set of rules that exercise knowledge made explicit • Modelling View • Need to build models • Incremental, evolutionary process • Model is an approximation of reality • Models are subjective

3. KA Typology unstructured interview semi-structured interviews interview structured interview natural techniques observation techniques group meetings questionnaires card sorting three card trick rep grid technique KA techniques limited time contrived techniques constrained tasks limited information 20-questions commentating teach back laddering process mapping modelling techniques concept mapping state diagram mapping

4. Natural Techniques

5. Interview Techniques • Knowledge engineer asks questions of the expert or end user. • Essential method for acquiring explicit conceptualisations and knowledge, but poor for tacit knowledge. • Variations: • Unstructured interview • Free flowing, used in early stages of elicitation, can produce basics of knowledge domain, basically broad chat • Semi-structured interview • Main technique for elicitation • Pre-defined questions sent to expert prior to interview, supplementary questions asked at interview. Can be used as part of validation. • Structured interview • Pre-defined set of questions, can simply be filling in a questionnaire at the interview.

6. Interview Techniques • Dependent on • The questions asked • Ability of the expert to articulate the knowledge • Model built on knowledge elicited during interview • Model reviewed by the expert

7. Modelling Techniques

8. Modelling Techniques • Use of knowledge models with experts • Used as validation and refinement • Can show a basic model to an expert and prompt them to modify. • Can show a complete model of knowledge provided by one expert to a second expert to cross-validate. • Can create one from scratch with an expert – start with a blank page

9. Model Based Knowledge Acquisition • Each model emphasizes certain aspects of thesystem to be built and abstracts from others. • Each model is indicative of one view of the world • Models provide a decomposition ofknowledge-engineering tasks: • while buildingone model, the knowledge engineer cantemporarily neglect certain other aspects.

10. Knowledge Modelling Process

11. Knowledge Modelling • Use skeletal models • Or generic tasks • Generic tasks are templates of problem-solving activities that can be configured together to describe any intelligent activity. • Modelling Frameworks

12. Knowledge Modelling • At least five different types of knowledge are distinguished: • Tasks-goals • correspond to the goals that must be achieved during problem solving. • Problem-solving methods • ways to achieve the goals described in tasks. In some knowledge modelling frameworks, problem-solving methods define subtasks to which other problem solving methods can be applied. We will call such a decomposition a task instance. • Inferences • describe the primitive reasoning steps in the problem solving process. • Ontologies • describe the structure and vocabulary of the static domain knowledge. • Domain knowledge • refers to a collection of statements about the domain.

13. Principles • Divide and conquer. • Configuration of an adequate model set for a specific application. • Models evolve through well defined states. • The model set supports project management. • Model development is driven by project objectives and risk. • Models can be developed in parallel.

14. Recommended Reading Knowledge Engineering: Principles and Methods Rudi Studer, V. Richard Benjamins and Dieter Fense Data & Knowledge Engineering (1998) Volume: 25, Issue: 1-2, Publisher: Elsevier • http://www.hubscher.org/roland/courses/hf760/readings/studer98knowledge.pdf

15. Contrived Techniques

16. Knowledge Capture – Specialised Techniques • Contrived Techniques • Primarily for deep, tacit knowledge • Involve the expert performing tasks they would not normally do as part of their job. • Most of these techniques come from psychology

17. Knowledge Capture – Specialised Techniques • Important types: • Concept (card) Sorting • Three Card Trick (Triadic) • Repertory Grid Technique • Constrained Tasks • 20-questions • Commentary • Teach Back • Usually involve expert doing two types of task: • Tasks they normally perform • Commentary is useful here • Tasks designed to probe the expert • Concept sorting or Triadic

18. Concept (Card) Sorting • Way of finding out how an expert compares and orders concepts • Can reveal knowledge about classes, properties and relations • Works best in small groups • Simplest form is card sorting • Collection of concepts (or other knowledge objects) are written on separate cards • Cards sorted into piles by an expert in to piles - each card in a pile must have something in common • Each time the cards are sorted it will be based on an attribute and each pile will represent a value • Enables significant elicitation of properties and dimensions • Used to capture concept knowledge and tacit knowledge • Use in conjunction with triadic method • Can also sort objects or pictures instead of cards

19. Concept Sorting – How To ? • Decide what classes of concepts you want to explore (in particular their properties – attributes and values) • Write the name of each concept on a separate card • At the session explain to the expert what is going to happen • Ask the expert to name the piles • Write down (or record) the results of the sort • Collect the cards and ask the expert to sort again • Repeat until the expert can’t sort anymore

20. Triadic Elicitation Method (3 card trick) • Used to capture the way in which an expert views the concepts in a domain. • Present three random concepts and ask in what way two of them are similar but different from the other one. • Answer will give an attribute. • A good way of acquiring tacit knowledge. • How does it work ? • Select 3 cards at random • Identify which 2 cards are the most similar? • – Why? • – What makes them different from the third card? • Helps to determine the characteristics of our classes • Picking 3 cards forces us into identifying differences between them • There will always be two that are “closer” together • Although which two cards that is may differ depending on your perspective

21. Triadic Elicitation – How To? • Explain to the expert that you are trying a technique to draw out deeper knowledge • Collect all cards previously used • Shuffle cards and randomly select 3 • Place them on the table, two close together one further away • Ask how the two close together are similar and the other different • Write down (or record) what the expert says using an attribute • Use the results to find another card sort to find the values of all concepts for this attribute • If the expert can’t identify an attribute, just pick another 3 cards • Repeat until the expert can think of no more differences

22. 20-Questions • Expert asks questions of the engineer • The Knowledge Engineer thinks of an object/concept in the domain • Expert asks yes/no questions to the knowledge engineer in order to deduce an answer. • Knowledge Engineer • notes the questions and the order in which they are asked • need not know much about the domain, or have an answer in mind, just answer “yes” or “no” randomly • The questions asked provide a good way of quickly acquiring attributes in a prioritised order. • Can provide an insight into the key aspects, properties or categories and their relative priorities. • Note that the main purpose of this exercise is not really to try and find out what the Engineer is thinking of, but to determine the important properties!

23. 20-Questions – How To? • Decide on set of concepts you need to explore in more detail • Explain to the expert what is going on • Ask the expert to imagine that you the engineer have the same level of knowledge they do about the set of concepts • Instruct the expert that they should ask the least number of questions to deduce the answer • Engineer can only answer yes and no • Explain that the best way is to ask questions which split the concepts in half so that each question eliminates half the possible solutions • Start • As each question is asked write it down (or record it) • When a number of questions have been asked take the expert back to an earlier question and change the answer you gave to prompt the expert to ask further questions • After the session extract the attributes and values (or new concepts) from the questions asked and these will be added to the knowledge base

24. Laddering • Involves the construction, modification and validation of trees. • Accessing personal construct system • Take a group of things and ask what they have in common • Then what other ‘siblings’ (brothers/sisters) there might be • A valuable method for acquiring concept knowledge and, to a lesser extent, process knowledge. • Can make use of various trees: • concept tree • composition tree • attribute tree • process tree • decision tree • cause tree

25. Example Source: Bourne and Jenkins , Eliciting Managers' Personal Values: An Adaptation of the Laddering Interview Method, Organizational Research Methods, SAGE 2005

26. Concept Tree • Hierarchical diagram of concepts showing classes and members • Activities to create • Move nodes (concepts) around the tree • Add new node • Deleting nodes • Renaming nodes • Difficulty is avoiding the problems which requires knowing: • All links on the tree represent an ‘is-a’ relationship • Terminology to describe the tree • What classes to use in the tree • Naming conventions to use • How to deal with complex cases (e.g. multiple parents, synonyms)

27. lorry car traffic steam ship sailing ship ship vehicle shipping lanes pollution congestion traffic issues Road safety Concept Tree – ‘is-a’ relationship • Is-a = is a type of • Different to ERDs What are the mistakes in this tree?

28. Concept Tree - Terminology • Root node • Leaf node • Branch • Parent • Children • Descendants

29. Concept Tree – What classes to use? • Class is a concept which has children on a tree • Other concepts are related to it by an is-a relationship • To develop classes use either a top-down or bottom-up approach • Top-down start with a set of general classes and refine • Bottom-up start to develop classes by grouping those concepts that are similar

30. Repertory Grid technique • Used to elicit attributes for a set of concepts • Used to rate concepts against attributes using a numerical scale • Uses statistical analysis to arrange and group similar concepts and attributes • Allows the expert to provide a rating of each concept for an attribute in concept sorting • A useful way of capturing concept knowledge and tacit knowledge • When many ratings are provided using many attributes statistics can be applied to find clusters and correlations • Requires special software

31. Repertory Grid – How To? • 1st stage • Concepts are selected (between 6 & 15) • Set of approx. same no. of attributes is also required • Should be such that values can be rated on a continuous scale (e.g. small to large) • Chosen from knowledge previously elicited • 2nd stage • Expert provides a rating for each concept against each attribute • Numerical scale is used • 3rd stage • Ratings are applied to cluster analysis to create a visual representation of the ratings called a focus grid • Concepts with similar scores will be grouped together, attributes with similar scores will be grouped • 4th stage • Engineer walks expert through the results to gain feedback and prompt for further knowledge about the groupings • If needed more concepts and attributes are rated and included in the grid

32. Repertory Grid Example • Domain elements are certain types of crime: petty theft, burglary, drug-dealing, murder, mugging and rape. • This is one expert’s view on the issue. • Consider carefully whether any pair of attributes are very similar, by comparing horizontal lines in this grid. • The closest is probably the personal/impersonal one and the major/petty one. • Beware, when making this comparison, that the expert may have inadvertently ‘inverted’ the scale for just one of two similar constructs. • For example, in the example the major/petty construct has a value of 5 for ‘major’. If the expert had chosen 1 instead, and 5 for ‘petty’, then this construct and the personal/impersonal one would look very different. • Further analysis may lead you to omit one pairing of constructs. • Following that you would draw up a table showing how similar or dissimilar each domain element is from the others. • For example, when the absolute-value metric is used, the (numeric) difference.

33. Constrained Tasks • Expert performs a task they would normally do, but with constraints. • Variations: • limited time • limited data • Useful for focusing the expert on essential knowledge and priorities

34. Commentary and protocol generation • Expert provides a running commentary of their own or another’s task performance. • A valuable method for acquiring process knowledge and tacit knowledge. • Variations: • self-reporting • imaginary self-reporting • self-retrospective • shadowing • retrospective shadowing

36. Knowledge Analysis and Modelling

37. Knowledge Analysis • Identifying the elements needed to build the knowledge base • Concepts • Things that constitute a domain • Main elements of the k-base • Attributes • Qualities or features belonging to a class of concepts • Values • Specific qualities or features of a concept that differentiate it from other concepts • Relations • Way in which concepts are associated with one another

38. Concepts • Physical concepts • Products, components, machines • Pieces of information • Plans, goals, requirements • Sources of information • Documents, databases, websites • People and roles • Experts, roles of experts • Organisations and groups • Producers, suppliers, consumers, departments • Areas of knowledge • Marketing, physics, chemistry • Functions • Purpose of components or roles • Tasks • Activities performed by experts • Issues • Problems, solutions, advantages, disadvantages • Physical phenomena • Mechanisms and forces • Other issues • Constraints, behaviours, states

39. Attributes • Of physical objects • Shape, age • Of information • Source, format, importance • Of people • Gender, age, personality • Of organisation • Size, turnover, product range

40. Values • Come in different varieties • Dependent on type • Adjective, number, sentence, paragraphs, hyperlinks, images, pictures • Categorical • For values that are adjectives • Numerical • For values that are numbers • Text • For values that are one or two sentences • Hypertext • For values that are chunks of hypertext

41. Relations • Has part • Performs • Followed by • Requires • Causes • Produces • Can have an inverse relation • Short exercise • Think of something that illustrates each one of these

42. Knowledge modelling • K-model = way of viewing the knowledge in the k-base • Each model provides a different perspective on the knowledge • Helps clarify the ‘mess’ that is the knowledge • Can be used in elicitation

43. Trees • Diagram showing hierarchical arrangement of nodes • Node = concept • Link = relationship • Concept tree • Composition tree • Cause tree • Mixed tree

44. Concept tree • Each link is an is-a relation • Taxonomy • Read from right to left Taken from www.pcpack.co.uk

45. Other types of tree • Composition tree • All links are has-part • Used to show components and sub-components of a concept • Process tree • Special form of composition tree • All nodes are tasks • Attribute tree • Shows attributes and values to describe a concept • Mixed tree • Contains more than one type of relation

46. Matrices • Attribute matrix • Presents set of properties of a concept (attributes and values) • Concepts on vertical axis • Attributes and values on horizontal axis

47. Relationship matrix • Shows two sets of concepts related to one another using a specified relationship • Cells show which pairs of concepts have the relationship

48. Maps • Shows an arrangement of nodes linked by arrows • Each node represents concept • Link represents relationship • Concept maps • Process maps

49. Concept map • Many different types

50. Knowledge Analysis – How to? • How do you identify concepts from interview transcripts and documents? • Need some codification • Highlighters – different colours for different things