Knowledge Capture and Deployment: Testing and Evaluation Process

1. SYSTEM TESTING AND DEPLOYMENT Lecture Seven (Chapter 7, Notes; Chapter 8, Textbook)

2. System Testing & Deployment after Knowledge Codification Capture Tools Programs, books, articles, experts Intelligence gathering Shells, tables, tools, frames maps, rules KNOWLEDGE CAPTURE (Creation) KNOWLEDGE CODIFICATION DATABASES TESTING AND DEPLOYMENT Explicit Knowledge KNOWLEDGE SHARING KNOWLEDGE TRANSFER KNOWLEDGE BASE

3. Evaluate Existing Infrastructure Form the KM Team Knowledge Capture Design KM Blueprint Verify and validate the KM System Implement the KM System Manage Change and Rewards Structure Post-system evaluation Stages of KMSLC Iterative Rapid Prototyping

4. Quality Assurance • KM system should meet expectations of members represented in the KM team. • Performance depend highly on quality of explicit/tacit knowledge stored in KB. • For expert, quality relates to reasoning process that produces reliable and accurate solutions. • For user, quality relates to system’s ease of use and efficiency. • For knowledge developer, quality relates to how valid the knowledge sources are and how well they are codified into KB.

5. KMS Testing • To control quality, performance and efficiency of both KB and KMS. • Types of Testing • Logical Testing, make sure that the system produces correct results. • User Acceptance Testing, follows logical testing and check system’s behaviour in a realistic environment.

6. Hurdles in KMS Testing • Subjective nature of tacit knowledge. • Lack of reliable specifications to guide testing • How to decide whether knowledge is complete • Human errors • Lack of time • User interface un-friendly

7. Logical Testing Approaches • Knowledge base formation • Structure of knowledge as it relates to circular or redundant errors • Consistency, correctness, and completeness of knowledge base rules • Knowledge base functionality • Confidence • Reliability of knowledge base

8. Logical Errors • Circular errors- contradictory in meaning or logic • Redundancy errors- offer different approaches to the same problem • Unusable knowledge- comes up no matter conditions succeed or fail • Subsumption errors- if one rule is true, one knows the second rule is always true • Inconsistent knowledge - the same inputs yield different results

9. User Acceptance Testing Approach • Select a person or a team for testing • Decide on user acceptance test criteria • Develop a set of test cases unique to the system • Maintain a log on various versions of the tests and test results • Field-test the system

10. Requirements for Test Team • Independent of the design or codification of the system • Understand the knowledge problem and computer technology • Well versed in the organization’s goals • Know how the system being tested will fit into the designated area of operation

11. Content of Test Plan • Items of KM system that require testing • Schedule of user acceptance testing • Description of test methods that are approved by user and company • Guidelines for documenting test results

12. User Acceptance Criteria • Accuracy and correctness of outcome for test items • Adequacy of the solutions • Adaptability to novel input • Ease of use • Face validity or credibility • Robustness • Technical/operational test

13. User Acceptance Test Techniques • Face validation • Test Cases • Sub-systems validation

14. Factors affecting System Deployment • Technical • Organizational • Procedural • Behavioural • Political • Economical

15. Steps in KMS Deployment • Transfer of KM System from knowledge developer to the organization’s operating unit. • Transfer of KM’s skills from knowledge developer to the organization’s operators.

16. Successful KM System Deployment • User has prior experience with computer applications • User has been involved in the building of the KM system • Benefits of the KM system is immediate and measurable • First impression • Management support the system from start • Adequate user training

17. Factors affecting User Training • Primary • The user’s knowledge of knowledge-based system • Complexity of KM system and user interface’s ease of use • Secondary • Trainer’s technical experience/ communication skills • Environment of training venue

18. Pre User Training Session • Define how the KM system agrees with organizational mission • Demonstrate how the system can support organizational goals • Allocate adequate resources for the training project • Advocate positive effects of the system • Explain cost-benefit analysis of the KM system

19. Overcome Resistances to Change • User attitude survey • Communication sessions • Role negotiation

20. Post-Implementation Review • How KM System has changed accuracy/timeliness of decision making • How KM System has affected attitude of end users • Whether the system has caused constructive organizational changes? • Whether the system has benefited the operation cost?

21. Security Issues • The new system should provide password / protocol protection. • Security procedures should be consistently observed. • Access should be restricted regarding update of the KB.

22. Circular Errors • Rule 1: • IF John and Rob are co-owners of company A THEN John and Rob own the same company • Rule 2: • IF John and Rob own the same company THEN John and Rob are co-owners

23. Redundancy Errors • Rule 1: • IF (Salary > 50,000 OR HOME > 100,000) THEN Loan = ‘yes’ • Rule 2: • IF (Salary > 60,000) THEN Loan = ‘yes’

24. Unusable Knowledge • IF humidity is 5% AND heavy_rain = ‘yes’ THEN some actions

25. Subsumption Errors • Rule 1: • IF A AND B, THEN D • Rule 2: • IF A AND B AND C, THEN D