Knowledge Discovery Systems: Systems That Create Knowledge

1. Knowledge Discovery Systems:Systems That Create Knowledge

2. Knowledge-Based Systemsfor KM • A knowledge-based system is “a computerized system that uses domain knowledge to arrive at a solution to a problem within that domain. This solution is essentially the same as one concluded by a person knowledgeable about the domain, when confronted with the same problem.” • Knowledge-based systems are an excellent platform for capturing, sharing, and applying knowledge (of certain kinds). • Knowledge-based systems were designed primarily for the purpose of being able to apply knowledge automatically.

3. Knowledge-Based Systemsfor KM • Key differences from conventional software: • The use of highly specific domain knowledge. • The heuristic nature of the knowledge employed, instead of exact. • The separation of the knowledge from how it is used.

4. Knowledge Synthesis through Socialization • To discover tacit knowledge • Socialization enables the discovery of tacit knowledge through joint activities • between masters and apprentices • between researchers at an academic conference

5. Knowledge Discovery from Data –Data Mining • Another name for Knowledge Discovery in Databases is data mining (DM). • data mining is the process of analyzing data from different perspectives and summarizing it into useful information. • Data mining software is one of a number of analytical tools for analyzing data. It allows users to analyze data from many different dimensions or angles, categorize it, and summarize the relationships identified. • Technically, data mining is the process of finding correlations or patterns among dozens of fields in large relational databases.

6. Data Mining Techniques Applications • Marketing–Predictive DM techniques, like artificial neural networks (ANN), have been used for target marketing including market segmentation. • Direct marketing–customers are likely to respond to new products based on their previous consumer behavior. • Retail–DM methods have likewise been used for sales forecasting. • Market basket analysis–uncover which products are likely to be purchased together.

7. Data Mining Techniques Applications • Banking–Trading and financial forecasting are used to determine derivative securities pricing, futures price forecasting, and stock performance. • Insurance–DM techniques have been used for segmenting customer groups to determine premium pricing and predict claim frequencies. • Telecommunications–Predictive DM techniques have been used to predict when customers will attrition to a competitor. • Operations Management - have been used for planning and scheduling, project management, and quality control.

8. Designing the Knowledge Discovery System –CRISP DM • Business Understanding–To obtain the highest benefit from data mining, there must be a clear statement of the business objectives. • Data Understanding–Knowing the data well can permit the designer to tailor the algorithm or tools used for data mining to his/her specific problem. • Data Preparation –Data selection, variable construction and transformation, integration, and formatting • Model building and validation– Buildingan accurate model is a trial and error process. The process often requires the data mining specialist to try several options, until the best model emerges. • Evaluation and interpretation–Once the model is determined, the validation dataset is fed through the model. • Deployment –Involves implementing the ‘live’ model within an organization to aid the decision making process.

9. Business Understanding process • Determine Business objectives–To obtain the highest benefit from data mining, there must be a clear statement of the business objectives . • Situation Assessment–Themajority of the people in a marketing campaign who receive a target mail, do not purchase the product . • Determine Data Mining Goal–Identifying the most likely prospective buyers from the sample, and targeting the direct mail to those customers, could save the organization significant costs. • Produce Project Plan–This step also includes the specification of a project plan for the DM study .

10. Data Understanding process • Data collection– Definesthe data sources for the study, including the use of external public data, and proprietary databases. • Data description– Describes the contents of each file or table. Some of the important items in this report are: number of fields (columns) and percent of records missing. • Data quality and verification–Define if any data can be eliminated because of irrelevance or lack of quality. • Exploratory Analysis of the Data– Useto develop a hypothesis of the problem to be studied, and to identify the fields that are likely to be the best predictors.

11. Data Preparation process • Selection –Requires the selection of the predictor variables and the sample set. • Construction and transformation of variables–Often, new variables must be constructed to build effective models. • Data integration– The dataset for the data mining study may reside on multiple databases, which would need to be consolidated into one database. • Formatting– Involves the reordering and reformatting of the data fields, as required by the DM model.

12. Model building and Validation process • Generate Test Design • Building an accurate model is a trial and error process. The data mining specialist iteratively try several options, until the best model emerges. • Build Model • Different algorithms could be tried with the same dataset. Results are compared to see which model yields the best results. • Model Evaluation • In constructing a model, a subset of the data is usually set-aside for validation purposes. The validation data set is used to calculate the accuracy of predictive qualities of the model.

13. Evaluation and Interpretation process • Evaluate Results • Once the model is determined, the predicted results are compared with the actual results in the validation dataset. • Review Process • –Verify the accuracy of the process. • Determine Next Steps • –List of possible actions decision.

14. Deployment process • Plan Deployment • This step involves implementing the ‘live’ model within an organization to aid the decision making process.. • Produce Final Report • Write a final report. • Plan Monitoring and Maintenance • Monitor how well the model predicts the outcomes, and the benefits that this brings to the organization. • Review Project • Experience, and documentation.

15. Barriers to the use of DM • Two of the most significant barriers that prevented the earlier deployment of knowledge discovery in the business relate to: • Lack of data to support the analysis • Limited computing power to perform the mathematical calculations required by the DM algorithms.

16. Knowledge Capture Systems: Systems that Preserve and Formalize Knowledge

17. What are Knowledge Capture Systems? • Knowledge capture systems support process of eliciting explicit or tacit knowledge from people, artifacts, or organizational entities • Rely on mechanisms and technologies to support externalization and internalization

18. Using Stories for Capturing Organizational Knowledge • Organizational stories: • “a detailed narrative of past management actions, employee interactions, or other intra-or extra-organizational events that are communicated informally within organizations” • include a plot, major characters, an outcome, and an implied moral • play a significant role in organizations characterized by a strong need for collaboration

19. Using Stories for Capturing Organizational Knowledge • Guidelines for organizational storytelling: • Stimulate the natural telling and writing of stories • Should not represent idealized behavior • An organizational program to support storytelling should not depend on external experts for its sustenance • Organizational stories are about achieving a purpose, not entertainment • Be cautious of over-generalizing and forgetting the particulars • Adhere to the highest ethical standards and rules

20. Using Stories for Capturing Organizational Knowledge • Important considerations: • Effective means of capturing and transferring tacit organizational knowledge • Identify people in the organization willing to share how they learned from others • Use metaphors to confront difficult organizational issues • Storytelling provides a natural methodology for nurturing communities because it: • builds trust • unlocks passion • is non-hierarchical

21. Where can storytelling be effective? • Igniting action in knowledge-era organizations • Bridging the knowing-doing gap • Capturing tacit knowledge • To embody and transfer knowledge • To foster innovation • Enhancing technology • Individual growth • Launching/Nurturing communities of practice • thematic groups(World Bank) • learning communities or learning networks(HP) • best practice teams(Chevron) • family groups(Xerox)

22. Knowledge Representation through the use of Concept Maps • Based on Ausubel’s learning psychology theory • Concepts, enclosed in circles or boxes. are perceived regularities in events or objects designated by a label • Two concepts connected by a linking word to form a proposition, semantic unit or unit of meaning • Vertical axis expresses a hierarchical framework for organizing the concepts • inclusive concepts are found at the top, progressively more specific, less inclusive concepts arranged below • relationships between propositions in different domains are cross-links

23. Knowledge representation through context-based reasoning • Tactical knowledge • human ability that enables domain experts to assess the situation at hand(therefore short-term) • myriad of inputs, select a plan that best fits current situation, and executing plan • recognize and treat only the salient features of the situation • gain a small, but important portion of the available inputs for general knowledge

24. Knowledge representation through CxBR • Context-set of actions and procedures that properly address the current situation • As mission evolves, transition to other context may be required to address the new situation • What is likely to happen in a context is limited by the context itself

25. Knowledge representation through CxBR • Mission Context-defines the scope of the mission, its goals, the plan, and the constraints imposed • Main Context-contains functions, rules and a list of compatible subsequent Main Contexts • Sub-Contexts-abstractions of functions performed by the Main Context which may be too complex for one function

26. Knowledge Capture Systems based on CxBR • Context-based Intelligent Tactical Knowledge Acquisition(CITKA) • 􀂊uses its own knowledge base to compose a set of intelligent queries to elicit the tactical knowledge of the expert • 􀂊composes questions and presents them to the expert • 􀂊result is a nearly complete context base can be used to control someone performing the mission of interest in a typical environment

27. Barriers to the use of knowledge capture systems • Barriers to the deployment of knowledge capture systems from two perspectives: • 􀂊the knowledge engineer who seeks to build such systems • 􀂊the subject matter expert, who would interact with an automated knowledge capture system to preserve his knowledge

28. Barriers to the use of knowledge capture systems • Knowledge Engineer requires developing some idea of the nature and structure of the knowledge very early in the process • must attempt to become versed in the subject matter, or the nature of knowledge • An automated system for knowledge capture, without a-priori knowledge of the nature, is essentially not possible