1 / 25

Modeling Human Reasoning About Meta-Information

This research focuses on modeling human reasoning and behavior by incorporating meta-information in Bayesian Belief Networks. It explores the impact of meta-information on decision-making processes and aims to replicate human cognitive processes in real-time environments.

roxannew
Download Presentation

Modeling Human Reasoning About Meta-Information

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Modeling Human Reasoning About Meta-Information Presented By: Scott Langevin Jingsong Wang

  2. Introduction • Human decision-making in real-time, dynamic environments is becoming more complex • Decision-makers must manage large amounts of incoming information and integrate it with previous knowledge to develop a “situational awareness” • Relies on domain-knowledge but also on the qualifiers (meta-information) describing the information • Problem: To replicate human reasoning or behavior, need to model both information and meta-information • Most approaches have focused on representing the information, but little discussion of the meta-information

  3. What is Meta-Information? • Definitions • Data is output from a system that may or may not be useful to decision-making (radar reports storm is coming) • Information is recognized inputs that are useable to decision-making (storm is coming that may affect UAVs) • Meta-data is qualifiers of data that may or may not be useful to decision-making (radar can locate aircraft with error of +/-1.5m) • Meta-information is qualifiers of information that affect decision-making, reasoning, or behavior • Information processing • Situation awareness • Decision-making • Definitions serve to explicitly identify the critical role of meta-information in human decision-making

  4. Human Behavioral Models • Attempt to replicate human cognitive processes • Attempt to model human behaviors must capture the impact of meta-information • HBM have wide variety of applications • Developing and testing theories of human cognition • Representing realistic human behavior in training • Expert and Decision Support Systems • Modelers typically do not address meta-information because of challenges acquiring, aggregating and integrating • Focus of this research is on modeling meta-information in Bayesian Belief Networks (BBNs)

  5. Uncertainty and Human Decision-Making • Human decision-making under uncertainty deviates from logical decision-making and largely based on experience-based heuristic methods • Often the heuristics represent how experts reason about the meta-information • Uncertainty of information is one type of meta-information • Different methods of classifying uncertainty: • Executional uncertainty • Goal uncertainty • Environment uncertainty • Lack of information, etc • While these classifications of uncertainty and an understanding of their impacts on decision-making have been useful, they may not generalize to other types of meta-information not based on uncertainty (recency, reliability, trust)

  6. Computational Approaches to Uncertainty • Probability Measures • Dempster–Shafer belief functions • Extensions to first-order logic (e.g., defeasible reasoning, argumentation) • Ranking functions • ‘plausibility” measures • Fuzzy set theory • Causal network methods (e.g., Bayesian belief networks, similarity networks, influence diagrams)

  7. Types and Sources of Meta-Information • Identified the main types of meta-information that impact the decision-making process • Research from over 30 domain experts, and over 500h of interviews, observations and evaluations • From this developed a list of sources and types of meta-information that was consistently encountered across application domains • Believe this approach developed an understanding of expert reasoning and behavior sufficient to understand the impact of meta-information at a level that supports modeling

  8. Types and Sources of Meta-Information

  9. Modeling Human Reasoning and Behavior • Computational Representation of human reasoning and behavior • Model based on recognition-primed decision-making • Experts do not do significant amounts of reasoning and problem solving, but rather have been trained to recognize critical elements of a situation and act accordingly • Domain independent, modeling situation awareness-centered decision-making in high-stress, time-critical environments • SAMPLE is a general use HBM • Defined modules: Information Processing, Situation Assessment, Decision Making • Inputs processed by information processing module • Processed data (detected events) passed to situation assessment module • Assessed situation is passed to decision-making module • Rules, or lookup table of actions after situational assessment performed

  10. SAMPLE Model

  11. Bayesian Modeling about and with Meta-Information • Difficult aspect of modeling human cognition and behavioral processes is the need to reflect the known impacts of meta-information on those processes • Identified five features of reasoning that need representation within human behavior models: • Should succeed or fail to recognize relevant meta-information based on attentional and cognitive demands • Should support the representation of successful or unsuccessful human strategies to process information according to meta-information • Should represent the aggregation of meta-information • Should capture how effectively meta-information is understood relative to any prior understanding or knowledge • Should succeed and fail at incorporating meta-information-mediated situation assessments into behavior or decisions

  12. Methods for Representing Human Reasoning • Bayesian belief networks • Fuzzy set theory • Rule-based production systems • Case-based reasoning • BBNs address multiple types of modeling requirements • Two types of meta-information reasoning • Deductive reasoning • Abductive reasoning • BBNs support both types of reasoning

  13. Two Types of Reasoning

  14. Modeling the Recognition and Aggregation of Meta-Information • In many cases, human decision-makers will have to compute meta-information from multiple factors • Data and meta-data can map to meta-information in the following ways: • One-to-one mappings • Many-to-one mappings • One-to-many mappings • Many-to-many mappings • Once meta-information is calculated, it can influence the information gathering, situation assessment, and decision-making process

  15. Applying BBNs to Model Congnitive Computation of Meta Information

  16. Sensor Type as Node in Network: Sensor Type 3

  17. Sensor Type as Node in Network: Sensor Type 1

  18. Aggregating Meta-Information to Compute Overall Confidence

  19. Modeling the Recognition and Aggregation of Meta-Information • Knowing the best means to aggregate meta-information is challenging • Observation and study of human decision-making amongst subject matter experts may provide some justification, but will often unavoidably result in inclusion of biases • Using engineering data about sources may not adequately represent how a human would reason about meta-information, resulting in less reflective human behavior models

  20. Modeling the Impact of Meta-Information on Situation Assessment • Three Approaches • Simply filter or prioritize information based on meta-information • Include meta-information within BBN models of information gathering, situation assessment, and decision-making processes • Use the meta-information in a specific parameter

  21. Incorporating Meta-Information Explicitly into a BBN: No Confidence

  22. Incorporating Meta-Information Explicitly into a BBN: Low Confidence

  23. Examples of Computing the Probability of a Discrete Value for a BBN Node

  24. Conclusion • We described the application of meta-information and BBNs in modeling each of the following types of cognitive tasks: • Recognition of relevant meta-information based on aggregation of available data, meta-data, information, and meta-information into types of meta-information. • Filtering and prioritization of information based on meta-information. • Aggregation of different types of meta-information to acquire their combined impact. • Understanding of the impact of meta-information on existing knowledge • Incorporation of meta-information into mediation of situation assessment and decision-making.

  25. Questions?

More Related