1 / 27

Example-based models of control problems

Explore the effectiveness of example-based models in solving complex problems using Broadbent's TRANSPORTATION scenario and Berry's research. Two different models are tested, evaluating performance based on various criteria. The study suggests blending is a powerful mechanism for cognitive tasks.

cammie
Download Presentation

Example-based models of control problems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Example-based models ofcontrol problems Dieter Wallach & Christian Lebiere University of Basel Basel Carnegie Mellon University Pittsburgh

  2. Overview Research question Is ACT-R’s new blending mechanism really a „wonderfull mechanism“ (Gonzales, 1999)? Domain Complex Problem Solving: Creating a model for 2 new experiments with Broadbent’s TRANSPORTATION scenario Procedure Competitive argumentation: Two very simple example-based models, different validation criteria

  3. (+) t L f S L = 220t+80f S = 4.5f - 2t (+) (-) (+) TRANSPORTATION “We ask each subject to imagine that they are controlling the transport in a city. There are basically only two things they can do; they alter the time interval between the buses [t]entering the city, and they alter the amount charged for the use of the city’s car parks [f]. By altering these quantities, they can affect the load on the buses [L], which is usually expressed in the number of people carried per 100 buses. They can also affect the number of empty spaces [S]remaining in the car parks”.

  4. Interface

  5. (+) (+) (+) (-) Implicit knowledge ( ???? ) If, however, the equations are such that there is a unique pair of input values for each output pair subjects must take the crossed relationships, as well as the direct ones, into account when controlling the system. This is the feature of the task that ensures that successful performance cannot be based on the salient relationships alone. Berry & Broadbent (1987) Scores on the crossed relationship questions actually detoriated, even though performance required subjects to take account these relationships Berry (1993) t L f S

  6. Modeling approach Example-based learning “Instance-based learning meets blending” Objective A “pure” test of the power & cognitive adequacy of blending: • Very simple model(s) • NO kind of explicit causal knowledge encoded/acquired • NO parameter tweaking (a priori setting of the models’ single parameter)

  7. Model 1 (p fee-retrieval =goal> isa spaces-fee spaces =spaces =fact> isa spaces-fee spaces =spaces fee =fee ==> =goal> fee =fee ) (p interval-retrieval =goal> isa load-interval load =load =fact> isa load-interval load =load interval =interval ==> =goal> load =interval ) Difference-based representation

  8. Model 2 (p retrieve =goal> isa transportation spaces =spaces load =load =fact> isa transportation spaces =spaces load =load fee =fee interval =interval ==> =goal> load =load fee =fee ) Difference-based representation

  9. Pre-Task Questionaire

  10. Post-Task Questionaire

  11. Classes of models • trained with a generic set of training instances (input-output pairs) for each model run • no use of actual subject data Prototypical • trained with input-output-pairs that an assigned subject produced in the experiment when working on the first 2 of 8 problems that were to be solved in the experiment Individualized • trained like an individualized model • fitting is done individually Individual

  12. Model1 same for all runs 10 training instances 6 of Broadbent‘s problems Model2 P3 P4 P5 P6 P7 P8 prediction Prototypical Model Training Model Performance

  13. Load Problem space spaces

  14. First 2 of Broadbent‘s problems Model1 Individualized training data 6 of Broadbent‘s problems Model2 P1 P2 P3 P4 P5 P6 P7 P8 prediction training Individualized Model Training Model Performance

  15. Training Model Performance First 2 of Broadbent‘s problems Model1 Individualized training data 6 of Broadbent‘s problems Model2 P1 P2 P3 P4 P5 P6 P7 P8 prediction training Individual Model Same as individualized model, but individual fitting of noise parameter & individual decision tracing

  16. Evaluation of models … (… beyond sufficiency) • Similarity of the final performances • Correspondence of (number of) intermediate steps toward problem solution • Learning correspondence: Comparability in rate of improvement with practice • Error correspondence: number & kinds of errors • Correspondence of context dependency (like sensitivity to degradation by interfering contextual factors) • Temporal correspondence: Empirical and model latencies fall into a comparable range

  17. Overview: Experiment1 & Models Product correspondence

  18. Model1 (generic training) Intermediate steps Trials

  19. Model1 (indiv. training) Trials Intermediate steps

  20. Model2 (indiv. Training) Intermediate steps Trials

  21. Overview: Experiment2 & Models Product correspondence

  22. Model2 (generic training) Intermediate steps Trials

  23. Model2 (indiv. training) Intermediate steps Trials

  24. Model2 (indiv. training) Intermediate steps Trials

  25. Errors (Model1, indiv.) Errors

  26. Errors (Model1, indiv.) Errors Trials

  27. Conclusions Christian Lebiere’s criteria for when a mechanism should be part of the architecture (revisited): • Robost & general • Apply to a wide variety of tasks • Supported by empirical data … and … YES, Cleo — blending is a wonderfull mechanism!

More Related