eLearning for Decision-Making: Multiple Criteria Decision Analysis

1. eLearning Decision MakingeLearning sites on:Multiple Criteria Decision AnalysisDecision Making Under UncertaintyNegotiation Analysis Prof. Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki University of Technology

2. The OR-World project • Funded by the European Commission, IST Programme • Partners form industry and university • University of Paderborn (coordinator) • Helsinki University of Technology • Delft University of Technology • Lufthansa Systems Berlin • Regioworld • Dual-Zentrum

3. ORWorld • WWW based framework for sharing learning material for Operations Research • Interdisciplinary subjectMethods, applications, case studies • Well suited for hypermediaVisualization, simulation, animation • Joint effort to develop a modular study programme • To be used in universities and companies worldwide

4. SAL e-learning resources in decision making Value Tree Analysis Group Decisions and Voting Negotiation Analysis Uncertainty & Risk

6. Internet standards • Today’s standard HTML is unstructured • No clear separation between • Content, • Structure, • Representation • Reuse of the existing material problematic • Multilingual versions • No inherent possibility to add specific metadata • Need for XML (extensible markup language)

7. Split of content, structureand representation in XML Structure Contents Instructions Complex, confusing decision problems with multiple objectives have been made since the start of the civilisation. The history of the decision analysis is not that long, however. In 1730s Daniel Bernoulli (1738) first used the concept of utility when explaining the evaluation of a particular uncertain gable known as St Petersburg paradox. ... fdlgdflkgjdflgk jdlfkgj lkfjg kslkdjglkfjglfgjldfgjldfjgldjgldfjglk dflglkdjdgjlk jdlkgj reotiuoert dfgdf dfg fg fgd eteroituertfg fgdg fgryko ertuertiueryituyertret LMML LOM XSL XML Visual representation HTML PDF ljflkj sdlkfjlsd dksjflkj sdflksdjf ljflkj sdlkfjlsd dksjflkj sdflksdjf

8. Content module 2 Learning objects • Wrapping material on a higher level • Reusable components formulated in XML • Classification by meta tags HypermediaNetwork Thematic meta structure Content module 1 XSL Learning element TextPDFHTML Media Element -Text, animation,simulation, video, audio Course

9. CASE 1 1 1…* MODELLING PROBLEM SYNTHESIS 1…* 1...* Content elements Content elements 1…* 1…* ? 1…* 1…* * ANALYSIS Editing elements Editing elements STRUCTURING METHOD Partition elements 1…* 1…* Editing elements Partition elements Partition elements 1…* 1…* 1…* 1…* Editing elements Partition elements Editing elements 1…* * * Editing elements OUTPUT INPUT 1…* 1…* Partition elements Partition elements 1…* 1…* Editing elements Editing elements XML elements

10. System architecture Client Web browser HUT SAL server Software: Web-HIPRE Prime Decisions Joint Gains Opinions Online (voting version) Self Assessment & Grading Quiz Star Q&A Tool set OR-World server Learning material Evaluations: Opinions Online

11. Learning Paths Introduction to Value Tree Analysis Quizzes Videos Assignments Theory Cases Evaluation Module 2 Module 3 Value Tree Analysis Learning paths and modules Learning path: guided route through the learning material Learning module: represents 2-4 h of traditional lectures and exercises

12. Evaluation Value Tree Analysis Learning Paths Quizzes Videos Assignments Cases Theory Learning modules Introduction to Value Tree Analysis Module 2 Module 3 • motivation, detailed instructions, 2 to 4 hour sessions • Theory • HTML • pages • Case • slide shows • video clips • Web software • Web-HIPRE • video clips • Assignments • online quizzes • software tasks • report templates • Evaluation • Opinions • Online

13. Value Tree Analysis Theory introduces concepts and theory • XML documents • divided in sections • colourful graphics and animations • pages in HTML format

14. Value Tree Analysis Cases illustrates theoretical aspects, complements theory alternative learning route, learning by doing • Slide presentations • summary of theory • case specific material, problem description, methods, analysis,… • in Power Point and HTML formats • tests with XML + XSLT visualisation • Video clips • how to apply and use Web-HIPRE, Prime Decisions • an easy way to learn software use • help in practical issues

15. Value Tree Analysis Quizzes for revising, self assessment, online exams • Online Quizzes in Quiz Star • multiple choice, true/false, short answer questions • grouped in sections • correct answers or references to MCDA material • results available for the instructor

16. Value Tree Analysis Video clips • Recorded software use with voice explanations (1-4min) • Screen capturing with Camtasia • AVI format for video players • e.g. Windows Media Player, RealPlayer • GIF format for common browsers - no sound

17. Intro Quiz 1 Step 1 Assignments Quizzes Theory Cases Theoretical foundations Step 2 Quiz 2 Videos Step 3 Quiz 3 Problem structuring Quiz 4 Preference elicitation Step 4 Learning material onValue Tree Analysis • Divided in sections • Theory and cases closely linked, but independent entities

18. Value Tree Analysis

19. Value tree analysis in brief Bullet points to reduce reading time Simple animations, figures Links to case part Quizzes Videos Assignments Cases Theory Learning Paths Intro Theoretical foundations Problem structuring Preference elicitation Sensitivity analysis Behavioural issues Communicating the results Group decision making Software Value Tree Analysis Theory Systems Analysis Laboratory Helsinki University of Technology

21. Quizzes Videos Assignments Cases Theory Learning Paths Case X Value Tree Analysis Cases • Job selection case • basics of value tree analysis • how to use Web-HIPRE • Car selection case • imprecise preference statements, interval value trees • basics of Prime Decisions software • Family selecting a car • group decision-making with Web-HIPRE • weighted arithmetic mean method Theory Evaluation Assignments Intro Theoreticalfoundations Problemstructuring Preferenceelicitation

22. Value Tree Analysis Family selecting a car Cases Job selection case • group decision making with Web-HIPRE • weighted arithmetic mean method • basics of value tree analysis • how to use Web-HIPRE Car selection case Money versus design • imprecise preference statements, interval value trees • basics of PRIME Decisions

23. Assignments Learning Paths Videos Cases Theory Quizzes

25. Assignments Learning Paths Quizzes Videos Cases Theory • Report templates • detailed instructions in a word document • to be returned in printed format Value Tree Analysis Value Tree Analysis testing the knowledge on the subject, learning by doing, individual and group reports • Software use • value tree analysis and group decisions with Web-HIPRE Systems Analysis Laboratory Helsinki University of Technology eLearning / MCDA

28. Value Tree Analysis

29. Value Tree Analysis

30. Value Tree Analysis

31. Value Tree Analysis Learning material

35. Value Tree Analysis Working with Web-HIPRE

36. Learning Paths Assignments Quizzes Videos Cases Theory Videos Working with Web-HIPRE Structuring a value tree Entering consequences of ... Assessing the form of value... Direct rating SMART SMART SWING AHP Viewing the results Sensitivity analysis Group decision making PRIME method Value Tree Analysis Video clips

38. Value Tree Analysis Student evaluation • Value Tree learning module • Helsinki University of Technology • 59 students • mainly second and third year • The University of Paderborn • 95 students • mainly first and second year • Assisted and non assisted groups • Opinions-Online

43. Value Tree Analysis Summary of student evaluation • Students enjoyed the session • Only little difficulties • They would like to work in similar environments • Recommend the session to fellow students • No major gender differences Interactive results available at: http://www.orworld.hut.fi/mcdm/Learning-modules/Short-intro/evaluation-results.htm

44. Group Decisions and Voting module Evaluation Assignment Quiz Theory Learning material on Group Decisions and Voting • One learning module • All material included in the module

45. Group Decisions and Voting module Evaluation Quiz Assignment Theory Group Decisions and Voting Theory Group characteristics Brainstorming Nominal group technique Delphi technique Voting procedures Value aggregation Slide show in HTML + GIF format

46. Group Decisions and Voting Group Decisions and Voting module Evaluation Quiz Assignment Theory Quiz for revising, self assessment, online exams • Group Decisions and Voting quiz • in Quiz Star server • 10 multiple choice and 3 true/false questions • correct answers or references to MCDA material • results available for the instructor

47. Group Decisions and Voting Group Decisions and Voting module Evaluation Quiz Assignment Theory Assignment testing the knowledge on the subject, learning by doing, distributed decision making team, voting online over the Web • Software use • voting with Opinions-Online.vote • precompleted voting template • two voting rounds • Report template • detailed instructions in a word document • analysis of the voting results • to be returned in printed format

48. Group Decisions and Voting • Opinions-Online.vote: • voting • surveys • group decisions • advanced voting rules

49. Learning material onRisk and Uncertainty • Theory slides only • Used in decision making course at HUT

50. Group Decisions and Voting Theory Slideshow 1 Meanings of uncertainty Interpretations of probability Estimation of probabilities Biases in probability elicitation Calibration of experts Updating of probabilities Slideshow 2 Decision criteria On the concept of risk Risk measures Utility function Risk attitudes Stochastic dominance Decision trees Influence diagrams