1 / 33

Research on Authentic Assessment Using a Virtual World for Learning Geology

Research on Authentic Assessment Using a Virtual World for Learning Geology. Bernhardt Saini-Eidukat 1 , Donald Schwert 1 , Brian Slator 2 , Lisa Daniels 3 , and Jeffrey Terpstra 4 1 Dept. of Geosciences; 2 Dept. of Computer Science; 3 School of Education; 4 Department of Statistics

colton
Download Presentation

Research on Authentic Assessment Using a Virtual World for Learning Geology

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Research on Authentic Assessment Using a Virtual World for Learning Geology Bernhardt Saini-Eidukat1, Donald Schwert1, Brian Slator2, Lisa Daniels3, and Jeffrey Terpstra4 1Dept. of Geosciences; 2Dept. of Computer Science; 3School of Education; 4Department of Statistics North Dakota State University, Fargo, ND

  2. NDSU WWWIC World Wide Web Instructional Committee Paul Juell Donald Schwert Philip McClean Brian Slator Bernhardt Saini-Eidukat Alan White Jeff Clark Lisa Daniels Jeff Terpstra WWWIC faculty supported by large teams of undergraduate and graduate students WWWIC’s virtual worlds research supported by NSF grants DUE-9752548, EAR-9809761, DUE-9981094, ITR-0086142 and EPSCoR 99-77788, and Department of Education FIPSE P116B011528

  3. The Geology Explorer Project Educational Game designed to provide authentic learn-by-doing experience •Exploration of a spatially oriented virtual world •Practical, field oriented, expedition planning and decision making •Scientific problem solving (i.e., a “hands on” approach to the scientific method)

  4. Balancing Pedagogy with Play Games have the capacity to engage! • Powerful mechanisms for instruction • Illustrate real-world content and structure • Promote strategic maturity (“learning not the law, but learning to think like a lawyer”)

  5. Technical Approach • Networked, internet based, client-server simulation • UNIX-based MOO (Multi-User Dungeon, Object Oriented) • Java-based clients (text and graphical versions)

  6. The Setting •Planet Oit - similar to Earth, but opposite the Sun •Students “land” on Oit to undertake exploration •Authentic Geoscience goals - e.g., to locate, identify, and report valuable minerals

  7. The Simulation ~50 places: desert, cutbank, cave, etc. ~100 different rocks and minerals ~15 field instruments: rock pick, acid bottle, magnet, etc. ~Software Tutors: agents for equipment, exploration, and deduction

  8. Graphical Navigation Database Map of Planet Oit

  9. Carrying out Investigations: Players Identify Outcrops Using Tests Detail Images Results of Tests (in this case, acid reactivity)

  10. Carrying out Investigations: Players Identify Outcrops Using Tests

  11. Coarse-grained rocks Minerals Fine-grained rocks Other objects If the property acid_reactive is 2, the mineral fizzes actively when acid is poured on it. If the property acid_reactive is 1, and a powder is made of the mineral, the powder will fizz actively. If the acid_reactive property is 0, the mineral does not fizz. If the rock contains more than 10% calcite, it fizzes actively when acid is poured on it. If the rock contains between 0 and 10 percent calcite, its powderized form will fizz. Also, if the rock contains more than 25% dolomite, its powder will fizz actively. Finally, if the rock contains no calcite or dolomite, it will not fizz. Because each mineral in the rock can be distinguished, the test can be performed on each mineral. Each returns whether it is acid reactive or not. Most other objects in the game have a acid_reactive property of 0. Pouring acid on them will cause them not to fizz. Players are an exception to this rule. You are unable to pour acid on other players. Example of Programming Actions: Acid Bottle: "pour acid bottle on substance"

  12. Intelligent Software Tutoring Agents Students/players interact with the intelligent tutoring agents Players can ignore advice and carry on at their own risk • Deductive Tutors • Equipment tutor • Exploration tutor • Science tutor • Detects when a student makes a wrong guess and why (i.e. what evidence they are lacking); or when a student makes a correct guess with insufficient evidence (i.e. a lucky guess)

  13. LS Marble Dike LS Marble SS Qzite SS Qzite SS Qtzite Qzite SS Schist Schist Schist Gneiss Virtual Mapping Schematic map - not to scale • Locate yourself and outcrops using topo maps and aerial photos • General rock identification • Create a Geologic Map

  14. Virtual Mapping Tutorials on Concepts of Geologic Mapping are Available in the Environment A Player (appearance can be changed)

  15. Virtual Mapping Base Maps for Adding Data and Creating a Geologic Map “aerial photo”

  16. Virtual Mapping Base Maps for Adding Data and Creating a Geologic Map Topographic map

  17. Virtual Mapping Outcrop Locations are Shown with Markers You Are Here Rock Type Markers Show Locations of Identified Outcrops

  18. Virtual Mapping Player Creates a Geologic Map Based on Outcrop Locations, and Can Get Immediate Feedback

  19. Virtual Mapping: Formative Assessment

  20. Computer Scoring of Your Map Virtual Mapping: Assessment

  21. Assessment Using Scenarios: Summative Rejects the notion of standardized multiple choice tests Pre-game narrative-based survey • short problem-solving stories • students record their impressions and questions Similar post-game survey with different but analogous scenarios Surveys analyzed for improvement in problem-solving

  22. Assessment Using Scenarios You are in northern Manitoba on a fishing trip. Fishing has been good. At dawn on the day of your departure you cut across country to a remote bay and have more good luck: you catch your limit of walleye!

  23. Assessment Using Scenarios On the way back to the lodge, you stumble across a heavy, moss-covered rock on the shoreline, flipping it over. Looking down, you see the underside shines with a brilliant metallic yellow. You pick up the rock and lug it back to the lodge with you.

  24. Assessment Using Scenarios When it’s time to leave, the pilot notes that you are 45 pounds overweight...exactly the weight of the rock you found. She says, "At $8 a pound, eh, you owe $360." • As a geologist, what do you do? • List the things you would consider in your decision. • List the questions you would ask yourself, and give the reasons behind those questions.

  25. Assessment Method General Problem Solving (60 pts) 1.1 Form a Hypothesis: 20 pts 1.2 Suitable Test: 20 pts Decide to Test Experimental Design, Comparison, Etc. First Specific Test Second Specific Test 1.3 Conclusions based on evidence: 20 pts Stated and correct with correct explanation Stated and correct Stated and correct with incorrect explanation Implied and correct Stated and incorrect Just implied Implied and incorrect ?

  26. Assessment Method Specific Information (24 pts) 2.1 Color/Luster: 6 pts Mention testing for Describe how Specific properties 2.2 Malleability/Hardness: 6 pts (ditto) 2.3 Specific Gravity: 6 pts (ditto) 2.4 Other Test : 6 pts (ditto)

  27. Assessment Method Expert Knowledge (16) 3.1 Current price of gold: 5 pts High knowledge Some knowledge No knowledge 3.2 Specific Geologic Conditions : 5 pts (ditto) 3.3 Judges prerogative: 6 pts ??

  28. Validation: Multiple Graders Assessment Protocol Pearson Product Moment Correlation Coefficent between graders Grader 1 Grader 2 Sample Size Coefficient jennifert arico 263 0.747 kima arico 263 0.849 kima jennifert 263 0.734

  29. Current Assessment Research: Scenario Order; Team vs. Solo play Students Scen 9/10 Scen 10/9 Total Teams of 2 19 20 39 Solo 162 155 317 Total 181 175 356 Preliminary Data from 2004 – scoring is ongoing! Number of Observations Used Here: 190

  30. Conclusions: Scenario Order; Team vs. Solo play Net Improvement for 2004 Scenario Combination Solo Team P-val Signif Scen 9/10 9.00 6.83 0.6637 NS Scen 10/9 -0.16 11.31 0.0183 S The response variable is Net Improvement based on one grader, “Kelly”. The data were analyzed as a two-factor factorial design with interaction, based on ordinary least squares methodology.

  31. Geology Explorer Development Team • Graphics • Acey Olson (Rendered Landscapes) • Rebecca Potter (Rendered Images, Icons, and Landscapes) • Dan Reetz (Rendered Images, 3D) • Shannon Tomac (Rendered Landscapes, Icons, & Images, Comic Book) • Daniel Turany (Rock/Mineral Photographs) • Staff • John Bauer; Java Client (GUMI-bare and GUMI-game) Design and Implementation • Nem Schlecht • Mark Tinguely • Brad Vender; MOO-side HTML server • Students • Graduate Research Assistants: • Kim Addicott; Assessment • Otto Borchert; Software tutors, Java client development • Bob Cosmano; Makes Java Go; Client Implementation • Krista Dauner; Tutoring Scripts • Rahul Devabhaktuni; Hydrology (Water) on Oit • Kuo-Di Jian; Java Tool Integration • Yongxin "George" Jia; Java/LambdaMOO Hierarchy Tool • Guy Hokanson; Software tutors, dynamic HTML • Bhaskar Marthi; Instrument verbs • Vidyalatha Nagareddy; Java/LambdaMOO Entity Tool • Shanhong Wu; Java3D • Undergraduate Research and Independent Study Students: • Bryan Bandli; Assessment and Evaluation • Chris Fish; Endgame Design and Implementation • Kelly Gorz; Assessment and Evaluation • Julia Karst-Grey; Geologic Content and Assessment • Lucas Koehntop; Java consulting • Ned Kruger; Assessment and Evaluation • Acey Olson; Bryce Landscapes, Tutoring, Player History, Interface Design • Carson Rittel; Content Quality Control, Help Database • Joy Turnbull; Content Quality Control, Help Database • Dean Vestal; Content Quality Control, Help Database, Interface Design • Mindy Sue Vogel; Geologic & Quality evaluation, Alternative exercises • K.C. Vorthmann; Geologic Content and Assessment • Jeff Walsh; Geologic Content and Assessment • Jane Willenbring; Assessment and Evaluation Acknowledgments: Mark Tinguely, who saved our world when its universe imploded, and Dave Schmidt for the name: Planet Oit. • Aric Olsen; Geologic Content and Assessment • Jennifer Thorstad; Geologic Content and Assessment

  32. http://oit.ndsu.edu World Wide Web Instructional Committee (WWWIC) North Dakota State University Fargo ND

More Related