480 likes | 551 Views
Assessing the Benefits of Online Scenario Simulation Tools in Security-Related Studies at UMUC. Irmak Renda-Tanali , DSc . Collegiate Associate Professor Director, Homeland Security & Emergency Management Graduate School of Management & Technology
E N D
Assessing the Benefits of Online Scenario Simulation Tools in Security-Related Studies at UMUC Irmak Renda-Tanali, DSc. Collegiate Associate Professor Director, Homeland Security & Emergency Management Graduate School of Management & Technology University of Maryland University College Fourteenth Annual Emergency Management Higher Education Conference Federal Emergency Management Agency, U.S. Department of Homeland Security Thursday June 9, 2011
Outline • Background • Research Objectives • Methodology • Research Design • Tools • Results • Survey Results • Actual Performance Results • Conclusions • Recommendations Irmak Renda-Tanali
Background • Why such study? • Emergency Management and related fields are best learned through scenario studies • Textbooks and lecture-based learning do not effectively teach: • Consensus gathering • Collective decision making • Decision making under uncertainty • Testing and validating assumptions • How to manage major crises and emergencies • Textbooks and lectures do not capture the dynamic, time-sensitive, context-dependent, multi-disciplinary nature of the emergency/crisis scenarios. Irmak Renda-Tanali
Background • Why such study (cont’d)? • Course evaluations and teaching experience show: • Students want dynamic, hands-on exercises that simulate real world environments, real world thinking. • Textbook reading and conference participations need further stimulating tools to enhance student understanding of topics. • Students want to be an active part of decision making process as crisis events unfold. • They want to learn what are the good AND bad decisions and why, by actively seeing consequences of their decisions in reaction to events that unfold. • Case studies looking backward are not so interesting as the events and consequences are already known. • Students learn better by hands-on practice than just by reading. Irmak Renda-Tanali
Background • Examples of current practice: • Multi-media technology • Video, audio, web pages, web 2.0 tools: blogs, wikis, social networking sites, podcasts, vodcasts etc. • Play2Train, a virtual world platform, uses Second Life software with avatars (intelligent agents) mimicking human behavior. • Specific examples related to security studies: • A web-based scenario simulation tool used for public health undergraduate students (Spinello & Fishbach, 2004) • San Luis Rey town scenario simulation exercise for HS students at UMUC graduate studies (Boubsil & Gayol, 2006) • Play2Train, used to replicate tabletop exercises, e.g. Centers for Disease Control (CDC), Emory University, University of Illinois at Chicago’s Center for the Advancement of Distance Education (CADE), Seton Hall University, and Idaho State University (Hewitt et al, 2009) Irmak Renda-Tanali
Background • Examples (cont’d) • Specific examples related to security studies (cont’d) • Second Life and Play2Train in Master of Healthcare Administration online programs, used to enhance student discovery, critical thinking, and analytical skills. Irmak Renda-Tanali
Research Objectives • To capture the effectiveness of online scenario simulation tools used at UMUC, on student learning; • To identify the kinds of strengths and weaknesses of those tools; • To compare those tools to traditional methods of teaching; • Ultimately to enhance student learning in Emergency Management and related fields of study through the findings of the study. Irmak Renda-Tanali
Study Tools • Subjects: • 5 online sections of CCJS 320 – 145 students in total • 3 online sections of CCJS 420 – 78 students • 2 online sections of CCJS 425 – 45 students • 2 online sections of BSBD 641 – 51 students total 319 students • Instructors • Developer-instructors Taught in Fall 2010 semester • Data Sources • Student survey for all sections of the above courses • Phone calls to faculty developed and/or use simulation • Course statistics measuring student participation and interaction • Gradebook • The generic end-of-semester evaluations Irmak Renda-Tanali
Areas Surveyed • Learning: • Understanding the material • Processing, testing, validating assumptions • Decision making skills • Getting close to real-life experience • Connecting events and objects (or actions) • Visualizing, conceptualizing, comprehending • Consensus gathering • Effective crisis communication • Effective use of resources • Effective emergency response to large scale crises • Student-to-student interaction • Student-to-faculty interaction Irmak Renda-Tanali
Areas Surveyed • Technology use • Workload • Student Satisfaction • Demographics Irmak Renda-Tanali
Assumptions • Learning: • Online scenario simulation tools help understand the material better. • They help better process, test, validate assumptions • Improve decision making skills. • Help get close to real-life experience. • Help connect events and objects (or actions). • Help with visualizing, conceptualizing, and comprehending. • Increased student-to-student interaction contributes to learning (Community of Inquiry Theory by Garrison et. al, 2000) • Increased student-to-faculty interaction contributes to learning (CoI). • Simulation tool increased interaction learning Irmak Renda-Tanali
Assumptions • Technology use: • If there are no problems with the technology, student satisfaction is more better learning • Workload: • No assumptions about the time spent versus contribution to learning • Student Satisfaction: • Students who responded favorably performed favorably – or did they? • Demographics: • No assumptions – a correlation study may uncover unknown facts Irmak Renda-Tanali
Setting CCJS 320 • Project Name: “Crime Scene Simulation” • Actions: Students virtually move around a crime scene, examine items, process evidence, place them in an evidence box, interview the witness, and record details in a journal. After all the evidence is collected, students enter into a virtual Crime Lab and request the appropriate tests for each evidence item such as medico-legal autopsy, fingerprint analysis, forensic biology, toxicology, firearm examination, etc. • Deliverable: After the tests are performed a detailed Crime Scene Report is generated. Students are graded based on the correctness of the evidence collected and the tests generated. The crime lab yields no results if incorrect tests are requested. Irmak Renda-Tanali
Setting CCJS 320 (cont’d) Irmak Renda-Tanali
Setting CCJS 320 (cont’d) Irmak Renda-Tanali
Setting CCJS 320 (cont’d) Irmak Renda-Tanali
Setting CCJS 320 (cont’d) Irmak Renda-Tanali
Results: Crime Scene Simulation • Response Rates (CCJS 320): Irmak Renda-Tanali
Results: Crime Scene Simulation • Understanding the material Correlation to itself Large (strong) correlation Large correlation (Pearson Coefficient) Medium correlation Low correlation Irmak Renda-Tanali
Pearson’s Correlation Coefficient Pearson's correlation coefficient between two variables is defined as the Covariance of the two variables divided by the product of their standard deviations. Simple to do using Excel spreadsheet with the embedded function. Irmak Renda-Tanali
Setting CCJS 420 • CCJS 420 Medical and Legal Investigations of Death Cold Case Simulation • Actions: Students begin by examining the reports and evidence from the original investigation. Once they are familiar with the homicide case and the people involved, they proceed to interview persons of interest. As the simulation progresses and student encounter new evidence, they will be able to submit that evidence for various types of forensic testing. • As students proceed with their investigation, they gather evidence, test results, reports, and witness statements. These items are placed into their binder as they are created or discovered. Deliverable: Arrest Warrant Report Objectives: Gather enough evidence, secure an indictment from the district attorney, and solve the case. Irmak Renda-Tanali
Setting CCJS 420 Irmak Renda-Tanali
Results: Cold Case File Simulation • Response Rates (CCJS 420): Irmak Renda-Tanali
Setting BSBD 641 Irmak Renda-Tanali
Results: Biological Attack Simulation • Response Rates (BSBD 641): Irmak Renda-Tanali
Conclusions • Online scenario simulations add value to understanding. • They help learning through discovery. • Provide near real-life, hands-on experience. • Help visualize, conceptualize and make connections between events and tools/objects. • There is a correlation between student satisfaction and grades. Further studies with control group settings can uncover this or more facts. • May or may not increase interaction among students. Further studies should look into this.
Conclusions • There is evidence of increased student-to-faculty interaction. • Technology glitches diminish value and satisfaction. (e.g. dead ends, loops) • Information glitches diminish value and satisfaction. • Age plays an important role in designing the content for the student audience. • A negative correlation between time spent and learning the material was uncovered. Was the time spent on technical problems? Further studies should look into optimal use of time. Too much time spent may bring diminishing returns.
Conclusions • The more sophisticated (game like) features make the tool more satisfactory. • Students would like a sense of achievement at the end (e.g. solving piece of a puzzle, putting pieces together) • Caution: Make them game-like but emphasize pedagogy. • If designed as a group work with role playing ability may increase communication skills, help build core competencies in crisis management such as consensus gathering, effective resource allocation, understanding response structures in large scale crises and disasters.
Conclusions • Recommend a move from low budget productions to sophisticated online simulation tools if budget permits. • Expanded studies may show a link between the dollar amount spent for development and the student success outcome (Benefit-Cost Analysis). • The tools should allow room for changing/ alternating between multiple scenarios to prevent boredom among professors and student sharing of results. • Need buy-in from instructors. Engage them in designing the tools.
Conclusions • There are successful applications with proven results about other similar simulations employed by other higher edinstutions. UMUC can emulate on that. • The new generation as “Digital natives” learn better with game-like simulations.
Acknowledgments This research was funded by the UMUC Faculty Research Grant Program. Those who contributed to the research include Husein Abdul-Hamid, Susan Blankenship, Howard Krivan, and Howard Dobres.