Data Mining and Analysis in Multi-User Educational Environments

1. Data Mining and Analysisin Multi-UserEducational Environments Chris Dede Harvard University Chris_Dede@harvard.edu www.gse.harvard.edu/~dedech/

2. What is a MUVE? • A representational container that enables multiple simultaneous participants to access virtual spaces configured for learning. • A place where learners represent themselves through graphical avatars (persona) to communicate with others’ avatars and computer-based agents, as well as to interact with digital artifacts and virtual contexts. • A learning experience that provides diverse activities in support of classroom curriculum. Rich Laboratory for Studying Learning and Teaching

3. Immersive, Situated Learning constellations of architectural, social, organizational, and material vectors thataid in learning culturally based practices • Apprenticeship (novice to expert) • legitimate peripheral participation(tacit learning, like internships) Aids with transfer and with evolution of identity

4. Figure 2: River Water Sampling River CityCurriculum Figure 1: Lab Equipment inside the University

5. Summary of Significance of Implementations To Date • MUVEES wasmotivatingfor all students,including those with lower engagementand achievement in science. • MUVEES had apositive effect on academic self-efficacy. • MUVEES had the most positive effects for studentswith high metacognitive awareness of the inquiry process. • Students identifiedmultiple, interrelated causal factorsin the MUVE. • Language wasnota barrier to success • Gender wasnota significant factor. • The MUVE with embedded guidance can support students’ growth towardsself-responsibility in learning. http://muve.gse.harvard.edu/muvees2003/

6. Student generated: Lab book Letter to the Mayor Classroom behavior Attendance records Interviews/focus groups Curriculum generated: Pre-post assessment instruments Mini-mysteries Mini-goals to achieve next ‘level’ Technology generated: Log files Database Tracking of in-world activities Evidence of Student Work

7. Logfiles as a Database Indicates with Timestamps • Where students went • With whom they communicatedand what they said • What artifacts they activated • What databases they viewed • What data they gatheredusing virtual scientific instruments • What screenshots and notations they placedin team-based virtual notebooks • What hints they accessed The “Play Therapy” Hypothesis

8. Logfile Case Study: Princess • age 12, 7th grade, D+ in science • Reading below grade level • Content scores started very low—approximately 10% • Self-efficacy:The belief that one can successfully perform certain behaviors • At the start: just above average

9. Teacher Expectation of Student Success in Learning Content of Scientific Method and Disease (n=94) Princess

10. Teacher Rating of Student Behavior in Class(n=94) 54% Princess 27% 17% 2%

11. Teacher Rating of Student Engagement/Motivation in Class(n=94) 45% Princess 34% 10% 10% 1%

12. Princess, Age 12, 7th Grade

13. Princess, age 12, 7th Grade

14. Princess, age 12, 7th Grade Despite repeated conversational gambits,Princess kept off-task conversation to a bare minimum

15. Educational Outcomes—Princess • Self-efficacy: • At project end: improvement of 17% • Biology content: • 20% improvement • Scientific process skills: • 10% improvement • Her team’s score the second highest in the classon the “letter to the mayor” performance measure Two Week Intervention

16. Visualization of Student Activities

17. Illustrative Collective Logfile Analysis:Gender Patterns in Information-Gathering • More female residents (21) than male (13);4 of 5 resident experts are female • Intentional to make environment comfortable for girls • 96 students with same teacher logged about 3000 conversational gambits in two-week implementation • Boys and girls talk to male and female residentsin equal proportions • Girls are substantially more active than boysin talking to residents • Boys’ conversations are task-oriented;girls add a social dimension • Substantial individual variationin information-gathering patterns

18. Mechanisms forAutomated Logfile Analysis Easy • How often does a given student talk to a female agent,and is the conversation social or task -oriented? • What proportion of scientific data in a team’s virtual notebook was contributed by a given team member? Less Easy • What sequence of interactions in the world led toa given student’s becoming deeply engaged? Hard • Is a given student developing an increasingly sophisticated pattern of inquiry over time?

19. Potential Insights for Students Evolution over time of: • Engagement • Information-Seeking • Sources: context, agents, artifacts, databases,virtual scientific instruments, hints… • Collaboration, includinguse of virtual notebook • Content Mastery • Inquiry strategies

20. Potential Insights for Teachers • Diagnostic, formative information aboutindividual students • Engagement • Level and types of hints accessed • Skewed information-gathering patterns • Diagnostic, formative information aboutstudents collectively • Level of collaboration • Degree to which certain types of hints are needed • Degree to which some kinds of information resourcesare underutilized • Patterns of scores on embedded content assessments