The Learning Online Network with CAPA A Cross-Institutional Open-Source

1. The LearningOnline Network with CAPA A Cross-Institutional Open-Source Learning Content Management and Assessment System Gerd Kortemeyer Michigan State University PERG Seminar University of Maryland

2. Some of the History

3. 1992 CAPA Started by Ed Kashy et al. in Cyclotron Individualized assessment system for science and math Immediate feedback, multiple tries - mastery based Used paper copies of assignments and terminal input X-Windows problem editing Got Web student interface in 95

4. 1997 LectureOnline Started by Wolfgang Bauer, Walt Benenson, Gary Westfall, and Gerd Kortemeyer in Cyclotron Learning content management and individualized assessment system for science and math Sharing of content between courses Completely web-based interface

5. 1999 LON-CAPA The LearningOnline Network with CAPA Collaboration of CAPA and LectureOnline groups “The best of both worlds” Sharing of content between courses and institutions Reusability of content on different levels of granularity Distributed and Scalable

6. LON-CAPA Today Attracted NSF Information Technology Research grant “Investigation of a Model for Online Resource Creation and Sharing in Educational Settings” (more later) Has over 20 partner institutions (more later) Useful and usable for “casual” (non-”power”) faculty user outside science and math LON-CAPA is a full-featured course management system Software platform has GNU General Public License and is free of licensing fees 2003 ComputerWorld 21st Century Achievement Award Version 1.0 was rolled out this Fall

7. Today LON-CAPA is used … With material written by faculty teaching course or “re-used” from other faculty + helproom For homework in addition to “traditional” lecture and textbook + helproom Hybrid with “traditional” lecture, homework and textbook online + helproom Completely “virtual” courses AP courses Prelab quizzes In-class exercises Class sizes: 10-3,000

8. Today LON-CAPA is used … Helproom Helproom at MSU started out: Very few computers, no seating in front of computers Round tables Big sheets of paper Whiteboard all around the walls TAs present Basement “cave” Absolutely crowded The all-new helproom since last year: Lots of desktop computers in cubicles #@$%@!$&#

9. Student Numbers at MSU

10. Subject Areas MSU Fall 2003

11. System-wide Use 24,000 students/semester system-wide Middle Schools: 300 High Schools: 500 Community Colleges: 50 Four-year Colleges: 300 Universities: >22,000.

12. Cross-Institutional

13. Distributed Server Network LON-CAPA is built as a geographically distributed network of constantly connected servers

14. Logical Domains The network is logically divided into domains such as “MSU”, “FSU” or “Publisher X” Domains limit the flow of user information Domains can limit access to content resources Domains limit the extent of user privileges

15. Authentication Any user in the network can log into any machine in the network Example: Michigan State University user logs into server at North Dakota State University

16. Authorization Any LON-CAPA user can have roles for any domain. Example: Michigan State University faculty is Course Coordinator for a course at North Dakota State University

17. Virtual Filesystem The shared content repository appears as one virtual filesystem

18. Resource Assembly Pages can be constructed from fragments and other pages Sequences can be constructed from fragments, pages, and other sequences Courses point to (top-level) sequences Maps at every level are simply other content resources

19. Resource Assembly Graphical resource assembly tool (RAT) to construct maps Working on branching based on conditions using performance data, course data, and preferences Individualized curriculum

20. Shared Resource Pool Shared resource pool currently holds approximately 6,000 original homework and exam problems 5,000 images 150 movies 180 java applets 3,000 content pages. LON-CAPA Pool available in NSF National Science Digital Library

21. Individualized Assessment

22. “Individualized Assessment” Individualized problems: different numbers, different graphs, different options, … “Classical” online homework elements: multiple choice, option response, mix-and-match, string, etc “STEM” elements: numerical, multicomponent numerical, physical units, symbolic math, individualized simulations Free combination of the above elements Adaptive immediate feedback Multiple attempts - mastery based

23. “Individualized Assessment” Example: Individualized graph, numerical answer

24. “Individualized Assessment” Example: Individualized labels, options

25. Conceptual:Functions Graphed+RandomLabels

26. “Individualized Assessment” Example: Multicomponent numerical with individualized animation and physical units

27. Individualized Assessment Text for Applet

28. Applet:Initial and Final ActionFrames

29. Student evaluations of CAPA

30. Evaluations of CAPA by students in Physics for Scientists and Engineers CAPA as Tool for Learning/Understanding

31. Fall -Traditional… Spring - Technology Question: If I had a choice between the CAPA scheme as used in PHY184 and the recitation scheme used in PHY183, I would be: A - Strongly in favor of recitations B - Moderately in favor of recitations C - Indifferent D - Moderately in favor of CAPA E - Strongly in favor of CAPA Phy184 Spring 95 N=156 Birge, Cowen

32. Fall – Technology Spring - Traditional Did the different methods affect your learning/understanding of physics? PHY184 Spring ‘96 N=104 Berz, Tomanek

33. Time-on-task with and without CAPA

34. Educational Effectiveness in Physics

35. Technology Impact Prof. Danielewicz (Physics)

36. Distribution of grades in Introductory Physics for Scientists and Engineers • 1992-1994 w/o CAPA • 1996+ with CAPA

37. Gender Differences in Physics

38. Grade distributions for male and female students for a two-semester course (PHY231 and PHY232). • PHY231 was taught the traditional way, and in PHY232 CAPA was used.

39. Item Analysis

41. Doing Item Analysis on Randomized Questions <conceptgroup concept=”Fruit”> <foil>Apple</foil> <foil>Orange</foil></conceptgroup><conceptgroup concept=”Metals”> <foil>Copper</foil> <foil>Gold</foil></conceptgroup>

42. 1. Two masses have the same acceleration if the string does not stretch.2. Weight of the two masses is greater than the tension of the string attached to the ceiling3. The top tension equals the bottom tensions. (massless pulley)4. Tension holding the two masses are equal if mass of pulley = 05. Sub-system accelerates upwards or downwards accordingly6. Center of mass accelerates downward First 18.6 % of responses

43. Predicting Success, Learners at Risk

44. Covariates included ACT, College GPA, Initial Physics knowledge (Force Concepts Inventory), and number of absences

45. Success Prediction Power

46. “Cheating”

49. 3rd Party Monitored (posting) Monitored (looking) Homework 0.041 0.118 -0.109 Quizzes -0.302 0.098 0.069 Midterm Exams -0.352 0.166 0.160 Final Exam -0.348 0.147 0.129 Influence of Websites

50. The NSF ITR Grant Project