Diana Bajzek, Associate Director Office of Technology for Education Carnegie Mellon University Gordon Rule Professor De

1. Assessment and LearningTwo Sides of the Same Coin. Diana Bajzek, Associate DirectorOffice of Technology for EducationCarnegie Mellon University Gordon RuleProfessorDepartment of Biological Sciences Additional authors: Judy Brooks, William Jerome, Marsha Lovett, John Rinderle, Candace Thille

2. Outline • Content Delivery & Analysis of student responses. • OLI – Online Learning Initiative • DDL – Digital Dashboard for learning • Simulator/Animation environment. • Development of authentic simulator environment • Embedding in OLI • Coupling of simulator environment to assessment. • Detached • Proximal • Associated • Tightly integrated • Reporting of student data by OLI/DDL.

3. Carnegie Mellon UniversityOffice of Technology for Learning • Digital Dashboard for Learning (DDL): Uniquefeedback tool for instructors and students. • OLI: Research project creating exemplary online courses using the best information the learning sciences have to offer. • Working in courses which are part of Open Learning Initiative -OLI • Real courses, some fully online, some blended - all publicly available. • These courses are designed by teams of learning sciences researchers, technologists, and domain experts. • OLI courses provide rich interactive learning environments with comprehensive activity logging tools.

4. The Course Design Triangle Descriptions of what students should be able to do at the end of the course Tasks that provide feedback on students’ knowledge and skills Contexts and activities that foster students’ active engagement in learning

5. The Course Design Triangle Summative Assessments

6. The Course Design Triangle

7. The Course Design Triangle

8. The Course Design Triangle Formative Assessments

9. Feedback Loops in Instruction/Assessment Instructors and Students Would Like To: • Identify difficult concepts • Detect common misconceptions • Reveal murkiest points • Assess the depth of understanding of topics Feedback Loops in Learning Assessments Active Learning Instructional Improvement

10. Modification of Instruction by incorporating feedback.

11. Modification of Instruction by incorporating feedback.

12. Design of Animations/Simulations Glucose Transport into Liver • Select concept. • Static image. • Movie clip. • Simulation • Student Interaction Outside Glucose Cell Membrane Glucose Transporter Hexose Kinase (G → G-P) Inside

13. Design of Animations/Simulations Simulator Environment: Cast are given xi, vi. Follow Newton’s laws & instructor defined rules. Glucose Transport into Liver Outside Glucose: Can’t cross the membrane, unless it encounters its transporter. Membrane: Nothing goes through, in this example. Glucose Transporter: Only moves glucose across the membrane Inside Hexokinase: Enzyme that only binds glucose. Bound glucose converted to G-P Hexose_kinase.html

14. Design of Animations/Simulations Enzyme Kinetics • Select concept. • Static image. • Movie clip. • Simulation • Student Interaction 1. Simulator Stage 3. Data Output 2. Student Input

15. Providing Context for Simulations:

16. Disconnected Assessment:

17. Proximal Assessment

18. Associated Assessments

19. Coupled Assessments • Context given within assessment. • Student prompted to interact with the simulator to gain information to answer question. • Student input passed to simulator, changing its state. • Simulator can be used to drive data output. • Question posed: • Hints available • Individual feedback for answers. • Student response & assessment activity can control simulator environment. • Control of flow is possible as the student progresses through exercise. Inhibitors_simulator.html

20. Coupled Assessments • Context given within assessment. • Student prompted to interact with the simulator to gain information to answer question. • Student input passed to simulator, changing its state. • Simulator can be used to drive data output. • Question posed: • Hints available • Individual feedback for answers. • Student response & assessment activity can control simulator environment. • Control of flow is possible as the student progresses through exercise. tca_simulator.html

21. Assessment System • Familiar question types • Sophisticated evaluation (scoring) • Flexibly evaluated according to criteria specified by the instructor • Multi-part questions

22. Feedback Reportfor the assessment asseen by an instructor • Class Participation • Best and Worst • Questions • Individual Question

23. Feedback Reportfor the assessment asseen by an instructor • Class Participation • Best and Worst • Questions • Individual Question

24. Feedback Reportfor the assessment asseen by an instructor • Class Participation • Best and Worst • Questions • Individual Question

25. OLI project: • http://www.cmu.edu/oli/ • Biology project: • http://telstar.ote.cmu.edu/biology/ • Funding: • The William and Flora Hewlett Foundation • Howard Hughes Medical Institute