AScILS A ssessing Sc ience I nquiry and L eadership S kills

1. AScILSAssessing Science Inquiry and Leadership Skills Research Questions: How do activities implemented by support programs (especially research and mentoring) influence: - skills in science inquiry and scientific team leadership and membership - beliefs in efficacy and collective efficacy regarding these skills - stage-appropriate education and career outcomes? Are these influences similar for minority and non-minority students?

2. “Inquiry” in the context of AScILS Lisa Hunter Associate Director, Education & Human Resources, Center for Adaptive Optics AScILS Research Team Member Inquiry Strand Advisor/Mentor Focus Group, 9/13 & 9/15/05

3. What do we mean by “inquiry” The processes or methods that scientists use to gain an understanding of the world, and to create a body of knowledge (“research”)

4. Current AScILS Inquiry Strand Focus: Which science inquiry skill(s) or knowledge should be assessed? Education & Assessment Lit. CfAO Survey AScILS Alumni Survey Which “Science Inquiry Skills?” AScILS qualitative Studies (interviews) Intern Advisor/Mentor Meeting (9/13 & 9/15) Team Members’ Prior Experience

5. Elements of scientific inquiry • Skills or processes (what scientists do) • Knowledge about scientific inquiry (what scientists know)

6. Scientific Inquiry Processes 1. Chinn, Clark, et. al. (2002). Epistemilogically Authentic Inquiry in Schools. Science Education, 86:2. 2. National Research Council (2005). America’s Lab Report: Investigations in High School Science. Washington, DC: The National Academies Press.

7. Knowledge about scientific inquiry or the nature of science Lederman, N.G. et. al. (2002). Journal of Research in Science Teaching, 39(6), 497-521.

8. Why we are interested in “inquiry” • Many (10 of 14) of the AScILS study programs include research experiences, and is one of the 3 criteria for inclusion as a study program • Students engage in authentic researchexpect gains in inquiry (or research) skills and understandings about inquiry • NOTE: Student researchers clearly gain specific technical skills, but this is outside the AScILS focus

9. How can we assess interns’ gains in inquiry skills and/or understandings about inquiry?

10. If we could assess inquiry skills, there are many interesting questions • Do students gain inquiry skills and understandings about inquiry more in a research experience than in classes? • Do gains depend on the type of research experience? • Is there a connection between gains in inquiry skills and students’ confidence (or self-efficacy)? Students’ identity as a scientist? Persistence in science?

11. What is known? • Knowledge base on assessing inquiry skills gained from authentic research is very limited: • Papers reporting self-reported gains from interviews and surveys • “Thinking and working like a scientist” is a major gain/benefit from research experiences* • Papers reporting on how research experiences impacted students understandings about the nature of science • Significant amount of work on assessing inquiry skills at K-12 classroom level *Seymour, Elaine, et. al. (2004). Establishing the Benefits of Resarch Experiences for Undergraduates: First Findings from a Three-Year Study. Science Education, 88: 493-534.

12. AScILS Inquiry Strand • Determine the most fruitful inquiry skill to assess • Design a performance assessment (or simulation) to assess those skills

13. Current AScILS Inquiry Strand Focus: Which science inquiry skill(s) or knowledge should be assessed? Education & Assessment Lit. CfAO Survey AScILS Alumni Survey Which “Science Inquiry Skills?” AScILS qualitative Studies (interviews) Intern Advisor/Mentor Meeting (9/13 & 9/15) Team Members’ Prior Experience

14. Scientific Inquiry Processes

15. Scientific Inquiry Processes

16. As a result of doing research with your group, which skill(s) are most important for your students to develop?

17. ASSESSMENTBASICS AScILSStyle Jerome Shaw Assistant Professor Education Department

18. Source: National Research Council. (2001). Knowing what students know: The science and design of educational assessment. Committee on the Foundations of Assessment. Pelligrino, J., Chudowsky, N., and Glaser, R., editors. Board on Testing and Assessment, Center for Education. Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.

19. ASSESSMENT MANTRA • STANDARDS – what? • TASKS – how? • RUBRICS – how well?

20. Scientific Inquiry Processes

21. Explanation Rubric: Sample Criteria • Thoroughness and Clarity of Explanations • Use of Data • Ruling Out Alternative Explanations • Documenting the Limitations of Your Explanations • Source: • Sandoval, W. A., & Reiser, B. J. (2004). Explanation-driven inquiry:Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88, 345-372.

22. Scoring Guide: Using Evidence Source: Science Education for Public Understanding Program (1995) as presented in… National Research Council. (2001). Classroom assessment and the National Science Education Standards. Committee on Classroom Assessment and the National Science Education Standards. J. Myron Atkin, Paul Black, and Janet Coffey (Eds.). Center for Education, Division of Behavioral and Social Sciences Education. Washington, DC: National Academy Press.

23. Current AScILS Inquiry Strand Focus: Which science inquiry skill(s) or knowledge should be assessed? Education & Assessment Lit. CfAO Survey AScILS Alumni Survey Which “Science Inquiry Skills?” AScILS qualitative Studies (interviews) Intern Advisor/Mentor Meeting (9/13 & 9/15) Team Members’ Prior Experience