The Creation, Validation, and Reliability Associated with the EQUIP: A Measure of Inquiry-Based Instruction

1. The Creation, Validation, and Reliability Associated with the EQUIP: A Measure of Inquiry-Based Instruction Jeff C. Marshall Clemson University NARST 2009 Conference

2. Challenge • K-12 teachers report 39% of instructional time is spent on inquiry (Marshall, Horton, Igo, & Switzer, 2009). • Is this true? • Regardless of amount, what is the quality of the inquiry? • Current protocols were too broad, too limiting, or not valid for our purpose. • Goal: create and validate a more useful protocol

3. Inquiry Defined • NSES definition (NRC, 1996, p.23): • multifaceted activity that involves making observations; posing questions; examining books and other sources of information to see what is already known; planning investigations; reviewing what is already known in light of experimental evidence; using tools to gather, analyze, and interpret data; proposing answers, explanations and predictions; and communicating the results. Inquiry requires identification of assumptions, use of critical and logical thinking, and consideration of alternative explanations.

4. Existing Protocols • Inside the Classroom Observational Protocol(Horizon Research, 2002) • Reformed Teaching Observation Protocol (RTOP)(Sawada et al., 2000) • The Science Teacher Inquiry Rubric (STIR)(Beerer & Bodzin, 2003) • The Science Management Observation Protocol (SMOP)(Sampson, 2004) • Various teacher efficacy scales (Riggs & Enochs, 1990) have been used as a measure to predict whether reform is likely to occur. This approach is often used because self-reports of efficacy have been closely tied to outcome expectancy (Saam, Boone, & Chase, 2000).

5. Need for New Protocol In our search for a protocol, we found several instruments that all have significant value. None fully matched our needs. • Inside the Classroom Observational Protocol –too global • The Reformed Teaching Observation Protocol (RTOP) —focuses on constructivist classroom issues; uses Likert scale instead of descriptive rubric—we desired specific incremental targets to guide reformed practice; micro view not reliable, only macro is limiting   • The Science Teacher Inquiry Rubric (STIR) –aligned with NSES definition but lacks specifics necessary to facilitate each aspect of inquiry. • The Science Management Observation Protocol (SMOP)—emphasizes classroom management issues but not key components of inquiry-based instruction. • Various teacher efficacy scales—possible predictors but not instructional focused

6. Need for New Protocol In our search for a protocol, we found several instruments that all have significant value. None fully matched our needs. • Inside the Classroom Observational Protocol (Horizon Research, 2002) –too global lacks granular look at inquiry instruction • The Reformed Teaching Observation Protocol (RTOP) (Sawada et al., 2000) focuses on constructivist classroom issues, but goes beyond a look at inquiry-based instruction to more of an evaluation of teaching. Furthermore, the use of a Likert scale to assess classroom instruction was a limiting factor for our needs. We ultimately sought an instrument with a descriptive rubric that can be used to guide teachers and help them set specific incremental targets as they seek to improve their inquiry-based instruction.   • The Science Teacher Inquiry Rubric (STIR) (Beerer & Bodzin, 2003) aligned with the NSES definition but lacks insight into the specifics of inquiry that teachers must facilitate with each aspect of inquiry. • The Science Management Observation Protocol (SMOP) (Sampson, 2004) emphasizes classroom management issues and does not assess key components of inquiry-based instruction. • Various teacher efficacy scales (Riggs & Enochs, 1990)

7. Instrument Development • Protocol requirements: • measure the quantity and quality of inquiry instruction (validation focuses on quality) • work with various inquiry instructional models (e.g., 4E x 2, 5E, Learning Cycle) • appropriate for both researchers and teachers • view of instruction from micro and macro

8. Flowchart of Design

9. Instrument Validity • Face Validity • 11 Researchers (4 internal and 7 external) • Internal Consistency • Cronbach’s Alpha (.880-.889) • Inter-Rater Reliability • Cohen’s Kappa (.55-62) • R2 = .856 • Content and Construct Validity • Literature • SEM

10. Structural Equation Modeling (SEM) • Confirmatory Factor Analysis conducted • Model Trimming (26-14 indicators) • 3 Constructs • 4th Construct justified from remaining indicators • Resulting in 4 constructs • Instruction, Discourse, Assessment, Curriculum • 19 total indicators

11. Moving from 3-4 constructs

12. Summary of SEM Data All parameters within acceptable boundaries. 2 is significant p < .001, 2//df 2 indicates reasonable fit (Kline, 2005), root mean square error of approximation, RMSEA of .1 is on the threshold of reasonable fit (Browne & Cudeck, 1993), standardized root mean squared residual, SRMR < .1 is considered favorable (Kline, 2005), and the computerized fit index, CFI, of > .90 is considered a good fit (Hu & Bentler, 1999). The four-construct model, 19-indicator model, though not quite as parsimonious as a 14-indicator model, provides a good-fitting model that also is solidly supported by the literature base regarding effective inquiry instruction.

13. Discussion • EQUIP provides valid macro (construct and lesson level) and micro (indicator level) view of inquiry. • EQUIP’s descriptive rubric can be used formatively by teachers and researchers. • Valid for both math and science. • Likely effective for both pre- and in-service teacher training.

14. Further Information • Jeff C. Marshall, Clemson University • email: marsha9@clemson.edu • website: www.clemson.edu/iim (select research and evaluation tab for EQUIP; select outreach and dissemination for paper and PowerPoint)