1 / 16

Just Because They Say It’s ‘Scientifically-based’ Doesn’t Mean It Will Work!

Just Because They Say It’s ‘Scientifically-based’ Doesn’t Mean It Will Work!. Changing Landscape. P.L. 94-142 ----- IDEA ‘03 Decrease in experimental studies 1980 (61%) ---- 1995 (38%) Definition of “scientifically-based” = Random assignment/True Experiment National Reading Panel report

riva
Download Presentation

Just Because They Say It’s ‘Scientifically-based’ Doesn’t Mean It Will Work!

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Just Because They Say It’s ‘Scientifically-based’ Doesn’t Mean It Will Work!

  2. Changing Landscape • P.L. 94-142 ----- IDEA ‘03 • Decrease in experimental studies 1980 (61%) ---- 1995 (38%) • Definition of “scientifically-based” = Random assignment/True Experiment • National Reading Panel report • Institute for Educational Sciences • Configuration/role of OSEP • Status of Part D in reauthorization

  3. “….methodologically weak research, trivial studies, an infatuation with jargon, and a tendency toward fads.” National Research Council

  4. Educational Research: The Hardest Science of All!!!

  5. Standards for Field Testing Interventions (CRL) • Practical and doable • Easy for both teachers & students to learn • Yield meaningful”real world” outcomes • Broad in reach…..impacts non-SWDs • Impact performance of SWDs to enable them to compete within criterion environment

  6. Guiding Principles (CRL) • Deal with complex realities of schools • Participant input at all stages • Use sound research methodologies/designs • Collect many measures on interventions • Field-testing in multiple stages • Insist on both statistical & social significance • Translate field protocols into user manuals • Bring interventions to scale

  7. Designing High Quality Research in Special Education: Group Experimental Design Gersten, Lloyd, Baker (1999)

  8. The BIG Issue: Trade-off between Internal & External Validity “….the challenge to educational researchers is to sensibly negotiate a balance between those components that satisfy science (internal) & those that reflect the complexities of real classroom teaching (external).”

  9. Good Research is Largely Dictated by….. How well independent variables are conceptualized & operationalized, the definition of the sample, & the dependent variables are selected to assess the impact of the intervention

  10. On Independent Variables… • Precise definitions needed • Problems arise with PAR (flexibility) • Syntheses/meta need precision • Majority of literature: incomplete or very poor description of intervention • Gap between conceptualization & implementation (# min., support, tchr. Training, etc.)

  11. Improving Independent Variables • Intervention transcripts • Replications (others/component analysis) • Fidelity measures throughout implementation (amt of training, lesson length, time, feedback) • Good comparison groups (control teacher effects, feedback, time) • (see p. 13

  12. Improving Sample Selection & Description • Sample size (difficult with SWDs) • Stronger the treatment, smaller #s • Increase power by increasing homogeneity • Precise sample description (ELL, SES, achievement & cognitive levels, etc.) • Random selection (survey); random assignment (intervention )

  13. Quasi-Experimental Designs • Students used from intact groups • Determine similarity with pretests -- if variance exists, use procedures to adjust statistically • Problems when differences on pretests exceed 1/2 SD of the two groups

  14. Improving Quasi-Experiments • Adequate prestesting with measures with good technical qualities • Pretest data with more than .5 SD shouldn’t be used • ANCOVA shouldn’t be used if SD> .5

  15. Dependent Measures • Those measures used to evaluate the impact of the intervention. The conclusions of a study depend on both the quality of the intervention andthe measures used to evaluate the intervention.

  16. Improving Dependent Measures • Use multiple measures ( global and specific skill) • Select measures non-biased toward intervention (teaching to the test) • Ensure that not all measures are developed by researcher • Select measures with good technical adequacy

More Related