1 / 33

The Big Idea

The Big Idea. More studies are better than less studies. Let me ask a question . . Northwest CLD. Dave told us that some research studies are more powerful than others. I was wondering if there is any procedure in education that examines collections of studies?.

elina
Download Presentation

The Big Idea

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Big Idea • More studies are better than less studies. Let me ask a question. meta.ppt

  2. Northwest CLD Dave told us that some research studies are more powerful than others. I was wondering if there is any procedure in education that examines collections of studies? meta.ppt

  3. I am glad you asked that question. There is a procedure for ‘aggregating*’ data from a variety of studies. It is called meta-analysis. Meta-Analysis *Aggregate: To gather into a mass or whole. meta.ppt

  4. In meta-analysis, a researcher examines many studies on a particular strategy and derives a numerical indicator of the relative effectiveness of the strategy, averaged across all studies. This indicator is called an effect size (ES). Such an analysis of research not only provides a numerical indicator of the relative effect of a particular intervention, but the effect size also allows comparison with other approaches used in special education or related services. 2 Thanks NWCLD presenters. We ARE getting a lot of great evidence based practices from a research-summary procedure called meta-analysis. Check out what Steve Forness and his buddies wrote in Teaching Exceptional Children… 1 I heard something about this ‘ES’ in another special education class. Could you review how this works. 3 meta.ppt

  5. I’d love to. Let’s begin with what you know about the ‘normal bell-shaped curve since it is at the heart of summarizing studies. First, what does the curve look like? meta.ppt

  6. I hope I get the right answer: “It looks like a bell.” meta.ppt

  7. Let’s talk about a normal distribution – the bell-shaped curve. The peak of the distribution, which occurs at the ‘mean’ (the average of all the numbers).. meta.ppt Cite examples. Math performance, IQ, shoe size.

  8. Each side of the curve has inflection points where the curve makes shifts in direction. meta.ppt

  9. 1 SD The first inflection point to the right of the mean occurs one ‘standard deviation’ (SD) above the mean. Mean Mean + 1 SD Above Below meta.ppt

  10. Mean Mean + 1 SD + 2 SD The second inflection point to the right of the mean occurs two standard deviations above the mean. Mean Above Below meta.ppt

  11. Mean Mean + 1 SD + 2 SD Inflection points below the mean also occur at one and two standard deviations below the mean. Mean Mean Mean - 1 SD - 2 SD Above Below meta.ppt

  12. Mean -1 SD -2 SD +1 SD +2 SD Because all points are in reference to the mean, we will indicate the differences with the mean as plus or minus standard deviations.. We say, “-2 Standard Deviations” We say, “+1 Standard Deviation” Above Below meta.ppt

  13. Mean +1 SD +2 SD -1 SD -2 SD One of the most useful aspects of the normal curve is that there are specific areas under each part of the curve. meta.ppt

  14. 50% 50% Mean +1 SD +2 SD -1 SD -2 SD As we can see, 50% of the data (Math scores, IQs, shoe sizes) fall on each side of the mean. meta.ppt

  15. 50% 50% 34% 34% Mean +1 SD +2 SD -1 SD -2 SD Of this 50%, 34% falls between the mean and one standard deviation above and one standard deviation below the mean. Above Below meta.ppt

  16. 50% 50% 34% 34% 14% 14% Mean +1 SD +2 SD -1 SD -2 SD The area between one and two standard deviations from the mean holds 14% of the distribution. Above Below meta.ppt

  17. 50% 50% 34% 34% 14% 14% Mean +1 SD +2 SD -1 SD -2 SD Since the total area on each side must total 50%, we know there is 2% of the distribution beyond two standard deviations (SD) in each direction. 2% 2% Above Below meta.ppt

  18. 50% 50% 34% 34% 14% 14% Mean +1 SD +2 SD -1 SD -2 SD Generally, the ‘average range’ is considered between -1 and +1 standard deviations (SD). Average Range 2% 2% meta.ppt

  19. Here I am back at Diamondhead. Now that you can see characteristics of the bell-shaped curve, please take out the copy you received earlier. How about now. Let’s mix a break with a job. Tell your neighbor what percentile ranks begin and end the ‘average range.’ Discovery learning. Thanks for the handout, Dave. How soon can we break? meta.ppt

  20. The Curve meta.ppt

  21. Note: Other Bell-shaped Curves meta.ppt

  22. Let’s go to the races. meta.ppt

  23. Back to Meta-Analysis • Recall the description of meta-analysis. • In meta-analysis, a researcher examines many studies on a particular strategy and derives a numerical indicator of the relative effectiveness of the strategy, averaged across all studies. This indicator is called an effect size (ES). Such an analysis of research not only provides a numerical indicator of the relative effect of a particular intervention, but the effect size (ES) also allows comparison with other approaches used in special education or related services. • Suppose we had an intervention like giving vitamins (A) that was being compared to another intervention (B) like giving candy. • Suppose that the groups who received vitamins (A) and the groups who received (B) “candy” performed equally at the start on a math test. A B meta.ppt

  24. Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. Math Scores A The two collections of groups are equal. Math Scores B Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  25. Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. Math Scores A After one month, we see this pattern. Math Scores B Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  26. Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. Math Scores A After two months, we see this pattern. Math Scores B Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  27. Further, suppose that we monitored the progress of both groups over time, and the treatments were provided with fidelity. Math Scores A After three months, we see this pattern. Math Scores B Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  28. If this was the result, which method would you choose to use in your classroom, A or C? Math Scores A Math Scores B Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  29. A vertical line can be drawn to compare the relative positions of the means of the two treatments. Note that method A’s mean is actually 1 SD ahead of the B. Math Scores A B Math Scores Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  30. Effect size (ES) is based on aggregated data A Math Scores When several studies create the pattern we see here, we are more confident that the results are trustworthy (valid). B or C Math Scores Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  31. Effect size (ES) is based on aggregated data A Math Scores Since the performance of group A is +1 SD beyond group B, we say that method A (giving vitamins) has an effect size of +1. B or C Math Scores Start + 1 Month + 2 Month + 3 Month Time meta.ppt

  32. Is it always an even number like 1, 2, 3? The answer is ‘no.’ Effect sizes can be fractions (e.g., ES = +.5 (half an SD) or ES = +.35). It could also be negative (ES = -.6). Generally ESs that are worth paying attention to as a teacher are +.40 or higher. The highest EF that I know is 1.62 for using mnemonics as an instructional tool. Any questions from the class? meta.ppt

  33. Summary • Meta-analysis aggregates data from several studies that deal with a particular treatment. • The number that describes how well the treatment works is called effect size (ES). • ESs can be positive (+) or negative (-). • Generally the bigger the ES for the treatment, the more you can take it to the bank. • Distribute Forness’ article on meta-analysis • Display some sources of information that are based on meta-analysis. meta.ppt

More Related