1 / 17

Using Statistics Effectively in Statistics Education Research

Using Statistics Effectively in Statistics Education Research. Sterling C. Hilton Brigham Young University. OUTLINE. Background on SMER workshops Summary of SMER final report Connections to statistics education research Discussion. BACKGROUND.

nievesm
Download Presentation

Using Statistics Effectively in Statistics Education Research

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using Statistics Effectively in Statistics Education Research Sterling C. Hilton Brigham Young University

  2. OUTLINE • Background on SMER workshops • Summary of SMER final report • Connections to statistics education research • Discussion

  3. BACKGROUND • Richard Scheaffer was PI on ASA sponsored study supported by NSF grant • Three workshops • Jan. 2005 • Sept. 2005 • March 2006 • 11 statisticians and 9 math education researchers

  4. PURPOSE/MOTIVATION • No Child Left Behind Act • “scientifically based research” • “…involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge…” • Call to improve mathematics education in K-12 grades (e.g. TIMMS 3) • Widespread use of qualitative research methods in mathematics education research

  5. REPORT SUMMARY • Reporting guidelines for research program not research study • Goal to help build a useful and informative body of cumulative research • Help establish consistency in • Interventions • Measurement tools • Data collection • Data analysis • Reporting

  6. REPORT SUMMARY • Influential documents • Scientific Research in Education (NRC, 2002) • Advancing Scientific Research in Education (NRC, 2005) • Mathematical Proficiency for All Students (RAND, 2003) • Influential perspective • Medical research model (clinical trials)

  7. REPORT SUMMARY • Research program components (with characteristic activities and reporting guidelines) • Generate • Frame • Goals and Constructs • Measurement • Logistics and Feasibility • Examine • Generalize • Extend

  8. REPORT SUMMARY • Generate ideas • Develop a conceptual framework • Consider own values, beliefs, biases • Characteristic activities • Identify interesting issues/questions • Search research literature • Build case for importance of issue • No reporting guidelines given

  9. REPORT SUMMARY • Framing: Goals and Constructs • Characteristic activities • Formulate central ideas and underlying constructs • Formulate a research question • Identify relevant measures or need for new measures • Select and define research methods • Reporting guidelines • Define variables and measures used • State conjectures rather than causal statements • Exploratory and descriptive statistics

  10. REPORT SUMMARY • Framing: Measurement • Validity, reliability, fairness • Characteristic activities • Examine previously used measures • Develop new measures • Test new measures • Reporting guidelines • Describe key details of development process • State validity, reliability and bias of measures

  11. REPORT SUMMARY • Framing: Logistics and Feasibility • Characteristic activities • Formulate study design • Develop data collection protocols • Conduct pilot study • Reporting guidelines • Describe study design • Describe variables of interest • Describe the pilot tests of instruments / interventions

  12. REPORT SUMMARY • Examine • If warranted, expansion of the pilot study to a larger community (single institution) • Main goal is to establish efficacy • Characteristic activities • Explore and define setting • Refine measures and protocols • Identify extraneous sources of variability • Reporting guidelines • Provide enough information to allow replication • Give estimates of parameters • Report formal statistical inference

  13. REPORT SUMMARY • Generalize • If warranted, expansion of the study to larger community (multiple institutions) • Generally requires interdisciplinary work • Main goal is to generalize findings to larger population • Characteristic activities • Assess portability of measures and protocols • Design a multi-institutional study • Conduct rigorous statistical analysis • Reporting guidelines • Describe implementation • Describe efforts to minimize bias • Specify a statistical model

  14. REPORT SUMMARY • Extend • If warranted, extend the study over time and to multiple communities • Generally requires interdisciplinary work • Main goal is to understand long-term effects • Characteristic activities • Design and conduct longitudinal study • Document the need for program improvements • Reporting guidelines • Describe implementation of longitudinal study • Describe dropout rate and its handling in the analysis

  15. REPORT SUMMARY • Additional issues of interest • Stable unit treatment value assumption: the treatment is not dependent on the person administering it • The use of pre-post scores or gain scores is potentially problematic • Measurement development takes time and effort

  16. REPORT SUMMARY • Connection to medical clinical trial model Generate Pre-clinical Frame Pre-clinical, Phase I Examine Phase I, Phase II Generalize Phase III Extend Phase IV

  17. CONNECTIONS • Different research methods answer distinct research questions • Statistics education research is young (primarily in pre-clinical, phase I stages) • Researchers in statistics education need to: • talk the language of statisticians • allow the discipline to develop • not overstate their findings • have patience • persevere in doing quality research

More Related