1 / 14

Gianna Rendina-Gobioff Meta-Analysis: Spring 2004

A Meta-Analysis Reliability Generalization Study: Reliability of the Statistical Anxiety Rating Scale Subscale Worth of Statistics. Gianna Rendina-Gobioff Meta-Analysis: Spring 2004. Research Domain. Reliability Incorrectly reported “The test is reliable”

janna-mcconnell
Download Presentation

Gianna Rendina-Gobioff Meta-Analysis: Spring 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Meta-Analysis Reliability Generalization Study:Reliability of the Statistical Anxiety Rating Scale Subscale Worth of Statistics Gianna Rendina-Gobioff Meta-Analysis: Spring 2004

  2. Research Domain • Reliability • Incorrectly reported • “The test is reliable” • “Reliability induction” – reporting reliability estimates from test publishers or literature instead of for the sample of interest in the study • Reporting problems • Reliability dependent on sample characteristics (more heterogeneous samples will have higher correlations/estimates) • Accuracy of effect sizes reported is in question (if reliability is poor, the effect size is low)

  3. Research Domain • Statistical Anxiety Rating Scale (STARS) • Total of 51 likert 5-point response items • Statistics anxiety measure with six subscales: • Worth of statistics (16 items) • Interpretation anxiety (11 items) • Test and class anxiety (8 items) • Computational self-concept (7 items) • Fear of asking for help (4 items) • Fear of statistics teacher (5 items)

  4. Research Questions: • What is a typical score reliability for the Worth of Statistics subscale? • Do the reliability estimates for the STARS Worth of Statistics subscale vary across sample disciplines of study?

  5. Data Collection: • Search • ERIC, PsychINFO, and ISI Web of Science • Contacted researchers • Inclusion/exclusion criteria: • Timeframe: 1985 - present • Measure: STARS only • Required Information: coefficient alpha • Language: English only • Type of Publication: No restriction

  6. Data Collection • Coding: • Dissemination: Journal or conference • Discipline of sample: Multi or education only • Education: Graduate or undergraduate/graduate mix • Reliability induction: Yes or no • Cronbach alpha: Subscales, total, range, median • Inter-rater reliability • Moderator of interest, discipline, reliability estimate of 0.91 (Cohen’s Kappa)

  7. Data Analysis • Cronbach alpha transformed to z: • Weighted with inverse of Cronbach alpha variance: • Where i is the number of items and N is the sample size • Lipsey and Wilson (2001) fixed effects with moderator

  8. Results • Study search results: • Started with 39 studies • Out of 39 obtained studies only 10 included • 17 – Duplications, not applicable, descriptive, or unavailable • 7 – Reliability induction • 3 – Did not report alpha for Worth subscale • 1 – Used revised version of instrument • 1 – Sample overlap • The 10 studies yielded 11 effect sizes

  9. Evaluation of Analyses and Validity • Analysis concerns: • New weight methodology • Should have used mixed effects because random effects remained after including the moderator – Didn’t know how to run this analysis • Threats to Validity: • Apples/oranges minimized • Independence of effect sizes met • File drawer of concern • Methodological Rigor potential problem

  10. Conclusions • Internal consistency is high across various samples for the Worth of Statistics Subscale (M=0.9124) • Multi-Discipline samples produced higher estimates of the subscale reliability, compared to Education only samples • Implications: • Guidance for choosing a measure of statistics anxiety - should take discipline of sample into consideration

More Related