1 / 20

Developing a Statistics Teaching and Beliefs Survey

Developing a Statistics Teaching and Beliefs Survey. The National Statistics Teaching Practice Survey Project (NSF DUE-0808862). Jiyoon Park Audbjorg Bjornsdottir Department of Educational Psychology. Overview. Statistics Teaching Inventory (STI)

phuoc
Download Presentation

Developing a Statistics Teaching and Beliefs Survey

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Developing a Statistics Teaching and Beliefs Survey The National Statistics Teaching Practice Survey Project(NSF DUE-0808862) Jiyoon Park Audbjorg Bjornsdottir Department of Educational Psychology

  2. Overview • Statistics Teaching Inventory (STI) • PIs: Joan Garfield, Robert delMas, and Andrew Zieffler • NSF funded project to develop, pilot, and gather validity evidence • Background of Statistics Teaching Inventory (STI) • Development of Instrument • Process of STI Validation • Psychometric validation • Interview validation • Validation Results • Future plans

  3. Background of STI • Increasing calls for reform in undergraduate education in STEM disciplines • Implications for teaching and learning: • Students learn by constructing knowledge • Real-world problems provide effective ways to structure learning • Collaborative groups facilitate learning • Classroom discourse plays a critical role in learning • Well-designed technological tools can help students visualize and explore abstract concepts and processes.

  4. Background of STI - continued • Reform in statistics • GAISE report endorsed by the ASA (2005) • http://www.amstat.org/education/gaise/ • GAISE recommendations: • Emphasize statistical literacy and develop statistical thinking. • Use real data. • Stress conceptual understanding rather than mere knowledge of procedures. • Foster active learning in the classroom. • Use technology for developing conceptual understanding and analyzing data. • Integrate assessments that are aligned with course goals to improve as well as evaluate student learning.

  5. Background of STI - continued • Studies showing students’ lack understanding of statistical concepts and statistical reasoning • Teachers’ resistance to recommended ways of teaching • ARTIST project: need for instrument to use in research studies along with CAOS • INSPIRE project: need for data on teachers

  6. Development of STI • Objectives of the STI • To assess practice and beliefs of teachers in introductory statistics courses • To pilot an instrument and integrate it into a database to explore the relationship between teaching and student learning in introductory statistics courses • Funding from NSF (ARTIST, INSPIRE, and STEPS grants, 2002-2010) • Support from two mini grants from Dept of Ed. Psych, University of Minnesota

  7. Development of STI • Development process of the STI • The first version (102 items) • Pared down based on feedback from members of the statistics education community (e.g., the Research Advisory Board (RAB) of CAUSE) • Focus group conducted with faculty from the two different disciplines • Online pilot testing followed by focus group interviews • The resulting version of the STI administered to 101 participants of the 2009 US Conference on Teaching Statistics (USCOTS)

  8. STI • The latest version of the STI: 50 multiple-choice items • Four sections • Teaching Practice • Assessment Practice • Teaching Beliefs • Assessment Belief Also, course characteristics and additional teacher information

  9. STI - Examples • Part 1: Teaching Practice • e.g.) Small group discussions are used to help students learn. Never, Seldom, Some of the time, Most of the time, All of the time • Part 2: Course characteristics • e.g.) Please indicate the mathematical prerequisite for this course • Part 3: Assessment Practice • e.g.) My assessments evaluate students’ abilities to use formulas to produce numerical summaries of a data set. Disagree, Agree

  10. Examples of the STI • Part 4: Teaching Beliefs • e.g.) Students learn statistics more effectively from a good lecture than from a good activity. Strongly Disagree, Disagree, Agree, Strongly Agree, Undecided • Part 5: Assessment Beliefs • e.g.) Alternative assessments (e.g., projects, presentations, minute papers) should be used to evaluate student learning. Strongly Disagree, Disagree, Agree, Strongly Agree, Undecided • Part 6: Additional information (demographic)

  11. Data collection and Coding • Data collection for Pilot test • Administered to 101 participants of the 2009 US Conference on Teaching Statistics (USCOTS) • Responses

  12. Validation Results – Psychometric Properties • Validation based on the Classical Test Theory • Examination of Reliability, individual item properties, and scale scores • Analysis results

  13. Validation Results –Interview • Interview validation process • Interviewees: 16 instructors who participated in the STI • Face-to-face interview with 10 people at USCOTS • Phone interview with 6 people • Course syllabus and other course materials provided for validity evidence • Rating of interview based on the GAISE report • A consensus of rating reached from discussions with three professionals

  14. Validation Results – Interview • Correlated 16 interviewees’ scores on the STI with their interview rating • Four outliers found: 2 of them are high on STI but low in interview; 2 of them are low on STI but high in interview • Indicates that these four responses on two measurements (STI & interview) do not match.

  15. Validation Results – Interview • Why the outliers? • Thorough examination of interview recording and course materials • Some of them changed the course after taking the STI • Some of them have different ideas of use of technology or the teaching methods recommended in GAISE.

  16. Conclusions • Psychometric validation of the STI supported through high value of Cronbach-alpha (>0.80) • The instructors sampled use a moderately reformed approach in teaching statistics • Difference between the mean scores for the Teaching Practice (0.58) and the Assessment Practice (0.74) which suggests

  17. Discussion • Instructors have different conceptions in rating themselves • Their beliefs and practices were sometimes influenced by constraints • Revisions needed • some items have less than 0.30 (in item discrimination), one item has zero standard deviation • Mixed item format made the interpretation of the result inconsistent

  18. Limitations • A larger sample is needed to make better statistical and psychometric analyses • Biased sample (USCOTS particpaints) • The coding of items and the scaling of scores regarding the mixed-item-format

  19. Current work • Modular version with new parts for online and hybrid classes • International version, based on interviews at International Conference of Teaching Statistics (ICOTS) • National survey Fall 2011 (e-ATLAS project to NSF) • Linking STI results to new CAOS results (e-ATLAS) • Implementation of classroom observation rating instrument for further validation

  20. Thank You parkx666@umn.edu audbjorgb@gmail.com Jiyoon Park Audbjorg Bjornsdottir Department of Educational Psychology

More Related