1 / 13

John Ogier

John Ogier. Making summative unit surveys totally summative - risks or opportunities. Collings AHEEF 2007 I misheard David’s method!! What are the implications of a long survey window? Into and after exams? Post grades? Response demographics?? Immediate response Post exams Post grades

mercia
Download Presentation

John Ogier

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. John Ogier Making summative unit surveys totally summative - risks or opportunities

  2. Collings AHEEF 2007 I misheard David’s method!! What are the implications of a long survey window? Into and after exams? Post grades? Response demographics?? Immediate response Post exams Post grades General - gender, grades etc Online Survey Experiment Wanted a wide range of large “units” Divide each randomly into 4 groups But high female response rate AUSSE 2007 - so maintain gender balance Got small selection of units Group 1 - normal pre-exams 15 days Group 2 - day after exam 15 days Group 3 - day after grades 13 days Group 4 - open entire 7 weeks Day 1 - personalised invitation email Week before close - reminder email Survey Timing

  3. Also some other units - MATH200, ECON200 and 300

  4. Responses

  5. Variables C_FdBck “I received helpful feedback on my progress” C_Asmt “The assessments in this course measured my learning effectively” C_OAll “Overall, this was a good quality course” T_OAll “Overall, the lecturer is an effective teacher” Gender F, M Group 1, 2, 3, 4 (Survey Group) NumGrade -1 to 9 (“A+” = 9 through to “E” = -1) PostGrades PG (responded post grades/mid-year test return), NA (prior) Week W1 (responded Week 1), Rem (post-reminder email), NA (between) Also knew: - Ethnicity, International Student status, Citizenship, Age, intended Qualification, Overall GPA, Response Day & Time and Time taken

  6. First analysis

  7. What about grades? Figure 4: ANOVA – C_FdBck, C_Asmt, C_OAll and T_OAll by Group

  8. Dommeyer, Baum, Hanna & Chapman, 2004 - online surveys “do not produce significantly different mean evaluation scores” from paper UC Paper and Online survey history support this Not just means - also Std Deviations But are the responses “representative of the whole group”?? (Nulty, 2008) The dangers of “means”

  9. Gender - a response bias? No significant diff @ 95% CI in Likert means or across Groups by Gender

  10. Across the main units analysed: Bivariate fits Correlations for “Overall” Qtns Course R2 = 0.047 Teaching R2 = 0.0045 A+ is 9 ………… E is -1 What about grades?

  11. Survey timing now apparent! Group 2 - post exams Group 3 - post grades Broken into Groups

  12. Why not the full picture?

  13. Oct 6 - online surveys - 40 units Group 2 runs until 15th Dec. Grades released 5th Dec Where to from here?

More Related