I ain Weir, Rhys Gwynllyw & Karen Henderson CETL-MSOR 2014

1. An innovative use of technology to aid the service teaching and assessment of statistics to a large cohort Iain Weir, Rhys Gwynllyw & Karen Henderson CETL-MSOR 2014

2. Motivation • Level 2 research skills module delivered to 650+ Business School Students • Short course on statistics covering a challenging amount of material together with learning to use SPSS • Designed to provide a solid foundation for students to undertake Level 3 project work • Electronic assessment of bespoke data

3. Weekly schedule • Exploratory data analysis (EDA), one sample T-test & nonparametric equivalent • Two sample T-tests & nonparametric equivalents • One-way ANOVA & nonparametric equivalent • Correlation & simple linear regression • Multiple regression • Principal Component Analysis (PCA) & Cluster Analysis Two online tests: one-way ANOVA & Multiple regression

4. Basic statistical analysis tasks • Transfer from Excel files to required SPSS format • Exploratory data analysis for summary statistics, graphics and assumption testing • Identification of appropriate statistical test (parametric or nonparametric equivalent) • Interpretation and reporting of test output Typically assessed using a written report

5. Lectures • One hour per week • Notes covered statistics and use of SPSS • No reference in lectures to use of SPSS to gain output in examples presented • Students can concentrate on the statistical concepts and SPSS output interpretation

6. Computer lab preparation • Expectation between of engagement between lecture and following computer lab • Tasks requiring students to recreate the SPSS analyses in the notes • Transfer of data into required SPSS format covered • Self-learning aided by a suite of SPSS “how-to” videos • Students can concentrate on the mechanics of the creation of output and familiarisation of concepts

7. Computer labs • One hour per week • Students given questions that covered various data outcome scenarios (para/non-para, sig/not sig, post-hocs) • First question was designated for the student’s Learning Journal; a pre-written complete analysis template with parts removed for the student to complete • Students requiring SPSS “button pushing” help were directed to notes and videos • Staff concentrate in labs on giving statistical understanding and interpretation advice

8. Learning Journal extract

9. DEWIS test 1: One-way ANOVA • A parametric test to evaluate the equality of 3 or more population means using sample data • If the test assumptions not met then a non-parametric test to evaluate the equality of the population distributions is carried out • If there is evidence of a difference then Post-Hoc tests are employed to detect where the differences lie

10. Stage 1: Importing the data from Excel to SPSS Multiple submissions allowed Stage 2: EDA and identifying the appropriate test One submission allowed Stage 3: One-way ANOVA or Kruskal-Wallis test One submission allowed NB Students can quit after submitting a stage and marks already accrued will be saved

11. e-Assessment mechanics Student e-Assessment system DEWIS

12. e-Assessment system: DEWIS • Can communicate with the R statistical package which was employed to generate bespoke student data and generate answers that would match SPSS screen output • Run staged assessments • Implement continuation marking for a large number of inputs • Provide dynamic feedback specific to student inputs

13. Students raw data Students taught to pay attention to precision of recorded data

14. Correct SPSS format DEWIS marks using expected precision

15. Incorrect SPSS format

16. Stage 1 Download data Unlimited attempts • Checks: • transfer • precision

17. Submission

18. Feedback Answers not given Correct Incorrect Close • Extra information: • PDF • Video

19. Extra Information: PDF Link from within DEWIS to personal web page host 9 pages later:

20. Extra information: video • Captive slide shows with voice overs • Link from within DEWIS to personal web page host http://www.cems.uwe.ac.uk/~isweir/MoE/DEWIS1/ReadingDEWIS1Data.htm

21. Resubmission When correct the student is allowed to proceed to Stage 2

22. Stage 2 13 questions 67 activities only one submission

23. Feedback: The Question

24. Feedback: The Solution

25. Feedback: The Report

26. Stage 3 Only correct test choice allowed 6 questions 26 activities only one submission Need to design balanced questions for all competing scenarios

27. Marking scheme Stage 1 is worth 7 marks for successful data transfer otherwise 0 Stages 2 and 3 • Numerical value: correct=1 incorrect=0 • Multiple choice: correct=1 incorrect=-1 not answered=0 In total 100 activities each worth a mark Negative marks for Stages 2 or 3 are truncated to zero

28. Outcomes • Results have been excellent • Teaching and assessing in this way has made the challenging task of delivering and assessing this material in a short space of time achievable • The fact that students can refer to their Learning Journals and access “how-to” SPSS videos will be beneficial to their further studies

29. Colleague quote “…..we are getting them to carry out quite sophisticated analyses, with a lot more success than I believed possible at the beginning!”

30. Next? • Refine work in light of first delivery experience • Extend to other modules • Examples of DEWIS with R in public domain • Funding