CAPHIA TEACHING AND LEARNING FORUM 18-19 September 2014, Perth

1. CAPHIA TEACHING AND LEARNING FORUM18-19 September 2014, Perth Evaluating flipping: Development of the Flipped Classroom Student Engagement Questionnaire (FCSEQ) Jane Taylor Public Health Program University of the Sunshine Coast

2. Flipped team Exploratory Learning and Teaching Grant, 2013 Centre for Support and Advancement of Learning and Teaching, USC FCSEQ Movie Jane Taylor, University of the Sunshine Coast

3. Overview • A bit about flipping • Research aim and process • Flipped Classroom Students Engagement Questionnaire (FCSEQ) • Some very preliminary (limited) findings • Comments & questions Jane Taylor, University of the Sunshine Coast

4. What is a flipped classroom? • Pre-class online learning activities • Face-to-face workshops • Post class learning activities Jane Taylor, University of the Sunshine Coast

5. http://ctl.utexas.edu/teaching/flipping_a_class/what_is_flippedhttp://ctl.utexas.edu/teaching/flipping_a_class/what_is_flipped Jane Taylor, University of the Sunshine Coast

6. Why flip a classroom? Constructivist pedagogy • Student centred • Active and problem based learning strategies • Emphasis on student responsibility for learning • Facilitator role of teacher Students • Learn more deeply • Greater participation • Increase in interaction and learning from one another • Greater level of instructor feedback • More time on higher levels of learning e.g., analysis, synthesis and evaluation Jane Taylor, University of the Sunshine Coast

7. Flipped project overview Jane Taylor, University of the Sunshine Coast

8. Why this research? • Limited literature on assessing and monitoring impacts of flipped classroom approach on student engagement and learning • Engagement = involvement in purposeful / meaningful academic activities (Kuh, 2001) • Aim: To develop and pilot an impact evaluation questionnaire called the Flipped Classroom Student Engagement Questionnaire, to evaluate the impact of the flipped classroom on student engagement in learning Jane Taylor, University of the Sunshine Coast

9. Flipped Classroom Student Engagement Questionnaire Aim • To assess students’ overall level of engagement at the course level, and in pre, in class and post class flipped classroom components Development • Item generation – literature review, existing instruments and items, peer-review, pilot test • Validation (currently here) • Finalisation Jane Taylor, University of the Sunshine Coast

10. 1. Item generation • Literature review • Engagement measures at course level • Handelsman et al., 2005; Delialioglu, 2012 • National Survey of Student Engagement (NSSE), 2011 • Student engagement constructs & indicators • Skills, Emotional, Participation and interaction, Performance (Handelsman et al) • Active and collaborative learning (Delialioglu, 2012; NSSE) • Time on task (Delialioglu, 2012) • Overall level of engagement (Handelsman, et al., 2005) • Confounding variables Jane Taylor, University of the Sunshine Coast

11. Jane Taylor, University of the Sunshine Coast

12. Survey examples Emotional engagement Skills engagement Performance Participation/interaction engagement Active and collaborative learning Jane Taylor, University of the Sunshine Coast

13. Pre-testing instrument • Peer-review • Steering Committee • Two additional academics • Pilot test • OT class using flipped • Interpretability, usability, timeliness… Jane Taylor, University of the Sunshine Coast

14. 1. Item generation • Confounders (Handelsman et al) • Entity theory of learning • Learning can be extended and is not a fixed entity (Handelsman et al) • Incremental theory of learning • Certain amount of intelligence and you can’t do much to change it (Handelsman et al) • Socio-demographic characteristics • Age, sex, employment, level of education striving for; level of parent education; international/domestic student; living arrangements, access to internet at home, confidence in using technology (NSSE) • Assessment of flipped approach compared with usual/traditional • Perception of impact of flipped approach on academic performance, non-academic skills, working in groups, preferences Jane Taylor, University of the Sunshine Coast

15. FCSEQ structure • Four sections • Overall engagement • Participation in course • Achievement • Socio-demographic • 19 closed-answer questions • 2 open-ended questions • 10 minutes • Administration hard copy or electronic Jane Taylor, University of the Sunshine Coast

16. 2. Validation • Administration: four public health courses, one paramedic course • Electronic and hard copy administration • June-August 2014 Jane Taylor, University of the Sunshine Coast

17. 2. Validation • Factor analysis may collapse one or more constructs – in progress • How do these constructs then align with other student attributes and measures of performance? Jane Taylor, University of the Sunshine Coast

18. 3. Finalisation & next steps • Finalise instrument by end October 2014 • Recruitment of more flipped classrooms by end January 2015 • Full implementation 2015 • Snippets from public health courses Sem 1 2014 Jane Taylor, University of the Sunshine Coast

19. Overall engagement in course across semester? Most engaged n=51 Jane Taylor, University of the Sunshine Coast

20. Overall flipped or standard classroom? Most prefer flipped n=32 Jane Taylor, University of the Sunshine Coast

21. Overall level of engagement? Most more engaged n=51 Jane Taylor, University of the Sunshine Coast

22. Overall academic performance? Some felt it do, others not sure, some not n=50 Jane Taylor, University of the Sunshine Coast

23. Pre classDoing pre class activities?(Active & collaborative learning) Characteristic of most to do something n=52 Jane Taylor, University of the Sunshine Coast

24. Pre classHours per week preparing for workshops(Total time on task) Majority 1-5 hours n=51 Jane Taylor, University of the Sunshine Coast

25. In-classWorkshop attendance?(Skills engagement) Most characteristic to attend n=52 Jane Taylor, University of the Sunshine Coast

26. In-classWorkshops without pre-class activities?(Active & collaborative learning) Some do, some don’t n=49 Jane Taylor, University of the Sunshine Coast

27. In-classHaving fun in workshops?(Participation / interaction engagement) Majority having fun n=52 Jane Taylor, University of the Sunshine Coast

28. In-classActive participation on small groups discussion? (Participation /interaction engagement) Majority small group discussions characteristic n=52 Jane Taylor, University of the Sunshine Coast

29. In-classExplaining materials to others? (Active & collaborative learning) Majority sometime/ often explained course materials to others n=50 Jane Taylor, University of the Sunshine Coast

30. Post classMaking course material relevant to life?(Active & collaborative learning) Majority found ways to make course material relevant to life n=52 n=52 Jane Taylor, University of the Sunshine Coast

31. Post classRead additional materials(Active & collaborative learning) Majority read additional material related to course content Jane Taylor, University of the Sunshine Coast

32. FCSEQ qualitative feedback Think more engaged? • Interaction in class (n=9) • Having a supportive class learning environment (n=7) • Pre-class activities assisting in preparing for class (n=4) Supported engagement? • Getting to know teachers (n=4) • More in-depth learning and expanding on content (n=3) • Ensuring on-track or same page (n=3) • Getting to know/ learning from fellow students (n=3) • Being responsible for/being in control of own learning (n=2) • Not like pre-class activities (n=2) • 3 hour workshop too long (n=1) Jane Taylor, University of the Sunshine Coast

33. Comments & Questions Jane Taylor, University of the Sunshine Coast

34. Key References • Australian Council for Educational Research. Australasian survey of student engagement: Student Engagement Questionnaire. Australia, 2011. • Handelsman M, Briggs W, Sullivan N & Towler A. A measure of college student course engagement. The Journal of Educational Research 2005;98(3): 184-192. • Delialioglu O. Student Engagement in Blended Learning Environments with Lecture-Based and Problem-Based Instructional Approaches. Journal of Educational Technology & Society 2012;15(3): 310. • Biggs J. & Collis K. Evaluating the quality of learning: The SOLO Taxonomy. New York: Academic Press, 1982. • Krathwohl D. A revision of Bloom's taxonomy: An overview. Theory into Practice. 2002;41(4): 212. Jane Taylor, University of the Sunshine Coast