A Science of Subjectivity: A physicist reflects on teaching, learning, & Q methodology

1. A Science of Subjectivity: A physicist reflects on teaching, learning, & Q methodology Susan Ramlo, PhD Presentation at Wichita State University, Department of Physics 2019

2. A little background… • 6.5 years as an Industrial Physicist (making radiation detectors) • University of Akron faculty • PhD in Physics Education Research (dissertation – quantitative) • Learned about Q methodology from a friend at an education research conference… • Started using Q for some research studies… • Learned Q’s creator, William Stephenson, double-PhD in physics & psychology • Read Stephenson’s work relating Q to quantum mechanics… • and viola… I was more of a social science methodologist than PER person… • More philosophy of science focus… • I’m the only physicist within the international Q community.

3. A little more… • International Society for the Scientific Study of Subjectivity • 2008-9 VP & Conference research chair • 2009-10 President • Host of 2010 conference in Akron, OH • Journal editorial board • Involved in mixed methods research in social sciences • 1 published book chapter • 3 more in process / editor review • Over 50 publications including book chapters, journal articles, etc. • Journal reviewer • Etc.

4. everything is inherently subjective.William Stephenson, 1987

6. Presentation abstract Q methodology [Q] is an 80-year-old mixed method developed by William Stephenson to scientifically study subjectivity. More specifically, Q offers an empirical method for studying subjectivity by revealing and describing the multiple divergent viewpoints that exist within a group of interest. In Q, a researcher collects statements about the topic, reduces their number in a way that preserves the communications, and then participants sort those statements into a grid that ranges from something like “most like my view” to “most unlike my view.” Q’s technique is the sorting of statements related to the topic and its method is the factor analyzing of those sorts to group people with similar viewpoints. Factor analysis in social science research has the same mathematical basis as the mathematics of quantum mechanics (QM). Yet, beyond the mathematical similarities, there are conceptual similarities between Q and QM. This becomes less surprising when we understand that Q’s creator had PhDs in both psychology and physics. Within this talk, I will introduce the audience to Q methodology and demonstrate how I have used this somewhat obscure mixed method to investigate and improve teaching and learning of physics.

7. William Stephenson PhD in Physics PhD in Psychology Wanted to study Psychophysics Combined concepts from psychology and quantum mechanics to create “Q methodology” Ended up teaching journalism research at University of Missouri

8. Q methodology • Scientific way to determine divergent perspectives within a group about a topic. • Mixed method (qualitative-quantitative hybrid) • 80+ years old • Used in a variety of fields: education, political science, environmental studies, journalism, marketing • Concourse – diverse communications • Written communications • Interviews • Open-ended surveys • Newspaper articles • Q-sample – subset of the concourse • Typically between 40 & 60 items • Purposefully selected across themes (Fisher’s Design of Experiments)

9. Why not use Likert-scale surveys? • Averaging process obscures individual differences  Loss of meaning • Different viewpoints exist even after the same experience (think student evaluation of teaching results)

11. Method: Q methodology (2 of 2) • Sorting – participants sort the items (on slips of paper or individual cards) into a grid • An operation of interpretation • Self reference • An individual’s expression of feeling • Post-sort interviews or written comments. • P-set (participants) • Students, faculty, etc. • Analyses • Factor analysis • Factor arrays • Distinguishing statements • Consensus • Interpretation of factors (views)

13. Q & QM Parallels • The form of factor theory in Q-methodology fits squarely with the mathematical formulation of quantum theory in physics* • Treatment of causality and probability* • Status of the uncertainty principle* • Inseparability of object and measuring instrument* • Quantum States – States of Mind (Feeling) • Heisenberg (1975): It is a matter of preparing the phenomena of nature so that they can display their structure. • Stephenson (1980, 1982): As it is for Q-factor theory in psychology in its search for natural subjective phenomena. It is a matter of preparing phenomena of mind, so called, so that it can display its structure. • *Stephenson (1982)

14. What’s in this talk? • Evaluating Flipped Physics Classrooms • Student Views Regarding Offering a Physics Course Online

15. Evaluating Flipped Physics Classrooms Ramlo, S. (2015). Student views about a flipped physics course: A tool for program evaluation and improvement. Research in the Schools, 22(1), pp 44-54. Ramlo, Susan (2015). Giving students choices in a flipped physics classroom. Ohio Section of the American Association of Physics Teachers, Cleveland State University, Cleveland, OH. Ramlo, Susan. (2014). Evaluating three flipped physics classrooms: Refining an ongoing Q Study. Paper presented at the International Society for the Scientific Study of Subjectivity / Q Methodology Conference, Salt Lake City, UT. Ramlo, Susan (2014). Putting theory into practice: Describing and evaluating flipped physics classrooms. Paper presented at the Eastern Education Research Association, Jacksonville, FL.

16. Flipped classrooms • Pedagogical model in which the typical lecture and homework elements of a course are reversed. • Lectures are recorded via video (screen-capture) & watched outside of class time. • Active learning (including homework collaboration and interaction with instructor) during class time. • Innovators include Eric Mazur (peer instruction) & Khan Academy (vodcasts)… but flipped classrooms combine their ideas and more…

17. What flipped looks like via online classroom management system:

18. In class (if everyone is prepared – e.g. watches videos &/or reads book &/or Ramlo Cliff-Notes): • Q&A about related videos at start of class • Clicker questions – examining student understanding of concepts • Group work – examining student understanding of problem-solving • Time to work on homework IN CLASS with peers & instructor.

19. Benefits • Instructors interact with students & learn about student difficulties. • Instructors more effectively adapt instruction to students. • Pedagogy is better aligned with learning theory. • Instructors & students have more flexible applications of technology inside & outside the classroom.

20. Problems • Some students are resistant to watching the videos • Instructors typically have to make their own videos tailored to the course / learning objectives / students. • Limited research on effectiveness and student views of flipped classrooms… primarily surveys that examine data in aggregate or based upon demographic information.

21. These Physics Classes • First & second semester, non-calculus, freshman physics for non-majors. • Serves variety of Engineering Technology programs (AAS & BS). • ~50% of these students had physics in high school (but not necessarily of good quality) • Inquiry based laboratories • Focus on problem-solving & conceptual understanding associated with force & motion. • Classroom Response System (clickers), PhET simulations, group work • Previous evaluations indicated that students wanted more group work & more examples.

22. Initial study: Findings(first semester only) Factor 1 (12): Active Learners Factor 2 (7): Unprepared Traditionalists Learning & grades are important Interactions with peers & instructors are important for their learning. Group work seen as beneficial to their learning; Instructor expectations were fine as was the level of the course and the level of math required. Watching the flipped videos improved their learning Grades are very important… but wish they were better. Realize they were unprepared for the exam & course, in general. They did not feel they had to prepare for the clicker questions (pre-group work assessments). Clicker questions did not help them assess their learning or understanding. Flipped videos were boring & didn’t help them learn Prefer having regular, traditional lectures during class time

23. Initial study… continued Consensus Grade Distribution Neutral about the textbook Neutral about difficulty finding time to watch the flipped videos outside of class. Grades are important (but what that means seems to differ between the 2 factors/views). Both views agreed that working on problems / assignments in class with classmates was enjoyable Factor 1 – more about usefulness related to learning Factor 2 – more about social aspects of interacting with peers

24. Next steps… • The instructor needed to make changes to help the Unprepared Traditionalists become more prepared • Presentation on why the course was flipped • Demonstrated how to access the videos and other resources on the online course management system (Springboard) including flexibility of watching the videos. (Cannot assume even traditional aged students are techy) • Preliminary results, interviews, & written comments revealed a need to expand on some of the ideas in the statements especially details about the use of videos and the new suggestion that students take notes during the videos as well as parcing out details about group work: • Active learning versus social / friendship • Edited the Q sample to “fine tune” the investigation

25. Quick comparison New Q sample Premier Q sample

27. Summary of Take#2 • Focus on 2nd semester students (many of whom were in initial study) • 42 statements in Q sample (same as initial) • Unchanged sorting grid • Sorted at final exam • Added new Likert scale responses to the following : Answer the following where the scale goes from 1 (NEVER) to 5 (ALWAYS) - circle your response • 1) I read the textbook (not just the problems assigned) 1 2 3 4 5 • 2) I watched the “lecture” videos for the various chapters 1 2 3 4 5 • 3) I tried to complete all of the homework assignments 1 2 3 4 5 • 4) I was excited about taking this course 1 2 3 4 5 • 5) I took notes while watching the lecture videos 1 2 3 4 5

28. Two factors… Distinguishing statements

29. Same class, same opportunities, different views

30. Factor 1 (+5 & +4) = 17 sorters

31. Factor 2 (+5 & +4) = 6 sorters

32. Comparing F1 to F2 ACTIVE PASSIVE

33. Two factors F1 (17): Active Learners F2 (6): Time efficient traditionalists Positive about course & instructor Focused on learning Liked course design (choices?) Disliked textbook Group work important for their learning Learning takes time & is important (epistemology) Active learners Neutral about course & instructor Wish they learned more Preferred text over videos Hoping to pass Videos boring & take time… “Time management” focus More “Traditional”? Passive learners

34. Conclusions • Revised Q sample provided new insights about student views of the flipped classroom. • Choices are important for motivated students so that they can select the methods that best enhance their learning. • Some students possess “naïve epistemologies” that focus on quick learning, passing tests / courses, and “efficiency” which they call time management… broader than “Unprepared traditionalists” • These results provide greater detail than the original study as well as those of Enfield (2013), Fulton (2012), & Herreid & Schiller (2013). • Changes to the course(s) seemed to assist both types of students: • Better understanding of “choices” for all of us • Print materials (Power Points) important for Factor 2 students especially • Promotion of taking notes during flipped lectures beneficial for some

35. Student Views Regarding Offering a Physics Course Online Susan Ramlo, PhD The University of Akron Presentation at EERA 2016 Ramlo, S. (2017). Student views regarding online freshmen physics courses. Research in Science & Technological Education, 35 (4), pp 461–476. doi: 10.1080/02635143.2017.1353961. Available at http://www.tandfonline.com/eprint/aqz6ZtS5FcxwjU4Jq6uX/full. Ramlo, S. (2016). Students' views about potentially offering physics courses online. Journal of Science Education and Technology, 25(3), pp 489-496. doi: 10.1007/s10956-016-9608-6. Available online at http://link.springer.com/article/10.1007/s10956-016-9608-6 & https://rdcu.be/6l0m

36. Status of online university courses • 1/4th college students (about 5.4 million students) took at least one distance (online) course during Fall 2012 (National Center for Education Statistics, 2014). • Associated with for-profit… but their enrollment has been dropping (Blumenstyk, 2016). • For profit embedded in traditional public universities (Blumenstyk, 2016) • Public university administrators see flexibility, potential for enrollment growth, and money.

37. Background for the study • Increased talk about putting classes online at UA. • Students complaining about online as only option for some classes. • My Dean started push for online physics. • Research question: How do university students who are enrolled in face-to-face engineering technology programs feel about taking courses online, especially courses that are perceived to be difficult like physics?

38. Q Methodology to investigate student views • Concourse of statements – students & administrators • Q sample selected to represent communications on the topic (45 statements) • 47 eng-tech students in 1st & 2nd semester of Technical Physics sorted the statements into a grid. • Post-sort survey • 60% with prior online course experience

39. Three factors/views emerged from analyses • Factor 1 – Keeping it real and face-to-face • 23 students • Reject the idea of online classes • Factor 2 – Online could be ok, depending on course and instructor • 9 students • It’s all about the instructor and the course… if it’s the right mix they would be willing… if the price is right (lower tuition for online) • Factor 3 – Online not for STEM classes • 4 students • No STEM type classes online • Humanities / writing-based courses ok • Not interested in social aspects of classroom like other views… prefer to work alone.

40. Written comments: Factor 1 – Keeping it real and face-to-face • Previous online course was “a decent course, not horrible but not great… I feel that the traditional classroom setting is a richer learning experience than online classes because in a classroom setting, there is interaction and collaboration between peers and instructors which I believe is a crucial part of learning… Simply, learning to me is more effective in a classroom setting.”

41. Written comments Factor 2… depends on course and teacher • #23 = he stated that “A good instructor is required regardless of the teaching medium.” • #36 = described how he likes “being in class interacting with people and talking to people outside of class. I like to work in groups to make sure we are all understanding the material.” • #41 = probably summarized this view the best: “I think it would be a bad idea if all classes were online. If the professor isn’t that good then the course is gonna be tough and students wouldn’t learn that much. Some online courses are alright depending on the course and instructor.”

42. Written comments Factor 3 – Online not for STEM • #15 = wrote “Engineering, physics, or any STEM degree online will deter students and fail miserably. It’s like teaching a mechanic to build an engine with a blindfold on…”

43. Consensus – agreement among the factors/views • 8 consensus statements • 4 from administrators – students agree to disagree • Students agree that simulated laboratories are not as effective as those using real laboratory equipment (#12, -4, -5, -5; #21, -4, -4, -5). • Statement #34 (+5, +4, +5) -students feel they learn best with hands-on activities in the laboratory and classroom. • Concern regarding: • Difference between interacting with an instructor in an online class compared to interacting with an instructor in a face-to-face class (#25, -5, -3, -4). • How future employers would perceive a student who took online classes and/or completed a program partially or completely online

44. Conclusions • Three different student views of online • These views & consensus are out of sync with the apparent views of administrators (based upon positions of statements from administrators). • #43 (I love technology) – administrator’s view; grid positions: 0, +2, +2. • “Just because our generation likes our phones and computers that doesn’t mean we want to take classes on them.” • Administrator views of students affected by goals related to enrollment, tuition, and budgets? • Future research – have administrators sort statements • http://link.springer.com/article/10.1007/s10956-016-9608-6.

45. Inside Higher Education • High School Students' thoughts about online classes

46. What is Next? • Examining large lectures versus smaller lectures in teaching first two semesters of physics • Survey of instructors • Student Evaluation of Teaching (using Q methodology) • Possible instructor Q sort (maybe online) • Continue to examine theoretical aspects of Q methodology especially in relation to QM and science.

47. Questions?