Drs. Steven C. Myers & Michael A. Nelson myers@uakron

1. Do On-Line Students Analyze, Synthesize And Evaluate Better Than Face-To-Face Students? Preliminary Evidence Drs. Steven C. Myers & Michael A. Nelson myers@uakron.edu Department of Economics - The University of Akron Presented to the Windows on the Future 2003 Conference OLN and ITEC-Ohio March 2, 2003 Online at http://gozips.uakron.edu/~myers/online

2. Goals for Presentation • Talk about our ongoing research • Research Questions • Modeling process • Results • Talk about the Instructional Design of the Online course • Mastery and Competency Based Learning • Encourage you to contact us

3. Research Questions (1) Does the mode of delivery (face-to-face or web-based) have an influence on learning outcomes?  (2) Are students in an online environment as likely to do as well as in face-to-face classes? Will they be able to equal the complex problem solving of the face-to-face students? (3) Will web-based students develop more favorable attitudes towards economics than the attitudes developed by students in the face-to-face class? (4) Do student myths about economics affect their learning outcomes and attitudes towards economics?

4. Acknowledgments • Grant from Carnegie Teaching Academy Scholarship of Teaching, Assessment and Learning Funds Institute for Teaching and Learning The University of Akron • Data collected under signed informed consent from our students subject to the Institutional Research Board for the Protection of Human Subjects at The University of Akron. • Very few failed to give informed consent • Data collection instruments validated by a number of independent reviewers and created by MA grad, Michael Lovette.

5. Course under study • Introduction to Economic Analysis • One-semester principles of economics • 3 cr. General education course • Required for engineering majors • Both face-to-face and on line • The online course is similar to a graduate course offered since Fall 2001

6. Face-to-face Offered by Michael Nelson Regular class times Lecture and active learning collaborative techniques Better than the norm of 83% “chalk and talk” (Becker and Watts) Some minor web-enhancements such as online gradebook Online Offered by Myers (F02) / Nelson (S03) No face-to-face meetings Mastery & competency based learning High professor-student interaction No student to student interaction Full use of the WebCT environment Comparison

7. Student Comments / online • “The fact that this course was completely internet based, had no bearing on the level of knowledge I gained from the course. I think I learned as much, or even more, than I would have in a traditional classroom. I contribute this most to the way the instructor organized the course.”

8. Student Comments / online • “I believe that the benefits of web-based courses … far outweigh any disadvantages such as lack of face-to-face interaction--at least in this particular course. Dr. Myers' course was, of the four I took this semester over the web, truthfully the best organized, most well-adapted to the web (by his efforts) of them all.”

9. Research design Student’s success =f( student’s initial endowments, awareness of the economy, attitudes about economics, course modality, student characteristics)

10. Research design Student’s success =f( student’s initial endowments, awareness of the economy, attitudes about economics, course modality, student characteristics) Pre-test scores total on recall and recognition on simple application problems on complex problems GPA Previous attempt of course

11. Research design Survey of economic facts 20 questions…on trends and facts in the economy. 2 questions: How does the student collect business and economic news. How many hours per week does the student spend collecting news Student’s success =f( student’s initial endowments, awareness of the economy, attitudes about economics, course modality, student characteristics)

12. Research design Three Survey Questions: Level of student report of being interested in economics. Level of student report of liking this economics course. Level of student report about likelihood of taking more economics courses. Student’s success =f( student’s initial endowments, awareness of the economy, attitudes about economics, course modality, student characteristics) 5=strongly agree 4=agree 3=indifferent/no opinion 2=disagree 1=strongly disagree

13. Research design Student’s success =f( student’s initial endowments, awareness of the economy, attitudes about economics, course modality, student characteristics) Online or face-to-face Major Level, e.g., freshman Credits completed Age Gender Ethnicity

14. Dependent Variable: Student Success Outcome Measures • examscore on first 6 chapters • In class exam for face-to-face class • Average of first three module quizzes for online class • Final examination • (FE, FE_recog, FE_apply, FE_complex) • First 3 or 4 levels of Bloom’s Taxonomy • Writing assignment • writing2 • 4th to 6th level of Bloom • Regular Grades in class

15. Bloom’s Taxonomy (from Carla Lane) • Knowledge: arrange, define, duplicate, label, list, memorize, name, order, recognize, relate, recall, repeat, reproduce state. • Comprehension: classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, restate, review, select, translate, • Application: apply, choose, demonstrate, dramatize, employ, illustrate, interpret, operate, practice, schedule, sketch, solve, use, write. • Analysis: analyze, appraise, calculate, categorize, compare, contrast, criticize, differentiate, discriminate, distinguish, examine, experiment, question, test. • Synthesis: arrange, assemble, collect, compose, construct, create, design, develop, formulate, manage, organize, plan, prepare, propose, set up, write. • Evaluation: appraise, argue, assess, attach, choose compare, defend estimate, judge, predict, rate, core, select, support, value, evaluate.

16. Online Students More likely to be To be older To be non-white To be female To have taken more credits To be decided in their major More favorably disposed to economics Face-to-face Students More likely To be male To be a freshman To be undecided in their major To spend less time gathering business and economic news. Student Characteristics

17. The online course design has influenced student success Grades Distribution MC Learning Assessment Breakdown / Online

18. Results Students’ Initial Endowments • All prior expectations are met • Higher pretest and higher GPA are positive on examscore, writing2 and less so on FE. • Higher complex pretest scores and higher GPA are positive for writing2 and FE_Complex

19. Results Economic Awareness & Attitudes • “Survey” is weakly positive for examscore and writing2, but has no effect on FE. • No combination of “News gathering” is ever significant. • “Attitudes” are strongly significant for examscore and writing2, but no effect on FE • When attitudes matter • take_more econ > interest in econ > will like econ

20. Results Student characteristics: • Those with an undeclared or undecided major do much worse than engineering majors for examscore and FE. • Older students do worse on examscore, no effect on writing2 and strongly positive on FE • Females do better in writing2 (strongly significance), but do worse on FE (almost significant)

21. Results Online Students: • Do much better on examscore • Show no significance difference in scoring on analytical and complex reasoning tasks. • Insignificant coefficients in writing2 • Insignificant and small coefficients in • FE • FE_recog • FE_Apply • FE_complex • There is some evidence of a possible interaction with females • Weak suggestion in the data that • Men in online classes do better and • Women in online classes do worse all things equal

22. Design of the online course • Built in modules • Modules completed in order • Competency based testing • Use student feedback • For student learning enhancement • For modifying & improving the course

23. CourseDesign • DL requires planning for contingencies • 14 Content Modules • Active Learning vs. Passive Learning

24. Pre-class and Module 1: Preparing Students to Learn • Email me! http://gozips.uakron.edu/~myers/online/ • Is Distance Learning for Me? • VARK – testing learning styles • Orientation Module—‘ How to logon to WebCT’ • Orientation Module—‘ How to Use WebCT’ • Syllabus • Graduate course • Undergraduate course

25. Pre-class and Module 1:Additional Orientation • How to Communicate with Dr. Myers • How to access your online text • http://www.economicsplace.com/econ5e/ • Rules of the game • Building a relationship – • Breakdown the anonymity • Survey “Tell me about yourself”

26. Content Modules 2-15 • Module Introduction & Objectives • Chapter Introductions • Content • Supportive Materials • Assessment of Learning - Quizzes on objectives with multiple trials • Evaluation

27. 2 Research & Writing Assignments – some objectives • Practice and experience in reflecting on a topic in the current economy. • Practice in analysis of economic trends. • Gaining of confidence about talking about the economy. • Ability to know and use the resources of economic commentary, prior analysis and data.

28. Introductions& Content • Mostly Passive • Learner Centered • Students progress without intervention • Micro – Modules 2-8 • Macro –Modules 9-15

29. GradedAssessment • Competency based • Everyone strives to get a perfect 10 (Mastery) • Three attempts, 15 min. time limit • Questions a mix of (1) Recognition, (2) Conceptual, &(3) Analytic~70% C&A • Random intervention by Professor

30. Role of Module Evaluation • Planned intervention • Forced contact • Focus on the learning • Professorial encouragement • Decreases dropout rates • Process repeats

31. Module EvaluationA Classroom Assessment Technique from Angelo and Cross (1993); tested by Chizmar and Ostrosky (1998) • What comments do you have on this module and your experience in completing it? • What main point have you learned that you did not fully understand before? • What questions … Include any points that still remain muddy or unclear. Do consider posing the muddy points to your fellow students in the discussions. • What recommendations do you have for us as we continue to change and enhance the course?

32. Do On-Line Students Analyze, Synthesize And Evaluate Better Than Face-To-Face Students? The answer is NOand they don’t do any worse either. Contact: Steve Myers myers@uakron.edu The University of Akron Or get this paper and presentation Online at http://gozips.uakron.edu/~myers/