Assessing Learning in Hybrid Courses

1. Assessing Learning in Hybrid Courses Rosemary Capps, UC Davis March 6, 2012

2. Agenda • Housekeeping: Preparing for coming weeks • What this webinar will not cover: • Integrity (cheating, proctoring, etc.) • Extensive rubric or test item creation • Student resources on writing • Best practices in designing hybrid course assessment: • Alignment • Timing • Tools • Small group work • Small groups report • Homework:

3. Housekeeping • Coming up: • In-person workshop (3/13) • Two weeks of asynchronous discussion, including peer review of course materials based on rubrics for hybrid course design • Review: Interaction in Hybrid Courses • Follow-up insights or questions? • Preview: Dyrud’s “Group Work and Peer Review” • How can we adapt this to a hybrid course?

4. How can I transform students? What can I see students do that tells me they’ve been transformed? What should students be able to do when they finish? How will I know they can do it?

5. Assessing Hybrid Courses • Best Practices: • Alignment • Timing • Tools

6. Best Practice: Alignment • Course goals • Students will attain the rudiments of visual literacy in the arts of Asia, and learn the basic skills of stylistic analysis. Students will learn interpretive strategies that can reveal cultural ideas and ideals as manifested in visual form, and discover how these disparate factors relate to their historical contexts. Critical thinking skills will be honed through discussions, written homework assignments, and examination essays.

7. Best Practice: Alignment • Homework assignment #1. Due in TA sections by Week 6. • Go to the Asian Art Museum, SF. (In fact, to do well on this assignment and in this course, it will be wise to make repeated visits to the Asian Art Museum to view the collection, hence, a membership can be cost effective.) View the objects in the permanent collection. Select one object from India, China, or Japan, and discuss the following: • Identify the object as provided on the accompanying label. Include title, artist, date, media, country of origin, dimensions if provided, accession number, and other relevant basic information. • Photograph the object: a good all over view, and details of interesting parts. Illustrate your paper with these photographs. • How does this object exemplify an important aspect of Asian cultures? To answer this question, you will first need to describe the object, explain what it is and what it is for. • Length: approximately 500 words. Due in TA sections by Week 6.

8. Best Practice: Timing • Frequent low-stakes feedback • Self-quizzes (Food Freezing, Carnegie Mellon) • Automatically graded homework (Linear Algebra) • Quizzes in face-to-face class sessions (BIS 2C, Food Freezing) • Microvoice (War and Terrorism; see Carey, 2011 [unpublished])

9. Best Practice: Timing

10. Best Practice: Tools • Depends on choices of content and interaction • Primarily online • Piazza • SmartSite • Primarily face-to-face • Rubrics • Peer-group review • Both • Minimize instructor workload • Clarify expectations • Give quick feedback to students

11. Discussion Board 1 Online: • Read: • Mirabelli, Tony. "Learning to Serve: The Language and Literacy of Food Service Workers." What They Don't Learn in School. Ed. JabariMahiri. New York: Peter Lang, 2004. 143-62. Print. (also found in Writing about Writing pg. 538) • Discuss: • Write about your first job or volunteer experience. Consider the following questions: What was it like learning to do the job? What did you learn about the work or the people? Was the job difficult? Why or why not? What was the lexis of that job? How did the lexis and genres that you used in this job helped you do this job? Write a two hundred word response and post in the discussion board under My First Job by Sunday before 11:00 p.m. Reply to one other student post by Monday before 11:00 p.m. • Then, refer to the above post. What strategies do you now know that could influence your approach in a future job? Consider what you learned about your co-workers, managers, or various texts needed to complete the job. Will you navigate a new workplace the same or will you change your strategies? What can you do to learn the lexis and genres of a future job? Write a two hundred word response and post in the discussion board under Successful Workplace Interactions by Sunday before 11:00 p.m.

12. Discussion Board 2 • Discussion Board (Initial post due Wednesday, 2/9, 8pm EST; Response posts due Friday, 2/11, 8pm EST) Based on your experience and on the Totten and Anderson readings, please post your thoughts and respond to the postings of at least two others on the Writing Process Part III forum: • How comfortable and confident do adult learners feel with using a process (like a writing process) in the education setting? • In your experience, what are the pros and cons of teachers providing more or less structure and direction for the writing process? • How might these guidelines apply to writing for the GED test? • If you know of helpful resources on writing for the GED test, please share them with the group in your posts for this week.

13. Best Practice: Tools

14. Assessing Your Hybrid Course • Comments and Questions: • Alignment • Timing • Tools

15. Written Reflection • Write without stopping for 12 minutes, reflecting on one or more of the following questions. • Use your syllabus, your netiquette guide, and your hybrid activity to consider the following questions: • Do the course-level objectives and assessments align? How could I improve their alignment? • Does the hybrid activity align with course-level objectives and assessments? How could I improve its alignmemt? • What is the timing of assessment for the hybrid activity? How could I incorporate frequent, low-stakes feedback? • What tools do I need in order to assess both the face-to-face and online portions of the hybrid activity?

16. Small Group Discussion • In your breakout rooms, take 20 minutes to discuss your written reflections. • Give each group member a turn to describe his or her hybrid activity and the alignment, timing, and tools he or she will use to assess it. • Use the notes pad in the breakout room to write down questions, ideas, suggestions, and themes that show up in each of the hybrid activities. • Nominate a presenter to summarize the discussion to the whole group.

17. Small Group Presentations • How do alignment, timing, and tools affect the way you will assess your hybrid activity? • What questions, ideas, suggestions, or themes showed up in multiple hybrid activity plans?

18. Conclusions • Insights or comments on the Adobe Connect tool • Possible uses • Possible limitations • How might your “student” experience affect the way you use this tool?Or the way you introduce it to students? • Homework: • Schedule time to review assessment features of SmartSite • Write instructions for students about your hybrid activity • How it works • Why it works • What to expect from assessment (see following slide)

19. One more best practice: • Information learners should have about the assessment: • Know that an assessment is coming (unless it is intentionally unannounced) • Have some information on how to plan for the assessment • Content • Tools • Know what happens after the assessment • Have an idea of what participating in the assessment will look like • Content • Tools • Have available some practice with the assessment, perhaps via self-assessment