1 / 44

Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments

Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments. Dr. Blase Scarnati Chair, Liberal Studies Committee Associate Professor, Musicology, School of Music

RexAlvis
Download Presentation

Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Conducting an Assessment Audit of General Education Courses for Purposes of Identifying Embedded Assignments Dr. Blase ScarnatiChair, Liberal Studies CommitteeAssociate Professor, Musicology, School of Music Dr. Cynthia ConnAssociate Director, Office of Academic AssessmentClinical Instructor, Educational Technology Program Dr. Bruce FoxChair, Liberal Studies Committee Assessment SubcommitteeDirector, Honors Program

  2. Agenda • Overview • Definition of Skills • Pilot Study • Content Analysis • Group Discussion: Ideas, Projects, and Challenges

  3. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Northern Arizona University • Flagstaff, Arizona • State supported • Doctoral intensive • Significant distance learning

  4. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Northern Arizona University • 19,000 total students • Approximately 6,000 graduate students (3,800 FTE) • Approximately 5,500 students statewide

  5. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Northern Arizona University • Approximately 11,400 undergraduates on the Flagstaff campus • Students come to us in a variety of ways

  6. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Liberal Studies Program • Approved Spring 1998 • Fully implemented Fall 1999

  7. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Citizenship • Coherent program • Essential skills necessary for citizenship • Key themes

  8. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Liberal Studies Program Requirements • 35 credit total requirement • Foundation requirements • Distribution courses • Courses embedded within the academic major

  9. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Foundation Requirements • English composition course (4 credits) • Mathematics foundation course (3 credits)

  10. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Distribution Requirements (25 Credits) • Aesthetic and Humanistic Inquiry (3-6 credits) • Cultural Understanding (3-6 credits) • Laboratory Science (4 credits) • Science and Applied Science (3-6 credits) • Social and Political Worlds (3-6 credits)

  11. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Thematic Foci • Environmental Consciousness • Technology and Its Impact • Valuing the Diversity of Human Experience

  12. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Essential Skills • Effective oral communication • Critical reading • Effective written communication • Quantitative analysis and/or spatial reasoning • Creative thinking • Critical thinking • Scientific inquiry • Ethical reasoning • Effective use of technology

  13. Agenda Overview Skills Pilot Study Content Analysis Group Discussion University Requirements • Courses embedded within the academic major • Diversity requirement • Required freshman colloquium

  14. Agenda Overview Skills Pilot Study Content Analysis Group Discussion In the Academic Major • Junior level writing course (3-4 credits) • Senior capstone experience (3 credits minimum)

  15. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Diversity Requirement • U.S. Ethnic Diversity (3 credits) • Global Awareness (3 credits)

  16. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Required Freshman Colloquium (3 Credits) • Discontinued effective Fall 2004 • Redesigned

  17. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Liberal Studies Committee (LSC) • Faculty body • Six colleges • Student representative • Faculty Senate representative • Administrative and programmatic representatives

  18. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Assessment • Liberal Studies Committee Assessment Subcommittee • Liberal Studies Program Assessment Plan (Fall 2004)

  19. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Assessment Plan Goals (1) • Assess the achievement of NAU students • Effective writing • Effective oral communication • Critical reading

  20. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Assessment Plan Goals (2) • Create a more cohesive curriculum of study • Analytical procedures • Inform adjustments to the curriculum

  21. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Research Question 1 • To what extent are courses and embedded assignments that state they teach effective writing, critical reading, and/or effective oral communication aligned with the outcomes defined during the 2003-2004 academic year by the Liberal Studies Committee?

  22. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Research Question 2 • How should issues of non-alignment between courses and defined outcomes be addressed?

  23. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Research Question 3 • What expectations should be established for course content, assessment, and the broader curriculum?

  24. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Research Question 3 • Specifically, to enhance the cohesiveness of the curriculum, what changes should be made to the existing expectations (for faculty, departments, etc.) for submission and acceptance of courses into the Liberal Studies Program?

  25. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Assessment Audit • Courses and embedded assignments • Aligned with the outcomes • Pilot Project

  26. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Defining Skills • Working definitions created by LSC subcommittee • Working definitions tentatively approved by entire LSC • Definitions “vetted” through consultation with internal constituents, primarily departments • Consensus definitions approved by LSC

  27. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Defining Skills Agenda Overview Skills Pilot Study Content Analysis Group Discussion • Advantages: • Create buy in from other campus units • Liberal Studies process seen as open and participative • Better definitions! • Key strategy: visit folks personally. Do NOT rely solely on electronic communication

  28. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Effective Writing • Students will be able to: • Demonstrate steps in the writing process • Tailor writing to a specific audience • Focus writing on a specific purpose • Provide logically coherent pieces of written work • Apply general writing standards

  29. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Critical Reading • Students will be able to: • Summarize accurately and comprehensively • Recognize the most significant textual and visual elements • Incorporate/use specific examples and/or quotations • Recognize patterns and valid comparatives • Demonstrate an understanding/comprehension of the material • Demonstrate evidence of an ability to analyze and evaluate texts • Demonstrate the ability to explicitly identify criteria and use these criteria accurately to evaluate a piece of written work

  30. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Oral Communication • Students will be able to: • Organize and deliver content based on audience, location, allotted time, and purpose • Speak with appropriate oral communications techniques • Listen and respond thoroughly and thoughtfully to questions • Create appropriate professional supplemental materials that reinforce the presentation

  31. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Quantitative & Spatial Analysis Agenda Overview Skills Pilot Study Content Analysis Group Discussion (aka “Methods of Formal Analysis”) (aka “Quantitative Literacy”) Suggested titles: More inclusive?

  32. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Methods of Formal Analysis Agenda Overview Skills Pilot Study Content Analysis Group Discussion • Application of numerical, visual or symbolic perspectives for the purpose of drawing inferences, understand phenomena or making predictions in situations that involve natural phenomena or real-world decision making

  33. Pilot Project Agenda Overview Skills Pilot Study Content Analysis Group Discussion • Testing of Protocols • ITERATIVE • Foundation Principles • Based on embedded assignments • Actively involves faculty • Supplements direct evidence with indirect date sources

  34. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Pilot Project • Process • Identify faculty teaching courses that include desired skill • Recruit these faculty as sources of embedded assignments • Use faculty consultants to develop and evaluate rubrics • Use rubrics to assess level of skill mastery

  35. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Pilot Project • Process • Collect assessment data for a stratified sample of courses • Compile and analyze data • Triangulate using indirect data sources • Evaluate protocols

  36. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Pilot Project • Informing and Involving the Wider Campus Community • Communications Plan Strategy to inform entire campus community of assessment process in a deliberate and on-going manner • Create a Steering Committee of on and off campus constituencies

  37. Agenda Overview Skills Pilot Study Content Analysis Small Discussion Full Group Discussion Agenda Overview Skills Pilot Study Content Analysis Group Discussion Content Analysis • Purpose • Identify the alignment between Liberal Studies skills as they are now defined and syllabi of current Liberal Studies courses. • Identify embedded assignments that could be used for assessment purposes

  38. Agenda Overview Skills Pilot Study Content Analysis Group Discussion Content Analysis • How will findings be used? • Identify potential faculty for participation as consultants for Pilot Study • Target courses with strongest alignment for first phase of data collection • Recommend policy changes regarding content of syllabi and expectations related to assessment

  39. Group Discussion Agenda Overview Skills Pilot Study Content Analysis Group Discussion • Ideas: What works well at your institution with respect to assessing General Education student learning outcomes? • Challenges: What challenges concerning General Education and Assessment do you face at your institution?

  40. Agenda Overview Skills Pilot Study Content Analysis Small Discussion Full Group Discussion References • Walvoord, Barbara E. (2004). Assessment Clear and Simple : A Practical Guide for Institutions, Departments, and General Education. San Francisco: Jossey-Bass. • Neuendorf, Kimberly A. (2002). The Content Analysis Guidebook. Thousand Oaks: Sage Publications.

  41. Contact Information Blase Scarnati, Ph.D. Chair, NAU Liberal Studies CommitteeAssociate Professor, School of MusicP.O. Box 6040Flagstaff, Arizona 86011 Blase.Scarnati@nau.edu (928) 523-6537 Cynthia Conn, Ph.D. Associate Director, Office of Academic AssessmentClinical Instructor, Educational Technology ProgramP.O. Box 4091Flagstaff, Arizona 86011 Cynthia.Conn@nau.edu (928) 523-7624 Bruce Fox, Ph.D. Chair, Liberal Studies Committee Assessment Subcommittee Director, Honors ProgramP.O. Box 5689Flagstaff, Arizona 86011 Bruce.Fox@nau.edu (928) 523-6636

More Related