1 / 38

ASSESSMENT AS SCHOLARSHIP: IMPROVING PROGRAMS, INSTITUTIONS, AND DISCIPLINES

ASSESSMENT AS SCHOLARSHIP: IMPROVING PROGRAMS, INSTITUTIONS, AND DISCIPLINES. Thomas P. Pusateri, Ph.D. Florida Atlantic University pusateri@fau.edu Closing Plenary Session 13th Annual Georgia Conference on College & University Teaching March 25, 2006. RESOURCES FROM JOSSEY-BASS. 1990

base
Download Presentation

ASSESSMENT AS SCHOLARSHIP: IMPROVING PROGRAMS, INSTITUTIONS, AND DISCIPLINES

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ASSESSMENT AS SCHOLARSHIP:IMPROVING PROGRAMS, INSTITUTIONS, AND DISCIPLINES Thomas P. Pusateri, Ph.D.Florida Atlantic Universitypusateri@fau.edu Closing Plenary Session 13th Annual Georgia Conference on College & University TeachingMarch 25, 2006

  2. RESOURCES FROM JOSSEY-BASS 1990 ScholarshipReconsidered (Boyer) 1997 ScholarshipAssessed (Glassick, Huber, & Maeroff 2001 ScholarshipRevisited (Kreber) 2004 AssessmentClear and Simple (Walvoord)

  3. ASSESSMENT IN THE NEWSThe Atlantic Monthly (November 2005) • How College Affects Students (Pascarella & Terenzini)Going to college makes a difference But no differences among colleges, controlling for student quality • Hersh: Current assessment measures may not be picking up differences • Who will control the agenda for assessing the worth of college? U.S. Congress: Reauthorization of the Higher Education Act 40 State legislators: FL Board of Governors–Academic Learning CompactsBush administration: Commission on the Future of Higher Education • Hersh: Institutions ought to take charge to assess cumulative learning Formative assessment: Carleton College (MN); Alverno College (WI) Value-added assessment: Wabash College (IN) study of 16 institutions

  4. ASSESSMENT AS SCHOLARSHIP • This presentation focuses on assessment for program improvement • Accountability: Track individual students • Program improvement: Aggregate the data • Assessment is “action research” for “informed intervention” • Models from social science and educationSampling; Pre-post differences; Comparisons • Quantitative and Qualitative data

  5. GOALS OF THISPRESENTATION • Provide practical examples of assessment that you might adapt at your institution • Examples from Loras College and FAU • Suggest strategies for communicating findings to colleagues (“Lessons learned”) • Graphing data; Conducting/Reporting audits • Identify venues for presenting and publishing research on assessment • SoTL organizations, conferences, journals

  6. EXAMPLE 1MEASURING STUDENT ATTITUDES Loras College: Attitudes towards general education Faculty developed 41 outcomes statements“Loras graduates should be able to….” (“understand” or “appreciate” disciplines) Survey requested ratings on a 5-point scale from “Extremely important” to “No importance” Most students completed the survey twiceEarly in the first year and late in the senior year Comparison groups Longitudinal (First year – Senior) comparisonsValue added approach: Did students’ values change? Comparison groups: FTICs to Transfer studentsDid completing four years at Loras matter?

  7. LESSON 1MEANS CAN BE MEANINGLESS • Recommendation: Graph frequency distributions • In Microsoft Excel, choose “100% Stacked Bar”

  8. Based on 380 students who completed both versions of the survey.

  9. Based on 441 non-transfers and 43 transfer students.

  10. Differences in FTIC & Transfers

  11. Differences in 1st year & Senior year

  12. EXAMPLE 2ASSESSING WRITING SKILLS Loras College: Sophomore writing portfolios • Phase 1: Assessment committee (2 summers) • Collected portfolios (3 papers from each student) • Read a subset of portfolios; wrote a scoring rubric • Evaluated remaining portfolios (interrater reliability) • Provided students feedback • Phase 2: Faculty workshops (2 summers) • Training on using the scoring rubric • Evaluating portfolios (interrater reliability) • Discussing findings and applying to courses • Provided students feedback

  13. Based on assessment by 21 faculty who read a subset of 49 student portfolios

  14. LESSON 2MISSED OPPORTUNITIES Carleton College (MN): AAC&U News (December 2004) Sophomore writing portfolio requirement Students submit papers from at least 2 of 4 divisions Faculty ratings: “Exemplary” “Pass” “Needs work” 14% 78% 8% (resubmit) Evaluation criteria: KEY WRITING SKILLSQUALITY OF WRITINGReport on observationAttend to audienceAnalyze complex informationClarity of proseProvide interpretationClear organizationUse and document sourcesUse of evidenceArticulate and support thesisDistinctive voiceAppropriate dictionControl of error

  15. LESSON 2MISSED OPPORTUNITIES REVISITED University of South Carolina – Aiken “Communicating Assessment Results Online to PromoteCurricular Change” at SACS-COC 2005 Annual Meeting Rising Junior portfolio WRITING OUTCOMES/EVALUATIVE RUBRIC Clarity of Purpose Quality of Thought Organization of Content Use of Sources Language and StyleGrammar and Mechanics

  16. LESSON 2MISSED OPPORTUNITIES RECLAIMED? CARLETON COLLEGE LORAS COLLEGE USC AIKEN Attend to audience Creativity/Voice/Audience Clarity of purposeand purpose Distinctive voice Clarity of prose Demonstrates critical Quality of thought thinking Clear organization Organizes the writing Organization of content Use of evidence Supports ideas with Use of sourcesevidence Appropriate diction Uses standard English Language and style Control of error and effective prose Grammar and mechanics

  17. EXAMPLE 3USING STANDARDIZED TESTS • Loras College: Assessment of General EducationCAAP Tests in selected sophomore-level courses • Reading • Essay Writing • Science Reasoning • Mathematics • Loras College: Senior Majors in Psychology • ETS Major Field Test in Psychology

  18. LESSON 3STANDARDIZED TESTS CAN BE USEFUL(IF SELECTED & USED APPROPRIATELY) • Comparisons to other institutions • Essay writing: Provided additional evidence for faculty concerns • Mathematics: Supported the math department’s recommendation for a curriculum change • Science reasoning & Reading: Added little of valuePsychology: Provided useful information concerning the quality of content knowledge among majors

  19. EXAMPLE 4FLORIDA’S ACADEMIC LEARNING COMPACTS • 2004 Florida Board of Governors resolution to implement Academic Learning Compacts for each baccalaureate degree program at 11 universities • Discipline-specific knowledge/skills • Communication skills • Critical thinking skills • Each baccalaureate program must also • Identify where and how students are assessed • Provide students access to this information • Monitor performance for program improvement

  20. Implementing Academic Learning Compacts • Review of past assessment plans • Develop flexible definitions of outcomes • Assist departments to • Identify discipline-specific outcomes • Identify location of student assessments • Courses (Core requirements; Gateway; Capstone)[Shared learning outcomes/syllabi/assignments] • Standardized examinations for some programs • Develop methods to track student achievement • Embedded questions on examinations • Scoring rubrics on shared assignments • Performance on standardized examinations

  21. SCHOLARSHIP OF ASSESSMENT • Presentations at conferences • Using Psychological Expertise in College-Wide Assessment and Departmental Program Review presented at MACTOP ’97 • Assessing Classes, Courses, and Curriculapresented at MACTOP ’02, MISTOP ’01, ITOP ’00 • Aligning Assessment for Program Improvement with Accountability for Individual Student Learningpresented at SACS ’05; NCSU ’06 Assessment Institute • Publications • Designing and Implementing Psychology Program Reviews (2004 Measuring Up book chapter) • A Decade of Changes since the St. Mary’s Conference (2002 Teaching of Psychology interview)

  22. DISCIPLINE-BASED JOURNALS ON TEACHING PEDAGOGY Accounting:Issues in Accounting Education Anthropology:Anthropology and Education Art:Art Education Biology:American Biology Teacher Business:Business Education Chemistry:Journal of Chemical Education Communication:Communication Education Computer Science:Computer Science Education Economics:The Journal of Economic Education Education:Journal of Teacher Education English:Research in the Teaching of English Finance:Journal of Financial Education Geography:Journal of Geography in Higher Education Geology: Journal of Geological Education History:Teaching History - A Journal of Methods International Business:Journal of Teaching in International Business

  23. DISCIPLINE-BASED JOURNALS ON TEACHING PEDAGOGY Journalism: Journalism Educator Management:Journal of Management Education Marketing:Journal of Marketing Education Mathematics:Journal for Research in Mathematics Education Modern Languages:Modern Language Journal Music:Journal of Research in Music Education Nursing:Journal of Nursing Education Philosophy:Teaching Philosophy Physics:Physics Teacher Political Science:Political Science Teacher; Teaching Political Science Psychology:Teaching of Psychology Social Work:Journal of Teaching Social Work Sociology:Teaching Sociology Statistics:The American Statistician Theater:Theater Topics Women's Education:The Feminist Teacher

More Related