The Promise and Challenges of Portfolio Assessment

1. The Promise and Challenges of Portfolio Assessment Dave Dempsey, Professor of Meteorology (dempsey@sfsu.edu) Feb 23, 2009

2. Department of Geosciences, SFSU • Graduate Program • M.S. (Geosciences) • Undergraduate Programs • Geology (B.S.) • Meteorology/Oceanography (B.S.) • Earth Science (B.A.) (new) • Faculty • Geology (~ 6) • Meteorology (~ 3) • Oceanography (2+) • Geology Students • B.S.: ~3-10 grads/yr

3. Assessment & Evaluationof SFSU’s Geology Program A Brief Objective History: • 2002-2003: Had to come up with a plan. (Ugh) • Subcommittee drafted learning objectives and performance criteria (We liked that--fun!) • Chose portfolios for assessment (Good on paper) • Faculty matched assignments with objectives (Easy) • 2003-2005: Some partial student portfolios assembled (Herding cats?) • Two people created rubric (Good for them!) • Three faculty tried to evaluate portfolios (Hard!) • The End

4. Portfolios: The Promise • Good for showing student work • can see learning develop over time • can include maps, photos, writing easily • Useful to student for job hunt • (See literature for more)

5. Portfolios: The Challenge I. For Assessment (Gathering Data) • Problem:Must get assignments (etc.) into portfolio over 2-4+ years and not lose it. • Our “solution”:Students give instructor-designated assignments to advisors, who maintain portfolios.

6. Portfolios: The Challenge • For Assessment (Gathering Data) • Pitfalls: • instructors forgot to remind students to keep assignments for portfolio • students didn’t keep assignments • students forgot to bring them to advisor • advisors didn’t maintain portfolios • Death knell: Department stopped enforcing mandatory advising

7. Portfolios: The Challenge • For Evaluation • Portfolios incomplete • Learning objectives & performance criteria seldom explicit in assignments, syllabi

8. Portfolios: The Challenge • For Evaluation • Evaluators unsure what to evaluate: • Did assignment address learning objective?(Only need one portfolio for that!) • Did student work meet instructor’s expectations? (Assignment grade does that!) • Was instructor grade appropriate? (Academic freedom an issue? What value do evaluators add ?

9. Roots of Our Problems • Faculty participation, buy-in, and communication incomplete throughout • Mandatory assessment broke down • Point of portfolio evaluation unclear

10. Another Try: Some Ideas • Involve all faculty in learning objective development/revision (underway); require that objectives be explicit in course materials, communicated to students. • Assign responsibility for maintaining portfolios to the students. (Students would own their portfolios, a job marketing tool.) • Hold students accountable for maintaining them. (Make it a graduation requirement?)

11. Another Try: Some Ideas • Reintroduce & enforce mandatory advising. • Assign responsibility for advising to one faculty member per semester, and give that person release time to do the work. (Advisor monitors student portfolio maintainance.)

12. Another Try: Some Ideas • Exit interview as part of senior capstone or seminar course: • Student meets semi-formally with interview committee (2-3 faculty, rotating task). • Faculty pose problem or simulate job or grad school interview. • Student responds, using portfolio to illustrate what/where/how learned relevant stuff • Faculty score response and/or portfolio against performance criteria on Likert scale • Capstone/seminar instructor meets with committee, summarizes & reports results

13. Another Try: Bottom Line • Get faculty and student buy-in and participation through ownership of process (and a little pressure/persuasion) • Hold faculty and students accountable • Devote a few resources for some of the work • Keep the process visible Make assessment/evaluation an expectation and a regular part of Departmental culture