Assessment of High Impact Practices: Student Learning Portfolios and Other Tools

1. Assessment of High Impact Practices: Student Learning Portfolios and Other Tools Monica A. Devanas devanas@rutgers.edu http://CTAAR.rutgers.edu

2. AAC&U, LEAP, HIPs, VALUE Rubrics American Association of Colleges and Universities Liberal Education and America’s Promise – 2005 21st Century Liberal Education Essential Learning Outcomes High Impact Practices Authentic Assessment - Valid Assessment of Learning in Undergraduate Education = VALUE Rubrics

3. High Impact Practices What? techniques and curricular designs for teaching and learning Why? proven to bebeneficial for student engagement and successful learning students from many backgrounds How? intentional program design and pedagogy Why? enhance student learning narrow gaps in achievement across populations

4. High Impact Practices • First-Year Seminars and Experiences • Common Intellectual Experiences • Learning Communities • Writing-Intensive Courses • Collaborative Assignments and Projects • Undergraduate Research • Service Learning, Community-Based Learning • Diversity/Global Learning • Internships • Capstone Courses and Projects

5. Bloom’s Ranking of Thinking Skills Know->Comprehend->Apply->Analyze->Synthesize->Evaluate Design Hypothesize Support Schematize Write Report Justify Evaluate Choose Estimate Judge Defend Criticize Summarize Explain Interpret Describe Compare Paraphrase Differentiate Demonstrate Classify List Name Identify Show Define Recognize Recall State Visualize Analyze Organize Deduce Contrast Compare Distinguish Discuss Plan, Devise Solve Illustrate Calculate Use Interpret Relate Manipulate Apply Modify

6. Assessment of Learning in HIP What is the goal of High Impact Practice? What is the context of student learning? What assignments will be used to grade? How will these assessments give feedback on progress of student? … of the program?

7. Assessment Vocabulary • Formative – ongoing feedback to improve learning, areas for improvement “low stakes” few points, but frequentEx: first draft, questions on reading, concept map • Summative – evaluate learning for final grade compare to standards or benchmark “high stakes” high point valueEx: midterm, final, final project, paper, performance

8. Assessment Vocabulary Qualitative - data fits interpretive criteria, not easily analyzed by quantitative methods Quantitative – numerical data that can be analyzed with statistical methods

9. Assessment Vocabulary Direct – students demonstrate knowledge, skills, attitudes, data measures achievement of expected outcomes Indirect – students reflect on knowledge, skills, attitudes, give their opinion, easy to administer, focus on attitudes, values, beliefs

10. Direct Methods of Assessment Pre and posttests Course-embed assessment(homework assignment, essay, locally developed tests) Comprehensive exams National Major Field Achievement Tests GRE subject exams Certification & licensure exams Grading criteria or rubrics (QUANTITATIVE) Portfolio evaluation Case studies Reflective journals Capstone projects Juried performance, exhibits Internship, clinical evaluation External examiners Peer review Senior thesis Major project (QUALITATIVE)

11. Direct Methods of Assessment Pre and posttests Course-embed assessment(homework assignment, essay, locally developed tests) Comprehensive exams National Major Field Achievement Tests GRE subject exams Certification & licensure exams Grading criteria or rubrics (QUANTITATIVE) Portfolio evaluation Case studies Reflective journals Capstone projects Juried performance, exhibits Internship, clinical evaluation External examiners Peer review Senior thesis Major project (QUALITATIVE)

12. Indirect Methods of Assessment • Departmental survey • Student satisfaction surveys • Self-reported gains • Exit interviews • Alumni survey • Employer survey • Focus groups • Job placement statistics • Percentage of students who study abroad • Graduation and retention rates

13. High Impact Practices Documentation: assignments, reports, essays Collaboration: instructors, peers, advisors Reflection: can student see “impact” on learning, on skill development, on beliefs, attitudes and values?

14. Components of Learning Portfolio Learning Portfolio: focus process of learning Documentation = products for evaluation Collaboration = mentor to understand process of reflection and documentation Reflection = think about process of learning, metacognition, thinking about thinking

15. Components of Learning Portfolio Learning Portfolio: focus process of learning Zubezarreta’s “order” effect of planning Reflection = think about process of learning, metacognition, thinking about thinking Documentation = products for evaluation Collaboration = mentor to understand process of reflection and documentation

16. Documentation • Organization, selectivity driven by purpose • Course, activity, program, curriculum • Concise reflective narrative on documents • Relevance, impact on learning, applications • Selected “evidence” described in narrative and link in appendix • OpenLab, web-based, paper-based

17. Mentor is Important • “Collaborative” Mentor: to keep focus, manageable, current, accurate, organized, relevant, time on task, deadlines, coach • Need some objective review • Peer mentor • Advisor • Draft reader, editor • Portfolio “coach”

18. Size, Scope of Learning Portfolio Represent one set of assignments and activities from one course OR experiential learning activity or over a longer period of time, core courses courses in major undergraduate curriculum

19. Student Portfolio = Learning Portfolio without reflection Even process of collecting artifacts has value Students see evidence of significant learning • connections between learning experiences, i.e. in courses, in co-curricular activities. • Valuable as summation of work in learning in some context as a product Useful for assessment of course, program

20. Flavors of Portfolios Student Portfolio: Product – collection of student work in some context Learning Portfolio: Product AND PROCESS reflective learning = process of reflection

21. On Learning Portfolios Zubizarreta… “organized documentation of growth and achievement that provides tangible evidence of the attainment of professional knowledge, skills and disposition. Each portfolios is goal-driven, original, and reflective. The Learning Portfolio value lies in student reflection, construct the notion of learning as a coherent, unified, developmental process, that it is active and lifelong.”

22. Students ask, "Do e-Portfolios Have To Be "E"? Fundamental First Steps For Successful Learning Portfolios. http://eportfolioca.org/index.php?option=com_content&view=article&id=141&Itemid=159 The E-Portfolio Forum: Deepening High Impact Learning Sessions January 29, 2011 - San Francisco, CA

23. Assessment of High Impact Practices Assortment of Assessments Methods Formative – Summative Qualitative – Quantitative Direct – Indirect Balance and Variety

24. Assessment of High Impact Practices Assortment of Assessments Methods Balance and Variety Difficult to Assess Activities Diversity/Global Learning Internships Capstone Courses and Projects Student Portfolios – Learning Portfolios

25. Portfolios at City Tech Student Portfolios - Reflection with Portfolios • Utilized work from open learning environment • Link to past to represent what you have developed through the college years • Binding evidence from past, alternately reflect back on how past work informed current work, has past work driven interest • Is there maturity over time / enrichment • Reflection what student learned through guidance Karen Goodlad, notes from 12/2/13

26. High Impact Practices Goals important for assessment Kinds of assessment Student Portfolios, Learning Portfolios useful for student learning and assessment

27. ~~Wrap UP~~ Closing Ideas from Participants Questions?? THANK YOU