1 / 32

Introduction to Assessment

Introduction to Assessment. Jennifer Lowman Coordinator, Student Persistence Research. Credit for the content of this presentation should go to Marilee Bresciani and her many co-authors; & Gavin Henning & others with Academic Impressions. Outline. Define Assessment

roy
Download Presentation

Introduction to Assessment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Introduction toAssessment Jennifer Lowman Coordinator, Student Persistence Research Credit for the content of this presentation should go to Marilee Bresciani and her many co-authors; & Gavin Henning & others with Academic Impressions

  2. Outline • Define Assessment • Identify different Types of Assessment • Review Cycle of Assessment • Define Outcomes • Writing “Learning Outcomes” • Practical Assessment Techniques • Ethics • Concluding Remarks

  3. What is Assessment? • Assessment, Evaluation, & Research • Assessment • An effort to gather, analyze, and interpret evidence to understand effectiveness and improve programming • Evaluation • Using assessment information to make an informed judgment of program quality or worth • Research • Collection and analysis of data to build or confirm theory or conceptual foundations

  4. Institutional Assessment • Mission • Goals or Objectives • Program, Product, Service Inputs • Outcomes • Improvement or Accountability • Assessment Planning part of Programming • Embed Assessment • Design, Measurement, & Analysis • Reporting & Learning

  5. Embedded Assessment • Make assessment part of your program, event, service, or activity • Decades of research shows that students respond to intentional activities that are linked to positive outcomes (Bresciani, Zelna, & Anderson, 2004). • If you tell students what you expect them to learn and how it will benefit them, they are more likely to learn

  6. Embedded Assessment is… • Continual assessment for continual improvement • Track changes in program impact over time, as well as… • Improve measurement over time • Test new measures, while still using old • Make it a habit – a state of mind!

  7. Culture of Assessment • Build shared trust: Begin by lowering social and interpersonal barriers to change. • Build shared motivation: Collectively identify goals worth working toward and problems worth solving. • Build a shared language: Develop a collective understanding of new concepts needed for transformation. • Build shared guidelines: Develop a short list of research-based guidelines for using assessment to promote learning

  8. Types of Assessment

  9. Types of Assessment Tracking Usage Needs Assessment Satisfaction Study Culture/Climate Assessment Outcomes Assessment Resource Effectiveness Study Benchmarking Program Review Strategic Planning Increasing Complexity & Integration

  10. Forms of Assessment Operational or Descriptive Assessment • Tracking Usage • Needs Assessment • Satisfaction Study • Culture/Climate Assessment Formative Assessment • Outcomes Assessment • Resource Effectiveness Study • Benchmarking Political Assessment • Program Review • Strategic Planning

  11. Types of Assessment? (activity)

  12. Cycle of Assessment

  13. Cycle of Assessment: External Perspective Define issue/problem Identify Goals Align mission & goals Identify stakeholders Identify theoretical/conceptual framework Develop measurable outcomes from goals Identify and measure inputs Classify strategies from framework to reach goals Isolate action steps to implement strategies 10. Develop formative assessment plan to understand process 11. Measure Outcomes 12. Make sense of results 13. Report / Present findings 14. Review, Reallocate, Advocate for Resources 15. Retool Program 16. Plan for Iteration #2

  14. Cycle of Assessment: Internal Perspective

  15. Outcomes-Based Assessment

  16. Assessment & Planning • Assessment and Planning are linked, intertwined, and interrelated • “Backwards Design” or “Start with the end” • Programs are designed this way too! Program Inputs Desired Outcome

  17. From Goals to Outcomes • Goals/Objectives – end result in broad terms (optimal outcomes) • Outcomes – more specific statements derived from goals that are measurable • Operational Outcomes • Learning Outcomes • Program Outcomes

  18. Learning v. Program Outcomes • Learning Outcomes (aka Individual Outcomes) – a learning outcome is participant-centered and describes the desired effect of a program on a student’s cognitive or intellectual state. • Statements of what students will value, do, or knowas a result of participation. • ASK: attitude, skills, & knowledge • Program Outcomes (aka Aggregate Outcomes) – the desired aggregate effect of a program.

  19. Program or Learning Outcome? (worksheet)

  20. Learning Outcomes • Participant - centered • Focus on what students can do • Not on what you did! • Purpose to reveal the impact of ‘Action Steps’ • Action Steps are the point of contact • May or may not be perceived by students • Outcomes may need to take variation in contact into account • Variation is key for identifying effectiveness!

  21. Learning Outcomes Assessment • What are students learning and why? • How do we know? • Are students learning what we intend for them to learn? • How do we use assessment information to improve learning? • Do the improvements we make work?

  22. SMART Outcomes • SMART  Specific Measurable Aggressive/Attainable Results-oriented Time-bound

  23. Writing Learning Outcomes • ABCD formula • Audience (Who) • Who is the target? • Behavior (What) • What should the target be able to know, do, value (ASK)? • Condition (How) • What Action Steps will facilitate the learning? • Degree (How much) • How much learning will be demonstrated?

  24. Writing Learning Outcomes Condition + Audience + Behavior + Degree As a result of participating in the event + students + will demonstrate + 3 of 5 skills.

  25. ABC v. ABCD…or why Degree is important! • The degree of impact you expect to have on your students is important to… • Set criterion for success AHEAD of program implementation • Demonstrate Minimal Benefit • Transparency & Accountability • Signal intention to students and stakeholders • Establish standard for data-driven decision-making

  26. Degree of Impact • Independent Indicators (Index) • Inter-changeable, equally important information • Differences between students at start/finish are not import. • Ex. 3 of 5 study habits • Ex. 5 of 7 coping strategies • Continuous Scale • Cumulative or increasing attitude, skill, or knowledge • Expect differences between students at start/finish • Increased leadership skills from baseline eval to next level (basic to intermed.; intermed. to adv.) • Process • Student must master step 1 to move onto step 2…etc. • Describe steps in process within LO • Break process in to multiple LOs, but avoid yes/no outcomes

  27. Degree of Impact ≠ Depth of Learning

  28. Degree of Impact & Depth of Learning • Influenced by the opportunity provided • Exposure • Relevance • Immediacy • Salience • Feedback • … • L.O. reflects depth of learning in the “Action Verb”

  29. Depth of LearningBloom’s Taxonomy (revised) • Remembering: recognizing, recalling, labeling • Understanding: interpreting, summarizing, contrasting • Applying: executing, implementing, classifying • Analyzing: differentiating, organizing, attributing • Evaluating: critiquing, integrating, generalizing • Creating: generating, planning, producing (teaching)

  30. List five ways identified in the readings that can help you reduce stress? (Remembering) Based on the floor meeting, discuss at least three reasons why we don’t allow alcohol in the residence halls? (Understanding) How can you use what you learned in these activities in your student organization? (Application) After participating in the retreat, compare the effectiveness of leadership tactics? (Analyzing) As you reflect on this past year as I have served as your organization’s advisor, in what areas have I been most effective and in what areas can I improve (Evaluating) Based on our conversation regarding career action steps, recommend four action steps to each of your group members? (Creating) The Depth of Learning expected is evident in the Action Verb!

  31. Action Verbs (Observable Behaviors) • Don’t Use • Appreciate • Become aware • Be familiar with • Know • Learn • Understand • Higher-Level • Solve, Apply • Analyze, Explain • Infer, Integrate • Plan, Create • Design, Generalize • Assess, Rank • Recommend • Compare, Teach Lower-Level List, Label, Name Define, Describe Identify, Interpret Contrast, Discuss Classify, Complete Relate, Modify Demonstrate

  32. Practice Writing Learning Outcomes (Activity)

More Related