1 / 37

Why I don’t understand the assessment results

Why I don’t understand the assessment results. (and what I can do about it). Hi…I’m the Assessment Director. Reasons for Assessment. Accreditors demand it Institutional leaders ought to care Faculty members probably do care Students should know what we think of them. The Theory.

coye
Download Presentation

Why I don’t understand the assessment results

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Why I don’t understand the assessment results (and what I can do about it)

  2. Hi…I’m the Assessment Director

  3. Reasons for Assessment • Accreditors demand it • Institutional leaders ought to care • Faculty members probably do care • Students should know what we think of them

  4. The Theory

  5. What Could Go Wrong???

  6. Take Action: Now what?

  7. Take Action: Now What?

  8. Assessment: The Process • Design • Implementation • Reporting • Follow-up

  9. Design

  10. Design: Validity and Reliability The emphasis [on validity] is not on the instrument itself; rather, it is on the interpretation of the scores yielded by a test. -- College BASE Technical Manual The CLA measures were designed by nationally recognized experts in psychometrics and assessment, and field tested in order to ensure the highest levels of validity and reliability. -- Collegiate Learning Assessment Advertisement

  11. Design: Epistemology We say that a sentence is factually significant to any given person, if and only if, [she or] he knows how to verify the proposition which it purports to express—that is, if [she or] he knows what observations would lead [her or him], under certain conditions, to accept the proposition as being true, or reject it as being false. – A. J. Ayer, Language, Truth, and Logic [T]he meaning of a word is its usage in the language. – L. Wittgenstein

  12. Design: Complexity and Validity • Collegiate learning is complex • Assessment should be: • Authentic • Contextual • Rigorous • Reliable • Understandable

  13. Design: Measurement Smeasurement • Measurement requires: • Units • Ability to aggregate • What we actually do is estimation

  14. Analysis

  15. Analysis: Using the Data Pieper, Fulcher, and Erwin – • Differences • Relationships • Change • Competency

  16. Analysis: Tools • MS Access • Spreadsheet software • Pivot Tables • Statistics packages: • Excel • SPSS • SAS • Logistic Regression:http://statpages.org/logistic.html

  17. Analysis: Averages

  18. Analysis: Proportions

  19. Analysis: Example

  20. Analyze: Example

  21. Analysis: Example

  22. Analysis: Example

  23. Analysis: Example

  24. Analysis: Example

  25. Analysis: Example

  26. Analysis: Example CIRP Self-Assessed Mathematical ability

  27. Analysis: Example

  28. Analysis: Example

  29. Analysis: Example

  30. Analysis: Example

  31. Analysis: Example

  32. Analysis: Example

  33. Analysis: Example

  34. Analysis: Example

  35. Analysis: Example

  36. Conclusions • At strategic level, averages are fine • At tactical level, proportions are easier to act on • Abstraction is the enemy • Don’t sacrifice validity for reliability without a good reason • For complex skills, subjectivity is your friend

  37. Last Requests? highered.blogspot.com

More Related