1 / 21

Evaluation Methods and Findings of the Field-tested Learning Assessment Guide

Evaluation Methods and Findings of the Field-tested Learning Assessment Guide. Sue Daffinrud Olga Lucia Herrera. The Motivation for the FLAG. Motivated by needs of faculty who wanted centralized place to learn about already-verified assessment techniques and tools (Third Annual NISE Forum)

trygg
Download Presentation

Evaluation Methods and Findings of the Field-tested Learning Assessment Guide

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation Methods and Findings of the Field-tested Learning Assessment Guide Sue Daffinrud Olga Lucia Herrera

  2. The Motivation for the FLAG • Motivated by needs of faculty who wanted centralized place to learn about already-verified assessment techniques and tools (Third Annual NISE Forum) • Does the FLAG satisfy part of this need?

  3. FLAG Structure • Five major sections • Getting Started • Matching Goals to CATs • CATs • Searchable Tools • Resources

  4. FLAG target audience • Instructors who: • teach freshman and sophomore-level science, math, and engineering courses and • are interested in learning about new approaches to assessment

  5. FLAG Evaluation • Does the FLAG website fulfill some of the assessment needs of its target audience? • Formative and Summative Evaluation

  6. Identifying target audience sample • Selected subset of SME disciplines • Biology, chemistry, engineering, mathematics, and physics • Selected range of institutions • By Carnegie classification • Associate of Arts, Baccalaureate, Masters, Doctoral, and Research • Region of country

  7. Identifying target audience sample • Selected faculty through their participation in some initiative which has student learning as its focus. • bioQUEST, Worskhop Biology, OCEPT • Chemistry Coalitions • Engineering Coalitions, EESP • Project NExT • Workshop Physics

  8. 50 faculty who have demonstrated an interest in “learning, not teaching”

  9. Contacting participants • Called and emailed faculty requesting their participation • Participation involved: • Browsing through FLAG for 30 minutes • Providing feedback in a 30 minute interview • High participation rate

  10. Interviews • Participants were asked about: • Their current teaching practice • How they learn about new assessment/teaching techniques • Impressions of FLAG, both specific and general • Whether would recommend FLAG to others

  11. Data Gathering Status • Have interviewed 32 faculty • Will interview the remaining 18 in December

  12. Preliminary findings • Demographics of respondents • Technical issues • Themes of Use and Need • Dissemination

  13. Demographics • Range of teaching experience (2-22 yrs) • Critical of the value of lecture • Goals: improving students’ interest in science • Role: facilitator for their learning • Info on assessment gained through conversations with fellow faculty, conferences in their discipline • Few “surf” the web

  14. Technical Issues – Navigation patterns • Participants spent between 30 minutes to 4 hours in the FLAG, most about an hour. • Many moved “left to right”

  15. Technical Issues • Almost all found the web site • Easy to read • Structured well • Could find what they needed • Enjoyed being able to download documents • Suggestions • Provide more obvious description of site structure • Provide better navigation instructions

  16. Themes of Use/Need - Overall • Overall response: Positive • Almost all respondents had already or had plans to recommend the site to other faculty, faculty developers, or other groups

  17. Themes of Use/Need - Sections • Has teaching component • Matching Goals to CATs • Primer • Availability of links to other web sites • Annotated bibliography • References • “Empty” sections create expectation • Searchable tools • “Experts” by state

  18. Themes of Use/Need - CATs • Time is a factor in browsing and in applicability • First page of CATs (why, how, what’s involved) • People selected: • CATs which were familiar to them • CATs which best fit their goals and time

  19. Themes of Use/Need – CATs (cont) • People want a complete picture • Like the case study approach • When and how to use the CAT • What to do with the information • How to get students “on board” • Clear and useful examples • People want “tools” from their discipline to varying degrees

  20. Dissemination of FLAG • Evaluation AS dissemination • Hope to find out how best to reach people like our sample • Discipline-specific professional meetings • Workshops • People on campus who do assessment • Email to appropriate email lists

  21. Final Notes • Submitted abstract for presentation at AAHE 2000 • Write monograph or paper • Provide more formative information • to Tony • To ILT group as they develop site

More Related