Project Evaluation Don Millard John Yu dmillard@nsf zyu@nsf March 27, 2012

1. Project Evaluation Don Millard John Yu dmillard@nsf.govzyu@nsf.gov March 27, 2012 Guy-Alain Amoussou Lou Everett gamousso@nsf.govleverett@nsf.gov March 28, 2012 Handout 1

2. Important Notes • Most of the information presented in this workshop represents the presenters’ opinions and not an official NSF position • Local facilitators will provide the link to the workshop slides at the completion of the webinar. • Participants may ask questions by “raising their virtual hand” during a question session. We will call on selected sites and enable their microphone so that the question can be asked. • Responses will be collected from a few sites at the end of each Exercise. At the start of the Exercise, we will identify these sites in the Chat Box and then call on them one at a time to provide their responses.

3. Framework for the Session • Learning must build on prior knowledge • Some knowledge correct • Some knowledge incorrect – Misconceptions • Learning is • Connecting new knowledge to prior knowledge • Correcting misconceptions • Learning requires engagement • Actively recalling prior knowledge • Sharing new knowledge • Forming a new understanding

4. Preliminary CommentsActive & Collaborative Learning • Effective learning activities • Recall prior knowledge -- actively, explicitly • Connect new concepts to existing ones • Challenge and alter misconceptions • Active & collaborative processes • Think individually • Share with partner • Reportto local and virtual groups • Learn from program directors’ responses

5. Facilitator’s Duties • Coordinate the local activities • Watch the time • Allow for think, share, and report phases • Reconvene on time -- 1 min warning • With one minute warning, check Chat Box to see if you will be asked for a response • Ensure the individual think phase is devoted to thinking and not talking • Coordinate the asking of questions by local participants and reporting local responses to exercises

6. Goal for Project Evaluation Session The session will enable you to collaborate more effectively with evaluation experts in preparing credible and comprehensive project evaluation plans…. it will not make you an evaluation expert.

7. Session Outcomes After the session, participants should be able to: • Discuss the importance of goals, outcomes, and questions in the evaluation process • Cognitive and affective outcomes • Describe several types of evaluation tools • Advantages, limitations, and appropriateness • Discuss data interpretation issues • Variability, alternative explanations • Develop an evaluation plan in collaboration with an evaluator • Outline a first draft of an evaluation plan

8. Evaluation and Assessment • The terms evaluation and assessment have many meanings • One definition • Assessment is gathering evidence • Evaluation is interpreting data and making value judgments • Examples of evaluation and assessment • Individual’s performance (grading) • Program’s effectiveness (ABET and regional accreditation) • Project’s progress and success (monitoring and validating) • Session addresses project evaluation • May involve evaluating individual and group performance – but in the context of the project • Project evaluation • Formative – monitoring progress to improve approach • Summative – characterizing and documenting final accomplishments

9. Activity Evaluation vs AssessmentFormative vs Summative • Think about your favorite course. What types of in-class activities could be called: • Assessment versus Evaluation • Formative versus Summative Evaluation • Exercise ---- 6 min • Think individually -------- ~2 min • Share with a partner ----- ~2 min • Report in local group ---- ~2 min • Watch time and reconvene after 6 min • Use THINK time to think – no discussion, Selected local facilitators report to virtual group • With one minute warning, check Chat Box to see if you will be asked for a response