1 / 18

Assessment & Evaluation

Assessment & Evaluation. NSF CISE REU Sites PI Meeting March 18, 2011. Project Supporters for 2010 Implementation. 2009 Working Group Members

oprah-cline
Download Presentation

Assessment & Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessment & Evaluation NSF CISE REU Sites PI Meeting March 18, 2011

  2. Project Supporters for 2010 Implementation 2009 Working Group Members Guy Alain Amousou Chris Aberson Wendy Cooper Teresa Dahlberg Andy Fagg Stephen Gilbert Manfred Huber Niels Lobo Sanjay Madrias Joan Peckham Eric Wong Yu-Dong Yoa Kevin Zeng 2010 Implementation Members Brooklyn College, CUNY Iowa State University University of Alabama UNC Charlotte Colorado Springs Jackson State University University of Central Florida University of South Carolina Dakota State University Louisiana State University University of Texas Austin Depaul University Montclaire State University University of Houston University of Wisconsin Oshkosh Depauw University Marshall University University of Illinois Urbana-Champaign Hope College Oklahoma University University of Massachusetts Amherst 13 used Common Application 20 used Survey

  3. 2010 Assessment Project • Common Application • a la carte Survey • Evaluation Toolkit • Coitweb.uncc.edu/reu/toolkit • Future Directions • Refinements • Longitudinal Follow up

  4. Common Application 2010 • 13 sites utilized • Total of 1,006 applicants • Most students applied to only 1 • One student applied to 30 • 2 to 3 schools was common • Application Range: 29-152 • Average Number of Applications per Site: 77 • ~ 130 accepted = 18% approximate acceptance rate

  5. Unique Applicant Gender Males: 500 Females: 187 Unspecified: 10

  6. Unique Applicant Ethnicity 68 124 52 4 426 14 9

  7. Applicant Level in School 86 191 294 123

  8. Degrees Considering Degree most interested in pursuing Masters programs: 479 (69%) Phd programs: 180 (26%) Undecided: 3 (<1%) 31 (5%) not considering graduate programs

  9. A la Carte Survey 2010 • Pre survey: May-June 2010 • Post survey: July-August 2010 • Total N: 339 REU Students; 20 Sites Pre Demographics (N=197): Gender- 69% Male (135), 28% Female (56), 3% Unspecified (6) Ethnicity- 61% Caucasian, 10% African American, 10% Asian, 7% Other, 6% Hispanic, 5% unspecified, 1% Native American Level- 57% Senior, 31% Junior, 11% Sophomore, 1% Freshman Post Demographics (N=142): Gender- 70% Male (99), 30% Female (43), Unspecified (0) Ethnicity- 63% Caucasian, 10% African American, 10% Other, 7% Asian, 5% Hispanic, 1% Native American, 5% unspecified Level- 62% Senior, 27% Junior, 10% Sophomore, 1% Freshman

  10. Methodology • Items • 4 point Likert type scale, 4 being positive • Some items were reverse scored • Collapsed into construct means representing 4 variables • Ethnicity collapsed into URM status • Reliability • Coefficient alphas above .547 • MANOVA • To test hypothesis that there would be differences between means based on time, gender, URM status

  11. What was measured • Self Efficacy (15 items) • I can formulate a research problem • Intent to attend graduate school (15 items) • I plan to apply to graduate school in a computing discipline • Attitudes towards computing (38 items) • I like to use computer science to solve problems • Help seeking/coping skills (30 items) • When I do poorly on an exam, typically I….skip class

  12. Outcomes • Increases at Post Assessment were not significant • No main effects, no interactions

  13. What does this mean? • We know there is impact from the REU experience • More investigation is needed • How to detect impact • Cultural nuance may be missed by survey with small sample of women and URMs relative to men

  14. Post Program Evaluation • 38% reported plans to participate in future REU • For 18%, this was not their first REU

  15. Effects are small given the time frame Effects may be delayed Are we measuring the right things? Need more exploration Self-report Sample Possible ceiling effect Consider time series design Implications & Limitations

  16. Common Application Track offers Follow up with applicants Compare accepted/declined on key indicators Survey Deeper analysis Larger sample More variables, or different variables Control groups? Learning outcomes? Next Steps Collaborations for Research & Writing Circles

  17. DiscussionWhere do we want to go from here?What do we want to know? CISE REU Evaluation Toolkit http://www.coitweb.uncc.edu/reu/toolkit/

More Related