BPC Common Core Indicators Examples from CAITE

1. Alan Peterfreund BPC Common Core IndicatorsExamples from CAITE NSF BPC – PI Meeting Los Angeles, CA Feb. 1, 2010

2. BPC Goals • Increase number of students receiving post-secondary degrees in computing, with an emphasis on underrepresented populations • Develop or increase organizational capacity to support goal 1 • Broadly spread good practices to support goal 1 SageFox Consulting Group

3. Core Indicators • Increase number of students receiving post-secondary degrees in computing, with an emphasis on underrepresented populations Individual Participation and Outcomes • Develop or increase organizational capacity to support goal 1 Organizational capacity increases • Broadly spread good practices to support goal 1 Alliance Impacts SageFox Consulting Group

4. CAITE CENTRAL: Middlesex Community College Worcester State College University of Massachusetts Lowell EAST: Northern Essex Community College Bunker Hill Community College Roxbury Community College University of Massachusetts Boston WEST: Greenfield Community College Holyoke Community College Springfield Technical Community College University of Massachusetts Amherst SOUTHEAST: Bristol Community College Cape Cod Community College Bridgewater State College University of Massachusetts Dartmouth SageFox Consulting Group

5. CommonwealthAllianceforInformationTechnologyEducation • Piloting outreach programs supported by educational pathways in four regions • Aiding students from rural, suburban, and urban areas • Identify best practices and disseminate, deploy, extend and institutionalize these statewide and nationally • Community colleges are centerpiece because of central role in reaching underserved populations and as gateway to careers and further higher education. SageFox Consulting Group

6. CAITE Goals • Outreach • Raising awareness of IT careers and educational pathways • Changing the image of computing/IT • Recruiting in IT programs by increasing student interest, opportunities and preparation • Pathways • Improving preparation and instilling confidence in the required skills • Providing role models, mentoring and community • Creating nurturing and understanding teaching and learning environments and pathways • Removing barriers to entry and transfer and providing flexible (multiple-entry/multiple-opportunity) pathways • Partnership • Leverage local, regional and national partnerships for obtaining and disseminating best practices and for cooperation and collaboration • Evaluation • Data-driven project management (understanding institutions, communities and issues with the CAITE Alliance and adapting/developing strategies and tactics) • Evaluation of CAITE outreach and pathway interventions • Evaluation of CAITE policies and management SageFox Consulting Group

7. Participation • Outreach • Raising awareness of IT careers and educational pathways • Changing the image of computing/IT • Recruiting in IT programs by increasing student interest, opportunities and preparation • Pathways • Improving preparation and instilling confidence in the required skills • Providing role models, mentoring and community • Creating nurturing and understanding teaching and learning environments and pathways • Removing barriers to entry and transfer and providing flexible (multiple-entry/multiple-opportunity) pathways • Partnership • Leverage local, regional and national partnerships for obtaining and disseminating best practices and for cooperation and collaboration • Evaluation • Data-driven project management (understanding institutions, communities and issues with the CAITE Alliance and adapting/developing strategies and tactics) • Evaluation of CAITE outreach and pathway interventions • Evaluation of CAITE policies and management SageFox Consulting Group

8. Organizational Capacity • Outreach • Raising awareness of IT careers and educational pathways • Changing the image of computing/IT • Recruiting in IT programs by increasing student interest, opportunities and preparation • Pathways • Improving preparation and instilling confidence in the required skills • Providing role models, mentoring and community • Creating nurturing and understanding teaching and learning environments and pathways • Removing barriers to entry and transfer and providing flexible (multiple-entry/multiple-opportunity) pathways • Partnership • Leverage local, regional and national partnerships for obtaining and disseminating best practices and for cooperation and collaboration • Evaluation • Data-driven project management (understanding institutions, communities and issues with the CAITE Alliance and adapting/developing strategies and tactics) • Evaluation of CAITE outreach and pathway interventions • Evaluation of CAITE policies and management SageFox Consulting Group

9. Alliance Impacts • Outreach • Raising awareness of IT careers and educational pathways • Changing the image of computing/IT • Recruiting in IT programs by increasing student interest, opportunities and preparation • Pathways • Improving preparation and instilling confidence in the required skills • Providing role models, mentoring and community • Creating nurturing and understanding teaching and learning environments and pathways • Removing barriers to entry and transfer and providing flexible (multiple-entry/multiple-opportunity) pathways • Partnership • Leverage local, regional and national partnerships for obtaining and disseminating best practices and for cooperation and collaboration • Evaluation • Data-driven project management (understanding institutions, communities and issues with the CAITE Alliance and adapting/developing strategies and tactics) • Evaluation of CAITE outreach and pathway interventions • Evaluation of CAITE policies and management SageFox Consulting Group

10. Evaluation Measurements:Individual Participation and Outcomes (1) • Do we see an increase in the participation of women and minorities in IT/Computing programs? • Are they “succeeding”? • HS to College • 2 year to 4 year pathways • Gateway courses • Degree Completion SageFox Consulting Group

11. Evaluation Measurements:Individual Participation and Outcomes (1) • Do we see an increase in the participation of women and minorities in IT/Computing programs? • Are they “succeeding”? • HS to College • 2 year to 4 year pathways • Gateway courses • Degree Completion • Challenges: • Data collection • Reporting trends of multiple institutions as core indicators SageFox Consulting Group

12. Students Starting in a Computer Science Major As a Proportion of the Enrollment into 2000-2001 SageFox Consulting Group

13. Percentage of Underrepresented Minority Students Matriculating into CS Majors SageFox Consulting Group

14. Women Matriculating into CS Majors SageFox Consulting Group

15. Transfer Students Matriculating into CS Majors SageFox Consulting Group

16. Reporting Change (Pre-CAITE) Data are from three university schools Changes are from 2005-06 to 2006-07 SageFox Consulting Group

17. Evaluation Measurements:Individual Participation and Outcomes (2) Level of Participation • Enrolled (multiple experiences, e.g., REU, mentoring, internship) • Multi-day (specific course, camp) • Single Day (lecture, workshop) • Web site Hits Type of participant • K-12, CC, 4 YR, Grad, Faculty, Other SageFox Consulting Group

18. Evaluation Measurements:Individual Participation and Outcomes (2) Level of Participation • Enrolled (multiple experiences, e.g., REU, mentoring, internship) • Multi-day (specific course, camp) • Single Day (lecture, workshop) • Web site Hits Type of participant • K-12, CC, 4 YR, Grad, Faculty, Other • Challenges: • Capturing all events and information about participants • Capturing info about number of events across Alliance SageFox Consulting Group

19. Activity Evaluation 30 CAITE activities evaluated in past year 1354 Survey responses from: • 4 Courses • 5 Career Fairs • 9 Conferences/Seminars • 5 Articulation Summits • 4 Teacher Development Workshops • 2 Website Examinations • 3 FSG/SI Sessions SageFox Consulting Group

20. Common CoreActivity Participation Matrix SageFox Consulting Group

21. Evaluation Measurements:Individual Participation and Outcomes (3) Changes in attitudes (surveys) • Interest in/awareness of topics, education, careers • Confidence of ability to succeed • Sense of cultural support • Intention to persist in education, career, or leave SageFox Consulting Group

22. Evaluation Measurements:Individual Participation and Outcomes (3) Changes in attitudes (surveys) • Interest in/awareness of topics, education, careers • Confidence of ability to succeed • Sense of cultural support • Intention to persist in education, career, or leave SageFox Consulting Group

23. Original Plan • Pre-survey prior to event • Post-survey at end of event • Compare change How much do you know about the following? SageFox Consulting Group

24. Original Plan • Pre-survey prior to event • Post-survey at end of event • Compare change How much do you know about the following? Problem: Pre-Surveys not always given SageFox Consulting Group

25. New Plan • Ask participants how participation changed their attitudes How much did each of the following change because of your participation in the event? SageFox Consulting Group

26. New Plan • Ask participants how participation changed their attitudes How much did each of the following change because of your participation in the event? New Question: How to Report as Common Core? SageFox Consulting Group

27. Evaluation Measurements:Organizational Capacity • Development of alliance members (member reports, site visits) • Organizational Capacity Rubric, Count of improvements • Changes in procedures and facilities (member reports, site visits) • Count of improvements • Enhancement of engagement with students (member reports, site visits) • Count of improvements • Institutionalization of practices (member reports, site visits) • Count of improvements SageFox Consulting Group

28. Evaluation Measurements:Organizational Capacity • Development of alliance members (member reports, site visits) • Organizational Capacity Rubric, Count of improvements • Changes in procedures and facilities (member reports, site visits) • Count of improvements • Enhancement of engagement with students (member reports, site visits) • Count of improvements • Institutionalization of practices (member reports, site visits) • Count of improvements • Challenge: • Collecting uniform institutional level information SageFox Consulting Group

29. Organizational Capacity Rubric Example Data are for illustration only SageFox Consulting Group

30. Evaluation Measurements:Alliance Impact • Development of partnerships (member reports, site visits) • Counts of new partnerships • Number of practices spread • Characterization of partnerships and practices • Structural change (member reports, site visits) • Count of new pathways • Count of new agreements • Characterization of pathways and agreements SageFox Consulting Group

31. Evaluation Measurements:Alliance Impact • Development of partnerships (member reports, site visits) • Counts of new partnerships • Number of practices spread • Characterization of partnerships and practices • Structural change (member reports, site visits) • Count of new pathways • Count of new agreements • Characterization of pathways and agreements • Challenge: • Creating agreed upon performance standards that allow measurement of progress SageFox Consulting Group

32. Assessing State of Partnership • From Partnership for Reform in Science and Mathematics • Initiative of Board of Regents of the University System of Georgia and Georgia Department of Education • Partnership Rubric used to rate qualitative differences in targeted performance or implementation of strategies SageFox Consulting Group

33. Partnership Rubric • 5 Dimensions • Vision and Goals • Communication • Decision Making • Responsibility and Accountability • Change and Sustainability • 4 Levels of Outcomes • Beginning • Emerging • Developing • Accomplished SageFox Consulting Group

34. CAITE Partnership (Jan 2010) • Vision and Goals – Emerging to developing • A shared vision is forming with recognition by some of “value-added”. Increasing collaboration on common goals • Communication - Developing • Communication promotes progress toward achieving a shared vision and common goals • Decision-Making – Emerging to developing • Most partners are represented by those with limited authority to make decisions that promote individual or organizational goals, but are less committed to making decisions toward common goals • Responsibility and Accountability - Developing • Partners commit to new roles of shared responsibility as a result of a commitment to the common goals • Change and Sustainability - Emerging • Partners learn that all can contribute. They work together to identify necessary changes that meet individual and common goals SageFox Consulting Group

35. Alliances are: • Complex • Ambitious • A challenge to evaluate - Key to scaling success and achieving the BPC agenda SageFox Consulting Group