Evaluation of the Quebec Community Learning Centres: An English minority language initiative

1. Evaluation of the Quebec Community Learning Centres: An English minority language initiative Natalie Lacireno-Paquet Learning Innovations at WestEd NPaquet@WestEd.org 1-800-347-4200

2. Evaluation Purposes • To support the implementation of the project • To gather data to assess the attainment of the short and intermediate expected outcomes and the longer terms impacts of the project • Examine and identify the processes that help or hinder the attainment of expected outcomes and impacts

3. Evaluation Approach • Multi-pronged, mixed-methods, and longitudinal, including: • Implementation of the CLC model at 22 sites, guided by the Project Theory of Change • Work of the PRT in building CLC staff capacity and the work of the PIC in supporting the CLCs through external relations • Evaluation of expected outcomes and impacts • Documentation of lessons learned

4. Evaluation Questions • Questions are at the implementation level, the impact level, and lessons learned • Questions developed in collaboration with PRT and informed by literature • Detailed evaluation questions and indicators found at: http://www.learnquebec.ca/en/content/clc/clc_res_eval.html

5. Evaluation Questions: Implementation • To what extent and in what ways do the PRT and PIC implement activities and processes designed to contribute to the capacity of the CLCs to achieve short and intermediate outcomes? Some sample indicators: • Types of support services provided by PRT • Perceived responsiveness of PRT to CLC needs • Types of collaborations established by PRT • Level of support in establishing connections with regional and provincial resources • Perceived use and value of trainings • Perceived challenges and resolutions

6. To what extent and in what ways do the CLCs, led by principals and CLC coordinators, implement their Action Plans? A small sample of indicators: • Level of use and valuation of Framework for Action (FFA) • Development and implementation of action steps • Identification of stakeholders/ Inclusion of stakeholder groups and steering committee in development • Types of community involvement activities/strategies planned and developed • Number and frequency of community members’ use of services • Number and types of community organizations involved in partnerships with CLCs • Teachers use of Community Based Learning • Types of services provided and extent to which services meet identified needs/gaps

7. Evaluation Questions: Impact • To what extent and in what ways do the 22 CLCs accomplish their goals? Sample of indicators: • Number, types, and characteristics of collaborative partnerships between schools, families, and communities • Student success measures (i.e. achievement, attendance, etc.) (collected by schools/CLCs) • Level and extent of participation in CLC activities and services by various stakeholders • Similarities and differences in implementation and impact across CLCs

8. To what extent and in what ways do the 22 CLCs show evidence of sustainability after the project funding and support end? • Level, types, and nature of community support, diversified funding, and formalized policies and practices that are developed by the CLCs • Additional support and resources identified

9. Evaluation Questions: Lessons Learned • To what extent and in what ways do the PRT’s and CLC Theories of Change adequately represent the processes and outcomes of the initiative? • What lessons does the project offer for policy and practice, particularly concerning establishment and support of CLCs in English-speaking communities in Quebec? • Factors identified that facilitate or hinder the work • Processes associated with successful attainment of results • Insights from project and CLC “theories of change”

10. Methods • Mixed methods evaluation • Qualitative and quantitative data collected • Site visits to each CLC • Observations of trainings, meetings, etc. • Extensive interviews and document review • Portraits / case studies of 22 sites with cross-site analysis • Longitudinal data collection: 2007 - 2009 • Collaborative approach • Frequent communication/feedback • Formative as well as summative reporting

11. Evaluation Activities to Date • Observations of multiple trainings and meetings with focus groups of principals and coordinators • Interviews with members of PRT • Interim reports on observations/interviews • Initial interviews with Phase 1 coordinators • 2-day site visits to Phase 1 CLCs with “Findings Memo” to sites • Initial quantitative portrait development for Phase 1 CLCs

12. Early Findings: Project Level • Project Resource Team (PRT) has been flexible and responsive to formative feedback • Changes to “Guidebook” • Reduction in paperwork • Introduction of “Concerns Based Adoption Model”- help understand process of change Importance of coordinator/principal in-put in planning activities • Importance of time for coordinators and principals to share and learn from each other

13. Tailored and one-on-support of CLCs • Realistic expectations • Responsive to unique situations • PRT has both shared work and specialized work • Share supporting CLCs through “pairing” of PRT member and CLC • Specialized roles relating to support and training in community based learning, self-evaluation, community partnerships and networking

14. Early Findings: CLC level • Coordinators are key • Work hard at establishing partnerships and CLC visibility • Principals rely heavily upon them • Some coordinators feel overworked but satisfied. Can be overwhelming at first. • Coordinators building a support network • Some coordinators driving vision- rather than developing shared vision

15. Early Findings: CLC level • Leadership and Governance • CLCs don’t advance in implementation without principal support • Leadership change may hurt or help • Some confusion over role of school Governing Board with respect to CLC • CLCs use steering committees to varying degrees • Not all school boards supportive affects implementation

16. Involvement of teachers, parents, and students • Wide variation among the CLCs from low to high • Expression of sense of pride “our school is a CLC” but some take a “wait and see” view • Some teachers and students associate CLC with video-conference equipment • Some concern about time to learn VCN and Community Based Learning techniques • Use of VCN and CBL have taken off in some communities

17. Community and Partnerships • Many new services are being offered • Many new partnerships established • Most CLCs engaging with partners and community; one or two are not at this time but renewed efforts appear to be underway • CLCs not always selecting partnerships strategically- some cast wide net, others more selective

18. Other findings and challenges • Two models emerging: woven and parallel • CLCs with multiple schools involved see varying levels of involvement and have additional challenges for implementation • Coordinators / principals value time to come together in PRT meetings and trainings • Sustainability is a major concern among stakeholders