1 / 31

5th QualiFLY Meeting Hamburg, 20. 02. 07 Evaluation of a Family Literacy Project in Hamburg Peter May

5th QualiFLY Meeting Hamburg, 20. 02. 07 Evaluation of a Family Literacy Project in Hamburg Peter May State Institute for teacher training and school developement Hamburg, Germany. Overview Internal and external evaluation Phases of evaluation Applied methods First Findings Problems.

viet
Download Presentation

5th QualiFLY Meeting Hamburg, 20. 02. 07 Evaluation of a Family Literacy Project in Hamburg Peter May

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 5th QualiFLY Meeting Hamburg, 20. 02. 07 Evaluation of a Family Literacy Project in Hamburg Peter May State Institute for teacher training and school developement Hamburg, Germany

  2. Overview Internal and external evaluation Phases of evaluation Applied methods First Findings Problems

  3. FörMig Förderung von Kindern und Jugendlichen mit Migrationshintergrund external evaluation internal evaluation University of Hamburg State Institute, Hamburg Sub-project: HAVAS Sub-project: Family Literacy diagnosis-based aid parental support Target group: children parents and children Form of evaluation: summative formative Data collection instruments: intelligence test participant observation language testing interviews questionnaire about children questionnaire parents questionnaire educationalist

  4. Phases of evaluation 1. Pre-Investigation of circumstances in the field school year 03/04 • Interests (parents, educationalists, headmasters of schools and kindergardens) • Circumstances in possible locations Implementation of the project in preschool classes school year 04/05 • Analysis of the concept for preschool classes • Description of target group • Evaluation of the educational concept Consolidation and extension of the concept to first grade classes school year 05/06 • Analysis of the concept extension into first grade classes • Circumstances in new locations Networking and sustainability School year 06/07 • Building up a network with other projects • Transitions from preschool to first grade • Learning effects of the children

  5. Phases of evaluation 1. Pre-Investigation of circumstances in the field school year 03/04 • interests (parents, educationalists, headmasters) • circumstances in possible locations Methods: guideline-based interviews • with parents from different ethnical and social groups: e.g. families from Turkey, Africa, Russia, German underclass families • with educationalists in possible locations (schools, day-care centres) • with headmasters from schools and day-care centres

  6. Phases of evaluation • Implementation of preschool classes school year 04/05 • analysis of the concept for preschool classes • description of target group • evaluation of the educational concept Methods: • participating observation - in introductory events and evening classes for parents - in groups of parents and children in the morning - in advanced training courses for disseminators (teachers, educators) • questionnaires, predominantly with open answers - parents at the beginning: motives, circumstances, interests • - disseminators: Implementation of the concept at locations • • questionnaires, predominantly with given answers - parents at the end of the courses: changes (behaviour, attitudes), satisfaction with the courses - teachers: assess the learning development

  7. Phases of evaluation Consolidation and extension of the concept to first grade classes school year 05/06 • analysis of the extension of the concept in first grade classes • circumstances in new locations Methods: • participating observation - in advanced training courses - in courses for parents in first grade classes • questionnaires, predominantly with given answers - parents at the beginning: interests, characteristics of the family - parents at the end of the courses: changes, satisfaction - disseminators: - design of the courses - estimation of success • testing of the children: - non-verbal intelligence - language competence at the beginning of the course

  8. Phases of evaluation Networking and sustainability school year 06/07 • building up a network with other projects • transitions from preschool to first grade • learning effects of the children Methods: • guideline-based interviews with new partners: - other aids in preschools and primary schools - Volkshochschule (language classes for parents) • questionnaires: interviews with parents before and after • systematic coverage of performances of children: - sequel testing (pre-, post- and follow-up) in preschools, first and second grade (language, reading, writing) - teacher’s assessment of educational development

  9. Methods of evaluation (1) participating observation - courses for parents - events with parents and children - meetings of disseminators advantages: - direct impressions („atmosphere“) - opportunities to get in contact with parents and educationalists - informal conversations with many details disadvantages: - high costly - influence of subjective impressions

  10. Methods of evaluation (2) Oral interview (guideline-based interviews) • with headmasters of schools and kindergarten • with single disseminators • with special experts • with single parents (occationally with translators) advantages: - private atmosphere - possibility to adjust the questions spontaneously - offers new, sometimes surprising insights disadvantages: - costly - can be subjectively influenced - difficulty to generalise

  11. Methods of evaluation • questionnaires (a) with open answers - disseminators: narration of concepts and procedures collection of suggestions • - parents: survey of perceptions collection of wishes • required: translations (Turkish, Russian, Farsi, English) • advantages: • less costly than interviews • - often offers new insights • disadvantages: • no systematic coverage of relevant characteristics • partially costly analysis, requires analysis of contents

  12. Methods of evaluation • questionnaires (b) with given answers - disseminators: systematic enquiry of circumstances in locations personal backgrounds and motives estimation and assessment • parents familial backgrounds personal characteristics of children changes in behaviour and attitudes (before - after) • required: translations • advantages: • - very economic • - systematic coverage of relevant characteristics • - statistical analysis possible • disadvantages: • - reduced answers • - sometimes difficult to interpret

  13. Methods of evaluation (4) Testing - intelligence test - language test: language competence at the beginning sequel testing (post, follow-up in first grade) - reading and writing in second grade advantages: - no subjective influence - very economic (group testing) - provides computable data disadvantages: - high input to establish objective circumstances for testing - „psychological barriers“ for parents and educationalists

  14. Outcomes of questioning of parents Number of questionnaires for parents: (8 locations) School year 2004/05: approx. 70; 2005/06: approx. 100; 2006/07: approx. 130 overall: approx. 300 parents Contents of questioning: • personal und familial background of parents • motives und wishes for participation in courses • changes in the families (reading habits, support for children) • rate the courses (feed back)

  15. First language of parents born in Germany: 33 %

  16. sex 18 % male 82 % female professions professional educated: 59 % employed: 40 % wide broad of professions (from non educated workers to university graduates)

  17. parents pre-questionnaire: reading and writing at home

  18. parents pre-questionnaire: reading and writing with the children

  19. Expectations of participants to course attendance 82 % Interest in new ideas and continuation of own education 56 % Support for the own child 42 % Exchanging experiences and to establish contacts 10 % cooperation with teachers

  20. parents post-questionnaire: reading and writing at home

  21. parents post-questionnaire: reading and writing with children

  22. parents post-questionnaire : benefit of the courses for parents

  23. parents post-questionnaire : benefit of the courses for the children

  24. Evaluation of the courses by the parents 27 % very good 72 % good 1 % satisfying 0 % dissatisfying

  25. Disseminators questionnaire:

  26. Disseminators questionnaire:

  27. Disseminators questionnaire:

  28. Disseminators questionnaire:

  29. Summary of evaluation results so far: • highly motivated disseminators (teachers and educators) • high satisfaction of the parents • few changes of parents literacy behaviour • more advancement in helping children reading at home • benefit for children’s learning in parental view • open question : children‘s benefit in objective view (tests)

  30. Problems with the evaluation • serious fluctuations in participation • lack of control groups • test conditions hard to control • analysis of effect conditions complicated, due to complex mass of arrangements.

More Related