Evaluation of the Project : OUTREACH MOBILE TEAMS FOR CHILD PROTECTION

1. Evaluation of the Project:OUTREACH MOBILE TEAMS FOR CHILD PROTECTION January, 2004.

2. OUTREACH MOBILE TEAMS FOR CHILD PROTECTION PROJECT • CONTENT • I Project overview • History of the project • Project description • Project team • Description of cases treated by MT • II Evaluation of the project • Evaluation based on judgment of MTs • Evaluation based on impressions of the caretakers of children who were involved in the project • III Conclusion • IV Annex: Results review of Evaluation based on impressions of the caretakers of children January 2004.

3. I PROJECT OVERVIEW 1. History of the Project A decade of conflict, isolation and sanctions has resulted in widespread poverty, inadequate social services and infrastructure, and outdated equipment in industry and agriculture. In particular, Centers for Social Work (CSWs) in charge of the delivery of social services countrywide suffer from lack of funding, material and human resources to sufficiently meet the growing needs of the socially and economically vulnerable. The lasting crisis has had a detrimental impact on the functioning of the family with particularly devastating impact on children as the most vulnerable. The CSWs currently cover about 150,000 children from families with disrupted relations, as well as some 5,000 children who are either permanently or temporarily deprived of parental care and placed in families or institutions. About 10,000 families with juvenile delinquents are under acute treatment. (Date from UNICEF “School for Parents” project proposal written in September 2000.) UNICEF funded the project, and its direct implementing partner JEN, a Japanese NGO, was an umbrella to three local NGOs (Amity, Horizons and Sunce) which currently covers twelve selected areas in Serbia. Amity covers Belgrade area in 6 municipalities (Obrenovac, Pozarevac, Mladenovac, Valjevo, Smederevo, Bor), Sunce covers central Serbia in 2 municipalities (Jagodina, Nis) and Horizons covers southern Serbia in 4 municipalities (Bujanovac, Presevo, Lebane, Leskovac). The project began as a pilot phase in July 2001, in 4 areas of Serbia, Obrenovac, Pozarevac, Bujanovac, and Presevo. It was one of the first projects that the governmental sector (CSW) and the non-governmental sectors (mobile teams of the local NGOs) worked together in collaboration to detect and resolve durable solutions for all the children in need of protection. It expanded to 8 more areas from May 2002, making the implementation to the 12 municipalities stated above. January 2004.

4. 2. Project Description • (a) Overview • The vision for this project, which is also the aim for the future, was“to provide the infrastructure for the governmental and non-governmental sectors to work together in the delivery of social services for child protection.” • In order to achieve this future vision, there were two goals that were set to achieve during the project: • To involve the municipal government directly through the implementation of the project to set the foundations for a sustainable process, • Awareness raising of children’s rights issues within the community through transparency within the Centers for Social Works and information campaigns. • In the initial stage, the concrete actions that the child protection Mobile Teams (MTs) took included: • identification of the children, • organization of case management conferences by CSW and the MTs and relevant sectors to find a solution best for the child, and • follow-up on all the individual cases. • However, the more the MTs became involved working with the CSWs, their roles and responsibilities haveexpanded. • The project activities also contained components of capacity building of the CSW through training and material support, such as office equipment, and the installment of the database to input the data for all the children protected. • Finally, this project intended to accord with the national reform strategy of the Ministry of Social Welfare and with other International organizations’ (NGOs) projects underway. January 2004.

5. 2. Project Description • (b) Training and Special Education • There were two phases for this project, the pilot phase and after. In the beginning of each phase, the project team (representatives from the CSW and the MT with the coordinators of the local NGOs) went under different training and education from the experts in the field for children issues. UNICEF and JEN organized these training sessions. The trainings are listed below: • School for Parents • Systemic Family Approach • Child Abuse and Neglect • Public Relations – Fundamental Rules • Participative Approach Techniques • Juvenile Justice and Delinquency • System of Social Protection and Family Regulations • Media Relations • Additional training to the MT members and the coordinators included: • Relations between Needs, Protection, and Children well being: Children protection process (for Bujanovac and Presevo) • The Convention on the Rights of the Child • Besides the specific training for child protection, all members of the project team (and some others who work in the CSW) received a certain hour of computer training. They had the basic knowledge of using the computer, MS Word and MS Excel. This training was necessary to input the data into the computerized database for all the children who have been identified and followed-up in the course of the activity. It was held also to build the capacity in the CSW for the people who work there. January 2004.

6. 3. Project team UNICEF Svetozara Markovica 58, 11000 Beograd, 011/3602-100, e-mail: office@unicef.org JEN Draze Pavlovica 1, 11000 Beograd, 011/ 2776-671, e-mail: nwp@jen.org.yu JEN Sub Office Nis Dusanova 40, 18000 Nis, 018/547-734, e-mail: jennis@eunet.yu Three NGO take part in the project: Amity, Sunce and Horizonti. The MTs consisted of 27 members in 12 municipalities. The mobile team members were psychologist, social workers, pedagogues, medical experts, or ethnic leaders. January 2004.

7. 3. Project team AMITY Spanskih boraca 4, 11070 Novi Beograd, 011/131-261, e-mail: amity@yubc.net Vesna Vicentijevic, Project Coordinator MT Pozarevac Milorad Jokic, psychologist Slobodan Jovanovic, psychologist Medzid Dzemaili, dervis MT Obrenovac Zivota Veljkovic, psychologist Branka Veselinovic, social worker Milena Nikolic, patronage nurse MT Mladenovac Vera Knezevic, social worker Vera Stevanovic, psychologist MT Smederevo Dejan Zivanovic, neuro-psychiatrist Vesna Emersic, social worker MT Valjevo Dragica Andjelkovic, neuro-psychiatrist Zivana Pantic, psychologist MT Bor Milojka Milivojevic, social worker Sladjana Stanojevic, psychologist January 2004.

8. 3. Project team HORIZONTI Takovska 16, 32000 Cacak, 032/222-455, e-mail: mastef@eunet.yu Vesna Kovacevic, Project Coordinator MT Bujanovac Milka Milanovic, psychologist Darinka Spiric, psychologist Jakupi Seljami, pedagogue MT Presevo Gafur Hiseni, pedagogue Nedzmedin Veliu, pedagogue Momcilo Stepanovic, social worker MT Leskovac Snezana Stojanovic, psychologist Vesna Apostolovic, social worker MT Lebane Goran Stamenkovic, paediatrician Budimir Bogdanovic, pedagogue SUNCE Luja Pastera 17, 34000 Kragujevac, 034/362-700, e-mail: sunce@infosky.net Marija Stojadinovic, Project Coordinator MT Jagodina Slavica Jakovljevic, psychologist Valentina Kocic, pedagogue MT Nis Ljiljana Stoiljkovic, social worker Sladjana Velickovic, psychologist January 2004.

9. 4. Description of cases During 3 years period 3170 cases have been involved in the project. 1516 children were paid more than one visit. Total number of visits to children by MTs is 8467. January 2004.

10. 4. Description of cases • Cases were sorted into 5 categories, according to the problems children were facing: • Children without parental care • Children with behavior disorders • Children with special needs • Children from dysfunctional families • Abused and neglected children Risk factors were estimated by members of MTs during the first visits to children. According to their judgments, 20% of children fell within the category of low risk, 45% within the category of medium risk and 35% within the category of high risk. 67% of all cases were identified for the first time by MTs members. 33% of cases had already been identified by other institutions involved in helping children (Center for social work). January 2004.

11. 4. Description of cases Children involved in this project were of both genders, up to 19 years of age. There was almost the same number of children from urban and rural area. January 2004.

12. II EVALUATION OF THE PROJECT • Evaluation ofOutreach Mobile Teams For Child ProtectionProject consists of two parts and is realized through two different methods. • Evaluation based on judgments of MTs • Evaluation based on impressions of the caretakers of children who were involved in the project • 1. Evaluation based on judgments of MTs • Goal of the first part of evaluation was quantitative measuring of effects of MTs’ work.Its realization wasbased on the data that members of MTs entered into the data base. After each visit to a child, members of MTs evaluated level of risk for that child. Level of risk was evaluated on a 3-degree scale (1-low, 2-medium and 3- high risk level). • Evaluation was conducted through observing if there came to a change of risk level that could be eventually ascribed to the work of MTs, during the whole period when a child was offered help. • 2. Evaluation based on impressions of the caretakers of children who were involved in the project • This part of evaluation refers to impressions of children’s caretakers of the MTs’ work.For that purpose there was a questionnaire made, with the aim to describe the impressions about Mts’ work from the angle of families themselves. January 2004.

13. 1. Evaluation based on judgments of MTs Difference between starting and final level risk depending on the sort of a problem HIGH RISK LOW RISK There is a difference between levels of risk noticed during the starting and the final visit. The difference is statistically important on level 0.01. (Analyses were conducted on 1407 children who had at least one repeated visit). No matter the sort of the problem, there came to a significant change (on level 0.01) in risk level– with all 5 sorts of problems observed, risk level was significantly decreased. January 2004.

14. 1. Evaluation based on judgments of MTs Difference between starting and final level risk depending on the action for help to children A sample of 240 children was formed for this purpose and categorization of applied actions was conducted. For examining of the efficiency of actions, a sample of children was formed out of the total number of children MTs worked with. First, children were divided according to the kind of a problem and the sample was formed including all 5 examined sorts of problems (40 children without parental care, 40 children from families with disturbed relations, 40 children with special needs, 80 abused and neglected children and 40 children with behavior disorders and in clash with the law). Only the newly identified cases with more than one visit were taken into consideration, but also with less than 7 repeated visits. Since there could be more different actions applied on one child, in this way there was 730 actions categorized into 7 categories. This result has to be taken with reserve for the following reasons: There was not one significant difference between starting and final level risk found, depending on a specially applied action. In other words, and in accordance with the previous results, there came to risk level decrease, but that effect could not be ascribed to any specific actions, but, as it seems, the work with children itself led to decreased risk. • The sample of children that was analyzed is small and it might be possible to get different results on a bigger sample • The examining situation wasn’t experimental, but real – not all the children passed through the same treatment. It is possible for some usually efficient actions to have been applied on children with more difficult problems, therefore the effects of those actions are opposed to the degree of difficulty of problems children were facing. January 2004.

15. 1. Evaluation based on judgments of MTs Difference between starting and final level risk depending on the action for help to children How was the efficiency of separate actions measured? Evaluation was conducted through observing if there came to a change of risk level that could be eventually ascribed to the work of MTs, during the whole period when a child was offered help. However, 1, 2 or more different categories of actions could have been applied to children. Therefore, during measuring effects of each separate action, effects of other actions were held constant. It was also possible to apply the same category of actions on a child during several visits (for example, during 3 visits to the same child, help with documentation was offered). Analyses were conducted just regarding whether there was a certain action applied to a child or not, and not how many times that action was applied. In order to measure the change between starting and final level risk for some child, a new variable was created: Risk change = Final risk – Starting risk Since both starting and final risk levels are measured on a 3-degree scale (Low, Medium and High), the new variable Risk change can be measured on a 5-degree scale (from –2 to +2, where –2 means worsened condition of a child compared to the beginning , and +2 means improving of condition compared to the starting point of examination). January 2004.

16. 1. Evaluation based on judgments of MTs Difference between starting and final level risk depending on the action for help to children The analyses show that no action applied had separate statistically significant influence (on level 0.05) on decrease of risk level with children. There is also no interaction between applied action and kind of the problem (in other words, there are no actions that give results applied on one kind of examined problems, and not on some other). The only decrease of risk level after some action applied that is on the edge of being statistically significant (0.05) is Help in gathering documentation in case of Children from families with disturbed family relations, and also Rounds (Gathering Information). January 2004.

17. 2. Evaluation based on impressions of caretakers of the children who were involved in this project GOAL: Goal of the first part of evaluation was quantitative measuring of effects of MTs’ work on children and families involved in the project. For that purpose there was a questionnaire made, with the aim to describe the impressions about MTs’ work from the angle of families themselves. RESEARCH TECHNIQUE: quantitative field research –direct interviews at respondents’ homes. Respondents were parents and other caretakers of children involved in this project. Interviews were conducted by pollsters from an independent research agency (SMMRI). Creating of the sample and data processing were also conducted by the same research agency (SMMRI). SAMPLE: For the evaluation of the project, there was a sample of 120 families visited by MTs formed. All the families in the sample had to have (according to the record) at least one repeated visit by MTs. These 120 families covered 12 municipalities where the project was realized, and the kinds of problems were equally represented in the sample. The sample structure was established in the following way: January 2004.

18. 2. Evaluation based on impressions of caretakers of the children who were involved in this project MAIN RESULTS OF THE SECOND PART OF EVALUATION All the respondents heard of Outreach Mobile Teams For Child Protection Project. All the families were also really visited by MTs. Average number of visits to each family is 4,8 (only the families with repeated visits were included into the sample). In the respondents opinion, the most often mentioned reason for visits of MTs is poverty of a family (36% of all answers). Also, opinions of caretakers about problems of their children is highly related to classification of problems given by MTs. 4/5 of respondents think their children needed help. This number is even bigger for children with behavior disorders (95%). Only 3% of respondents think they didn’t need help. Examined families are very satisfied with the work of MTs (average grade 4.23 on a 5-degree scale from 1 – not satisfied at all, to 5 – completely satisfied). MTs, in the respondents opinion, most often offered help of counseling (3/4 of all cases), material help (in ½ of the cases), help in involving other services in order to solve some problem (1/3 of cases). In 15% of cases, MTs directed them to other services, and in 13% helped in gathering documentation. Base:120 January 2004.

19. 2. Evaluation based on impressions of caretakers of the children who were involved in this project More than 80% of the families involved think they benefited from the MTs’ visits. MTs provided both emotional and material help to families. Base:120 • The guardians spontaneously mentioned benefits from the following services of MTs: • Material help – help in clothes, toys, food, products for hygiene, medications • Help at school – help with learning, intensifying motivation for learning and better marks, providing school material, help with adjusting, help with choosing an occupation • Help in child’s treatment- help with acquiring medications, help with child care • Improvement of relationships within the family - counseling, putting a caretaker in a treatment institution (in case of alcoholism, drug addiction), getting over conflicts • Psychological help to children - counseling, helping parents with their approach to children, help with trauma experiences of children and families, helping children become independent, decreasing insecurity and aggression of children • Help in gathering documentation – gathering documentation for custody, humanitarian help - MOP, children’s allowance, getting a passport, enrolling to school • Help with living conditions – material help, keeping clean, acquiring basic devices and furniture (stove, heater, wood, beds, sheets) • Getting in contact with other institutions – CSW, Home for abandoned children, counseling offices, speech therapists January 2004.

20. 2. Evaluation based on impressions of caretakers of the children who were involved in this project • The caretakers’ statements: • “They helped us put a parent alcoholic in an institution for treatment”. • “They talk to us, give advice, treat the child, help us in everything.” • “They helped me get custody of my nephew who was abandoned by his parents.” • ”They solved the problem of custody, they comfort us, advise me how to raise my children”. • “They gave me the sense of security, understanding. They help me solve my problems”. • “They brought over people from Center for Social Work to help us.” • “They help us when children are sick, when we have problems they tell us what to do and who to turn to”. • “My child’s problems with adjusting at school have stopped and he has better marks now”. • “They helped us get material help from the Center and get children’s papers out.” • Only 8% think the work was in vain, but they usually ascribe it to family’s bad material situation, and not MTs. These families, in their own opinion, are in need of material, but also continuous counseling help. • The caretakers’ statements : • “We don’t live any better,we’re poor,we need someone to take care of us, we have neither work nor apartment.”, • “Except for the advice that had really positive influence, they didn’t help us much.”, • “They brought us only things, but we also need money.” • “The team has fulfilled all its promises, but we expect further support”. January 2004.

21. 2. Evaluation based on impressions of caretakers of the children who were involved in this project ¾ of the caretakers think that their child’s condition has improved through the MTs visits. 14% don’t notice any difference, while 11% couldn’t judge. Almost all the families (93%) would again turn to MTs for help. MTs would be expected to offer the same kind of help as before. Base:120 January 2004.

22. 2. Evaluation based on impressions of caretakers of the children who were involved in this project ½ of the respondents heard of the Convention of The United Nations for the rights of a child. The most familiar rights of children are Right to education (56%), Right to health (45%) and Right to fun and games (39%). 1/3 of respondents think their rights are, at least partly, not respected. In case of threatened rights, they would first turn to Center for social work (57% of respondents) and non-governmental organizations, first of all MTs (40%). Police, courts and other institutions are ranked much lower. Base:120 70% of respondents involved in this project had heard for UNICEF, 38% for JEN organization, and 59% for local non-governmental organization that participated in this project (Amity, Horizont or Sunce). ¾ of families think that the main function of CSW is to provide material help for persons in need. ¾ of families examined had already turned to CSW, but only somewhat more than one half received help. Material help was asked for most often (51%), allowance for care and help (23%) or help in intervening between parents and children (21%). Caretakers were not ready to talk about aggression directed to their children. 15% of caretakers said that children had been punished by members of their family, that’s more often the case with abused and neglected children (25%). January 2004.

23. III CONCLUSION Based on all the issues presented, the following may be concluded: • Based on evaluation of MT members themselves, it can be concluded that there had come to improvement in the condition of children who were in contact with MTs. • Families were very satisfied with the help of MTs that was offered. That help was both of emotional and material nature to children and families, as well as help in getting in contact with other institutions (CSW). They were also provided assistance in gathering documents. • MTs discovered a large number of children who needed help, and who hadn’t been identified before (over 2000 cases). • Families who were in touch with MTs were ready to turn for help, in the future, to CSW or NGOs (MTs). January 2004.