An evaluation of the World Changers Academy’s Leadership programme

1. An evaluation of the World Changers Academy’s Leadership programme by Eleanor Hazell 24 June 2010

2. Presentation overview • Background • Evaluation design & methodology • WCA leadership programme theory • Evaluation findings • Recommendations to WCA

3. Background: About WCA • South African NPO, founded 2002 Vision –to develop the minds, hearts and spirits of people for life success Mission -to empower people for success through character building and values based life skills training Strategies • Life Skills Training • Leadership training for emerging leaders • Mentoring & Life Coaching • Networking students with people of influence and opportunities • Imparting positive values through training and modelling Target groups • unemployed youth, • community leaders, • high school students

4. Background: WCA leadership programme • Life Skills & Leadership programmes initiated 2002 for unemployed youth & community leaders • Programmes for high school students initiated in 2003 • Life Skills =4 week course run in communities • Leadership = 11 week course run at WCA headquarters, with outreach in communities • Life Skills provides students for leadership; leadership students support Life Skills during outreach • 68 leadership courses run & 1930 students graduated since August 2002.

5. Background: Literature review on evaluating leadership programmes • Plethora of leadership programmes, wide range of fields, different target groups • Considered difficult to evaluate; • Individuals participate, but change is expected on broader levels (e.g. organisational/community/society…) • Links between change on different levels not well established • Difficult to evaluate change on organisational/community/ society levels • Influenced by external factors difficult to control • Leadership = journey/process which develops/deepens over time • Few leadership programmes have been rigorously evaluated; • Few leadership evaluation tools exist • Most leadership evaluations done in USA

6. Evaluation design Nov-Dec 2009 • Gather background information • Document review • Meetings with WCA • Develop logic model • Design evaluation Dec 2009 • Collect data (observation & interviews) Jan-April 2009 • Transcription & data analysis May-July 2009 • Report writing & presentation

7. Evaluation design A part process & part outcomes evaluation of the WCA leadership programme for unemployed youth Process evaluation will investigate: • The number of courses run • Number of students enrolled • Where students come from • The completion rate • Course materials and • Course structure To uncover who is benefitting from the course, what students think about the structure of the course and whether the course is being implemented as planned The outcomes evaluation will investigate: • The extent to which course facilitators model good/positive behaviours • The extent to which students are able to describe and demonstrate improved leadership skills in identified areas at the end of the programme • Which behaviours/changes are difficult to sustain and why • What graduates go on to do afterwards • Significant changes which have occurred in graduates lives, which they attribute to the leadership programme.

8. Evaluation questions • How many leadership courses are being run? How many students are enrolled and how many complete the course? (Document review) • Where do the leadership students come from? How did they hear about the programme? Have they completed the life skills course? (Interviews) • Did life skills ‘lay the foundation’ for the leadership programme? What are the key differences between the life skills and leadership programme’s? What components of the leadership programme do students like/dislike? (Interviews & observation) • Is the leadership programme being implemented as planned? Residential and practical components? Is the course being effectively monitored and evaluated? (Document review, interviews and observation) • Are particular problems being encountered or specific barriers experienced in the implementation? (Interviews) • Are the course facilitators (former leadership students) modelling good/positive behaviours? (Interviews & observation) • In what ways are leadership students developing/improving leadership skills in defined key areas (see logic model)? (Interviews) • What significant changes have occurred in leadership students which they attribute to the leadership programme? (Interviews)

9. Respondents Interviews with key informants (4) and leadership programme students (6) Key informants • WCA programmes manager • WCA leadership programme manager • WCA leadership programme co-ordinators (x2) Leadership programme students selected at random 5 males, 5 females; aged 20-37

10. WCA leadership programme theory Goal • To develop young leaders who will contribute positively to their organisations, communities, society, countries, and change the world. Objectives: • To develop the hearts, minds and spirits of students from disadvantaged KZN communities for life success (focus = individual).  • To develop and enhance the leadership skills, effectiveness and impact of existing and potential leaders from impoverished KZN communities (focus = individual impacting on organisation/community/society). • To expand the reach of WCA by developing a leadership programme model, promoting and sharing it widely, and developing strategic alliances in KZN, South Africa and internationally (focus = expand reach and impact). Target group(s): • Primary = Unemployed youth (18-35) from disadvantaged communities in eThekwini, Pietermaritzberg, Ugu district and Northern Zululand. • Secondary – Grassroots community leaders. • Long term - Impoverished KZN communities.

11. WCA leadership programme theory Summary If disadvantaged youth complete WCA’s life skills (foundation) programme and apply what they have learned they will develop emotionally, mentally and spiritually such that their world within is changed resulting in more positive personal outcomes (interpersonal relationships, academic and professional success). If they go on to complete WCA’s leadership (advanced) programme and they consolidate and apply their learning practically, they will develop skills and competencies which will enable them to develop as leaders and impact positively on their organisations, communities and change the world around them. If graduates continue on their leadership journey, accept responsibility, consolidate and apply their leadership skills and develop new ones, they will motivate and inspire other disadvantaged youth, create opportunities and contribute to the upliftment (economically, socially, psychologically and spiritually) of disadvantaged communities.

12. WCA leadership programme theory See Logic Model handout

13. Evaluation findings

14. How many leadership courses are run? How many students are enrolled and how many complete the course?

15. Where do leadership students come from? 1) A range of KZN communities: South Coast, eThekwini, North Zululand 2) Students come from disadvantaged backgrounds and face an array of inter-related challenges: Poverty: “I’m coming from the family that is not good financially” (R1) “I wasn’t having enough money to pay for exams, I wasn’t have enough money for uniform and I just drop out [of school]” (R4). Unemployment: “Since 2003 I was searching for a job” (R6). Crime/prison: “I learnt different ways of doing crime, and it was terrible. So I went to prison for 3 months . That was for stealing, for shop-lifting”(R7). Drug & alcohol abuse: “I start thinking in others ways, start drinking alcohol, drug addiction, and having boyfriends so that I can have money” (R3). Teenage pregnancy: “ I end up even having a child at the age of 16 years” (R3). Relationships/peer pressure: “As the youth we are involved in different relationships which are not good, like we are involved with love relationships which we are not ready or mature enough to be involved in and having friends who are doing wrong things, but we involve ourselves for the sake of having a friend, and don’t know why”(R1). “I drop out of school, and I listen to my friends rather than my parents” (R3). Dropping out of school: Family problems/loss of parents: “In my heart I was having the pain, because I lost my parent[s], my mother she was sick and my father he was poked [stabbed]” (R4). “I was looking for my father actually, he left me and my mum when I was young, so I grow up didn’t know him in the age of 13 or 14 years, then I try to want to get connected with my father, cos the situation at my mother’s house was not good” (R3). Rape: “When I was young I was raped for three guys… afterwards when I went through to test and I test HIV positive” (R4). Depression/attempted suicide: “I tried to kill myself, I think about three times, I went to the road, I was waiting for the car to crush me, but there was no car come” (R4).

16. Have they completed the life skills course? How did they hear about the programme? • All respondents completed life skills (students in 2009, staff between 2002-2006) • 8/9 found out about leadership via LS course • 1/9 was recruited for the leadership course • Respondents found out about LS course via • Word of mouth (8/9) • Posters/pamphlets (4/9) • 6/6 student respondents passed an interview to attend leadership • 4/6 student respondents paid R500, 1/6 paid R200, 1/6 received a full scholarship

17. Does life skills ‘lay the foundation’ for leadership? Student perspectives: “It is like it’s the first step in changing life… [the] first step of a person who wants to change” (R1). “Life Skills developed, made me to see my vision, where I am going, where I am standing… and it make them to be clear. If I want to be a change, [I must first make] the change my life so that I can change other people’s life. That is when I started to be, and I would like to be a leader that will change other people’s lives” (R2). [During Life Skills] I’ve noticed that I was doing the wrong thing, like, like amabad relationships, the way I talk with others, the way, when I done something wrong, I tend to point fingers to someone… its helped me a lot, like to realise that I am responsible for my life” (R6). Staff perspectives: ‘”I think Life Skills is, does, provide a foundation, in that people come to the leadership with an understanding of their identity, who they are and their vision and their purpose, what they feel that they want to do in their lives… leadership then will build on top of what Life Skills has already laid “ (R7). “[Life Skills] more grills and builds up self-esteem, confidence. So when a person comes to a stage of discovery and self-realisation, that’s when a person is more equipped and more suited to, enrolin the leadership, because we believe that you cannot lead people if you are still broken yourself, so you need to deal with yourself first, because anything that you say has to emanate, emanate within you. It will be something that you have already gone through yourself, the transition that happened to you” (R9).

18. What are the key differences between life skills and leadership? Student perspectives: Life Skills highlights; leadership makes things clearer and goes deeper “Life Skills they just highlight things, they just take you step-by-step, but when you come to leadership, they go deeper and deeper… they need you to lose your mind, to think, not just think and make a decision, but think, think and think again, until you finally realise that it’s a decision that you really want” (R5). “Leadership is different, they are touching the real issues that can cause a human to not to succeed in life” R6 Staff perspectives: Leadership is community conscious “Leadership is more community conscious, and Life Skills is more personal. At a Life Skills level, we teach healing of the past, so that people will break free from whatever bondage or imprisonment that they are in, for life success, and in leadership, what we do is teach healing of the past, with the motive of them, imparting that to other people“ (R8). “We are introducing them to a world that says, even if yes, we are saying take care of yourself, but more so, are the people around you benefitting from what you are?... saying, I will be independent, but again my independency is an interdependent kind of a dependency because I am, so therefore people are, and they are, so therefore I am.” (R9), Students are introduced to God “Its where we introduce people to God, in leadership… you can have a vision, but if God is not part of that vision, if God didn’t give you that vision, that means, there isn’t much of a vision, you don’t have a vision, because you cannot have sight if God didn’t give you sight” (R9).

19. What components of leadership do students like/dislike? • Overall, students have very positive things to say about the leadership programme & WCA • Students liked the course structure (residential, practical, feedback) • Highlights include: • Healing of the past • Motivational speakers • Developing communication skills • Varied facilitation techniques • Making friends/building a support network • Outreach • Dislikes were: (*see also problems/issues & suggested improvements) • Students have personal issues which affect participation* • Too long spent watching movies • Need for computers & computer skills

20. Is the leadership programme being implemented as planned? Residential (Phase 1, 4 weeks) • Facilitators use a range of techniques to design/tailor the course to students needs • Motivational speakers are a key resource, but there are challenges finding/retaining high quality speakers • Other resources = books, movies, the Bible, CD’s, students & life; there is no manual • Leadership courses are run back-to-back throughout the year • Staff shortages mean the programme staff are over-stretched and can become stressed “This year, leadership was my secondary role. So if the students go out, I also go out, I co-ordinate [Life Skills], but over the past couple of months, it became my only role, because of shortages of staff… its either I’m co-ordinating [Life Skills], if I’m not co-ordinatingI’m a speaker in the Life Skills and still facilitating another leadership, because when one leadership goes, another comes. So there’s always people here for leadership… Its very hectic, it cracks your mind, you come during the week I won’t be smiling, as much as I like to… Today, I’ve gone already, I’ve gone to speak, when I come back at night I have to go and facilitate the leadership and I have to prepare for the class tomorrow, so it really becomes, really hectic” [R9]

21. Is the leadership programme being implemented as planned? Outreach(Phase 2, 6 weeks) • Students apply leadership skills practically via volunteering • Overall, students & staff find this component very worthwhile • Many students assist with Life Skills facilitation, when LS is not running they find their own placements • Students were volunteering in a wide range of places (YMCA, hospital, pharmacy company, KZN youth chamber of commerce) • Experiences varied; most students gained valuable skills; one placement was more linked to professional than leadership development • WCA staff were not able to visit all students on outreach • 2/6 volunteered for less than 6 weeks (shortest = 3 weeks) due to challenges finding placements/other issues • Students face financial & domestic pressures whilst on outreach, which helps to develop their resilience • 2/3 females encountered sexual harassment

22. Is the leadership programme being implemented as planned? Feedback(Phase 3, 1 week) • Students feed back/reflect on outreach and the course more generally • Facilitators draw the course together & motivate the students • Students graduate • Feedback was 2 days not 1 week due to WCA Christmas closure • One staff member critiqued feedback: “ You will find cases like this week, this week because we are closing down, they only came for 2 days, and there are cases where we have high schools, and they also come for two days also. So others will come for the rest of the week, which also is an indication that its, it’s a haphazard process, its not a well-structured process where first day this, and second day this happens, but its just more, its just activities that are not well structured” (R7).

23. Is the course being effectively monitored and evaluated? • M&E takes the form of: • Course evaluation for the lecture phase(???) • Student feedback forms(???) • Feedback phase (1 week) • Students could not describe M&E processes/tools other than the feedback phase

24. Are particular problems encountered or barriers experienced in implementation? Suggested improvements

25. In what ways are students developing/improving leadership skills in defined key areas? Identified key areas were (see logic model): • Hard work/diligence √ • Personal identity/taking a stand √ • Being a positive role model/exercising self control √ • Accepting responsibility √ • Leading others √ • Active in community/social issues √ • Communication skills √ • Empowering others √ Additional ‘areas’ which students/staff had developed in were identified via the interviews: • Being a servant leader √ • Forgiveness √ • Critical thinking √ • Computer skills √

26. Hard work/diligence “Sometimes I don’t even go to my lunch, my tea break, just serving patients. When its one o’clock they go to lunch, although they see patients’ sitting on the bench… the manager, the supervisor find me only one, for the whole hour, don’t go to the lunch just clicking those patients” (R3) “Sometimes I have to deliberately, you know, do stuff, do hard work, because I don’t want to be a hypocrite … I though my personality, I’m like , ‘oh lets go have fun’ you know, ‘lets wash dishes, arghh’ . So you can imagine the effort I had to put to washing those dishes. You know, so hard work and diligence, I think I’ve grown tremendously in those areas” (R8) Personal identity/taking a stand “Even now I can speak to the people about it [the rape], and even the person speak about it I didn’t care, because I know now who I am, I really know who I am” (R4). “I have a list of some, my printed list, values. So it helped me, cos before I wasn’t having and I didn’t know what values is” (R5). “Standing on, for what I believe in… it can be positive, it can be negative as well… I believe that the people who left, who resigned within the department, it was because of that, there are things that I couldn’t compromise on, I said ‘this is it, if you don’t take this, you can see what you can do’” (R10).

27. Being a positive role model/exercising self control “It was Friday, I used to go to the tavern, chill with friends, drinking. But on this day I go to the tavern, I sit down, I buy a cold drink, I put it on the table, I drank it, and they were so surprised, as ‘Hey…is not drinking alcohol, he is drinking Coca Cola’, from alcohol to Coca Cola [laughs], so they were surprised, by looking at me. I didn’t say ‘hey my friend, don’t do this, do that’, but by looking at me, they see that I have changed” (R1) “I was having five girlfriends and I slept with them… I decided that I have to go to the clinic, to know my status, and I don’t care, even if I’m positive, I don’t care, but at least I will know my status and know where I stand for the sake of other people. Then I go to the clinic and get tested, luckily I found out that I am negative. Then I started that I have to be responsible. I don’t have to do like having girlfriends. Then I decided that I have to have a right person at the right time “ (R1). “I was smoking drugs before, so I had a lot of friends who were smoking, as I am talking, they are still smoking. So as I was back at home, I realised that I have take responsibility for myself… I have decided not to take drugs like I was doing before” (R6). Accepting responsibility “My grandmother start send me to go and make grocery. Cos [previously] they were no more longer trusting me, they give me money for grocery, I would go drink alcohol and I would come back on Sunday, they send me on Friday, you know. And now they start trusting me” (R3). “It was a huge change, you know, from moving from the street, into a life where you now have guidelines and boundaries and completely are accountable to different people… from a situation where I was blaming and I was condemning and criticising and judging, where I actually saw my life and the problems that I had as, you know, as maybe the issue with my parents, and with my background. To a situation where I started to accept responsibility for my own actions, and said ‘my life is like this because I have messed up’ and so I need to work and put things back in place” (R7)

28. Leading others “They put me in charge [during outreach]… cos I was having a long time there then I was knowing everything how is operating, where is the file, it goes here and stuff, and how much money they pay per patients and those stuff. So yeah, I becoming a leader, you know leading people in a good way, they say I was leading people by doing it” (R2). “I have to be the one that motivates them, that ok, lets just go and attend, lets just make a meeting in this week, so that we can decide what we are going to do. I have to be the one who push them to do things” (R5). “Its only next year that I’m gonna take a formal role of leadership… I think I have the ability and the capacity to do that, but I’ve never practised that formally” (R8). Active in community/social issues ““We didn’t have a [Life Skills] co-ordinatorfor our area. So I thought, this would be a great opportunity for me to be involved in my community… I have been there for a long time, just sitting not doing anything, seeing the things that are going wrong, but still not doing anything about them… so that’s what motivated me to come and volunteer” (R9).

29. Being a servant leader “I believe in putting others before myself. Not neglecting myself, but I believe other people come first... we were talking about the queues when we go to fetch food, like ‘which one do you prefer to be in? The first one in the line or the last one in the line?’, and the others were debating, ‘why would you be the first one, [corrects], the last one in the line? you won’t get the best plate’, but I believe that being the last one in the line means there is a level of sacrifice, even if I don’t get the best, but if the people are happy, that’s more so what I’m about” (R9). Communication skills “Sis Precious said , ‘now it is an outreach report who will go first?’ I started, which means that I am not afraid anymore to be in front of people. It wouldn’t be a problem at anytime now to be in front of people talking” (R1). “I was a quiet guy before, but now I have starting to talk with some others people, like sharing ideas, like what we are going to do, like in future, what you like to do in future” (R6). “I can say the way how I can communicate, like even at home, like if I have a problem with my sister, we tend to, we like to fight, now I just tell him[her] ‘I don’t like that, just stop that’. If it doesn’t understand, I’m just leaving him like that… the communication skills helped me to understand others” (R6). “When I first came here, I couldn’t even speak, you know, I struggled to put words together and not to mention skills like computer skills and different things like facilitation, co-ordination, administration, I did all of that, within the organisation, slowly, you know” (R7).

30. Empowering others “That’s my job. I, had to grow in that area… I’ve developed tremendously as well, because I practise it every day. I have to break down mentalities and uplift and motivate and encourage at the same time, you know” (R8). Forgiveness/Healing the past “I learn even to forgive my mother for what she did to me. I learn even to forgive my father for what he did to me, although he has already passed away… I also learn even to forgive myself, for every bad things I have made” (R3). “I was having that guilty and I was having that grudge for them, I was holding everything in my heart. But from Life Skills I see that I mustn’t hold things for other people, because God create me.. ok that thing happen in those days, it hurts me very well, because I think I am a strong women now, I can stand for myself of what they did to me” (R4).

32. Findings • Development of leadership skills began during LS • Greatest improvement in Communication skills, identity/taking a stand, being a positive role model • Students found it difficult to provide examples of improvements in a number of areas • No change was cited in some areas (community/social issues, leading & empowering others) • Shallow vs deep examples

33. Do course facilitators/WCA staff model good/positive behaviours? Students generally very positive about leadership facilitators & WCA staff Approachable: “maybe during his or her lunch time and you said ‘I have a problem here’ and then they not just say ‘its my lunch time, I’ll see you later’ they just stay calm and discuss(R5). “they don’t look down to[on] you, like where you are coming from , or what kind of people, person you are, they just came here to help. It seems like they are coming, we are coming from the same background… they are great people” (R6). Good role-models “they preach what they practise” (R1). “they didn’t tell me things if they don’t do… they are people that have made a change to them, that’s why, those people have done a good job. And even the staff that are volunteers here are a good person” (R2). “I see anything wrong. I see the leadership, I see the leadership with them and I see the way they do things and I just wish to do like them” (R3). Minor complaints (contradictory) • Facilitators are too friendly with the students (x1) • Facilitator too formal (x1) Complaints about other WCA staff • Staff unfriendly (have not been through Life Skills)

34. Do course facilitators/WCA staff model good/positive behaviours? • WCA staff tended to have higher standards/judge themselves/ colleagues more harshly • Change is a long process (behaviour change takes time) • Emotions/misunderstandings/conflicts intolerance arise in the workplace • Leadership facilitators highly regarded amongst WCA staff • Facilitators face a steep learning curve (education & skills) • Living & working together places WCA staff under intense pressure • As does the high workload and • Staff shortages

35. What significant life changes do students attribute to WCA leadership programme? • Self-acceptance • Getting my life back on track • Attitude & behaviour change (alcohol, drugs, sleeping around) & foregiveness • Direction & values • Accepting responsibility • Becoming accountable • Spiritual consciousness • Developing a (self) culture of learning • Searching for God

36. Holding/moving into leadership positions • Starting a co-operative (student) • Selected to participate in youth parliament in Cape Town (student) • Moving into WCA management (staff x2) • WCA ambassador in USA (staff) Future plans • Family life • Start a CBO (x2) • Community/social initiative (x2) • Studying (x3) • Volunteering (x4) • Career/business plans (x4)

37. Recommendations: M&E • Initiate a proper course evaluation • Pick a few key indicators from the Logic Model & develop a simple M&E framework. • Monitor/track number of students enrolled as well as number of students completing WCA courses; calculate drop-out rate and identify key reasons why students drop-out. • Periodic follow-up with 10% of course graduates (selected randomly)

38. Recommendations: Leadership • Apply cost/funding criteria fairly (students eligible to apply for full/partial scholarship) • Develop a course framework & manual • Develop ‘guidelines’ for outreach; approve placements; establish/nurture relationships with suitable organisations; provide greater support • Consider assessment: Did students complete outreach? Can they apply learning? • Consider introducing an advanced leadership programme targeting community leaders; current programme does not target them & is making less impact in several key areas (leading, empowering, community/social issues)

39. Recommendations: Human resources • Consider dedicated position, focused on initiating & nurturing relationships with speakers • Leadership programme needs to be properly staffed • Staff need time off inbetween courses to prepare