Baseline surveys and impact analysis M.M. Escalada International Rice Research Institute Los Ba ñ os, Philippines

1. Baseline surveys and impact analysis M.M. Escalada International Rice Research Institute Los Baños, Philippines

2. Baseline survey Understand farmers’ needs and circumstances • Identify farmers’ perspectives • Find out the root cause of farmers practice • Determine social, economic, ecological factors that may influence choice of options • Identify key stakeholders

3. Uses of baseline survey • Set the research agenda • Test research hypotheses • Design extension strategies • Evaluate effectiveness of intervention

4. How to conduct a baseline survey • Identify the problem or issues • Develop survey objectives • Develop survey instrument (questionnaire) • Pretest the questionnaire • Choose survey respondents • Implement field survey • Code and analyze data

5. 1- Identify the problem or issues • Needs of a given organization or ministry • Research priorities • Gather information to identify research priorities

6. Using theoretical frameworks to examine belief and attitude Human behavior is guided by: • Beliefs about the likely outcomes of behavior and evaluation of outcomes (behavioral beliefs) • Beliefs about normative expectations of others and motivation to comply (normative beliefs) • Beliefs about the presence of factors that may facilitate or hinder performance of behavior • Perceived power of these factors (control beliefs)

7. Theory of Planned Behavior

8. Theory of Planned Behavior As a general rule, the more favorable the attitude and subjective norm, and the greater the perceived control, the stronger should be the person’s intention to perform the behavior in question.

9. Sample TpB questions • Intention How true is this? Next season, I will use the leaf color chart for N management in my field. ____ 1) Definitely not true ____ 2) In most cases not true ____ 3) May be true ____ 4) In most cases true ____ 5) Definitely not true

10. Sample TpB questions • Attitudes towards behavior How true is this? Using the leaf color chart for N management will increase my yields. ____ 1) Definitely not true ____ 2) In most cases not true ____ 3) May be true ____ 4) In most cases true ____ 5) Definitely not true

11. Sample TpB questions • Normative beliefs How true is this? High yield is important to me. ____ 1) Definitely not true ____ 2) In most cases not true ____ 3) May be true ____ 4) In most cases true ____ 5) Definitely not true

12. Sample TpB questions • Subjective norm How true is this? My neighbor expects me to use the leaf color chart for N management. ____ 1) Definitely not true ____ 2) In most cases not true ____ 3) May be true ____ 4) In most cases true ____ 5) Definitely not true

13. Sample TpB questions • Perceived behavioral control How true is this? Using the LCC as a decision guide for N management makes me feel in control of my rice crop. ____ 1) Definitely not true ____ 2) In most cases not true ____ 3) May be true ____ 4) In most cases true ____ 5) Definitely not true

14. 4- Pretest the questionnaire • Interviewing a small group of respondents to determine their reactions to draft questionnaire • clarity of wording and translation of technical terms • logical sequence of questions • adequacy of response categories • clarity of questionnaire instructions • estimate duration of interview

15. 5 -Choosing respondents • Standard social science sampling methods • multi-stage • stratified • systematic • cluster • simple random • Choice of sampling technique • nature of problem • cost and time factors • desired precision or reliability

16. 6 –Implement field survey • Select respondents • Establish rapport • Write down responses accurately • Edit responses well • Close interview

17. 7 – Code & analyze data Simple analysis • Determine % of respondents giving specific answers • List various ways in which farmers might use new practice Complex analysis • Use statistical package to encode, process and analyze data Choice of analytical software • Access/cost • Ease of use • Power

18. Impact analysis An approach which measures the outcomes of an intervention • Did the program have the desired effects on individuals, households, institutions and communities? • Can those effects be attributed to program intervention?

19. What is impact • Producing 5,000 posters & 10,000 leaflets • Running a daily radio broadcast • Training 500 farmers on use of leaf color chart (LCC) for N management • Conducting 1,000 farmer field schools • Distribution of LCC to farmers

20. What is impact

21. Impact analysis • If a farmer is trained on the use of LCC for N fertilizer management and his rice yield increases, can we say that it is the direct result of the LCC training?

22. Why conduct an impact evaluation? • How did the project affect the beneficiaries? • Were any improvements a direct result of the project? • Could the program design be modified to improve impact? • Were the costs justified? • Are the resources well spent?

23. Impact assessment design Quantitative approach • Use experimental and quasi-experimental designs • Treatment and control groups • Control groups are selected at random from same population as program participants • Control group does not receive program or intervention

24. Impact assessment design Qualitative approach • Focus on understanding processes, behaviors and conditions as perceived by individuals being studied • Uses relatively open-ended methods in design, data collection and analysis • Provide critical insights into beneficiaries perceptions about value of programs • Can provide a better understanding of stakeholder perceptions, priorities and conditions that may have affected program impact

25. Weaknesses of qualitative approach • Subjectivity involved in data collection • Lack of comparison group • Lack of statistical robustness, given small samples • Lack of generalizability to a larger population

26. Weaknesses of qualitative approach • Validity and reliability depend on methodological skill, sensitivity and training of evaluator • Data collected may be misinterpreted if field staff lack sensitivity to social and cultural norms and nonverbal messages • Impossible to determine causality of project impact, without a comparison group

27. Choosing an impact assessment method • Experimental designs are considered the best approach • Before-after control group design or “with and without” and “before and after” are strongly recommended • Baseline data will allow one to validate integrity of treatment and comparison groups, assess targeting

28. Data collection techniques • Case studies • Focus groups • Interviews • Observation • Survey • Document analysis

29. 1- Case study Collecting information that can be descriptive or explanatory and can serve to answer the questions of how and why 2- Focus groups Holding focus group discussions with stakeholders who are familiar with pertinent issues before developing structured questions

30. 3- Interviews Asking questions of one of more persons; interviews may be formal or informal, face-to-face or by telephone. 4- Observation Observing and recording situation in a diary; includes who is involved, what happens; when, where, and how events occur.

31. 5- Survey Developing a set of survey questions whose answers can be coded consistently 5- Document analysis Reviewing documents such as records, training materials, administrative databases, and correspondence

32. Impact analysis • Impact is creating change. • Projects have to be planned to create impact. - baseline surveys conducted - treatment and control groups designated

33. Thank you