2011 CAADP M&E :Data Response Analysis

1. 2011 CAADP M&E :Data Response Analysis ByRaymond Nkululeko Maseko Regional Strategic Analysis and Knowledge Support System for Southern Africa (ReSAKSS-SA)

2. Content • Introduction • Rational • Data Collection Process • Observations – Data collection & collected data • Results of Analysis • Suggestions

3. Introduction In June 2011 SADC country consultants were contracted to collect data for the purpose of Monitoring and Evaluating CAADP process; in particularly, progress made towards achieving the 10% allocation of national budget to agriculture and 6% growth in agricultural output.

4. Rational The main objective of the response analysis is to establish: • the overall response rate for all the SADC countries that collected data which are Angola, Botswana, DRC, Lesotho, Malawi, Mozambique, Namibia, South Africa, Swaziland, Tanzania, Zambia and Zimbabwe; • a response rate per question and section with a view to identifying gaps in the data; • which critical questions and sections are affected by gaps in the data;

5. Data Collection Process IWMI IWMI & Consultants Consultants Step 1 Finalise questionnaire preparation Step 6 Workshop Questionnaire Methodology Step 7 Develop Resource Schedule by Country Step 8 Setup data Collection Appointment Schedule Step 2 Normalise questionnaire (Format, questions, validation) Step 5 Issue an electronic Questionnaires and Checklist Step 10 Collect Data and Complete Questionnaire Step 9 Confirm Schedule Step 11 Perform High Level Data Validation, Complete Checklist and provide weekly status update Step 3 Develop Questionnaire Completion Checklist Step 4 Design Computer Database Structure Step 12 Carry out spot checks Step 16 Capture Data into a Regional Database Step 14 Submit electronic Questionnaire and Checklist Step 15 Project Coordinator Record Receipt of Questionnaires Step 13 Complete Checklist Step 17 Validate Captured Data Step 18 Update Checklist Step 19 Handover Database to Analysts

6. Observations on data collection and collected data • Most country consultants outsourced collection of data or submitted requests to various government departments to fill out the questionnaire; • In some cases there is no evidence to suggest that the questionnaire was thoroughly discussed with subcontractors or departments that were requested to complete the questionnaire or sections of the questionnaire; • Not all countries responded to all spot check issues that were raised with them. In fact some consultants choose to address the data issues in their country reports; • There is no evidence to suggest that some countryconsultants checked data before it was submitted to IWMI Project Co-ordinator; • When country consultants presented their draft reports during the workshop, most of the reports were not based on collected data but a different data source; • Questions that were asked by some country consultants during the second data workshop suggested that either the questionnaire was not clear or there was a communication breakdown / problem; • Some country consultants could not explain some of the ambiguities in the data because it was transcribed from source as is and without any explanation; • It is not clear: • if data is not available; or • at source it is not stored / collected in a manner that can easily relate to the way questions are structured in the questionnaire; or • there is inadequate skill to extract data in the manner it is required on the questionnaire.

7. Impact of gaps in the data It is not possible to produce comprehensive combined regional statistics for meaningful analysis and the table below is one of the example

8. The percentage in each column represents available data measured against an expected 100% response rate for each indicator. This table must be read in conjunction with indicator bar charts showing gaps in the data

9. Extract from country questionnaire • Specify Calendar Year: __________ or Fiscal Year from: month __________ year________ to month __________ year________ • Please note: All monetary values should be in the Local Currency Unit (LCU). In case an alternative currency is used, please state explicitly. Agriculture is defined to include crops, livestock, fisheries (captured and farmed) and forestry. • B1. OVERVIEW OF REVENUES AT NATIONAL LEVEL Note: Tax revenue includes for example taxes on income and profits, payroll and workforce, domestic goods and services, taxes on international trade and transactions as well as stamp duties and fees Note: Specify currency ___Millions USD______________________________________________ in: Thousands (1,000)  Millions (1,000,000)  Billions (1,000,000,000) 

10. Spot check report B1. OVERVIEW OF REVENUES AT NATIONAL LEVEL

11. Database Extract

12. B1. OVERVIEW OF REVENUES AT NATIONAL LEVEL

13. The percentage in each column represents available data measured against an expected 100% response rate for each indicator. This table must be read in conjunction with indicator bar charts showing gaps in the data

14. The percentage in each column represents available data measured against an expected 100% response rate for each indicator. This table must be read in conjunction with indicator bar charts showing gaps in the data

15. The percentage in each column represents available data measured against an expected 100% response rate for each indicator. This table must be read in conjunction with indicator bar charts showing gaps in the data

16. Suggested Data Sources

17. Q & A