Evaluation: GRANITE Project

1. Evaluation: GRANITE Project Lucknow, December 19, 2006 Presented by: S.V.Divvaakar and Pankaj Agrawal Ace Global Private Limited, New Delhi

2. Presentation Outline • Evaluation Objectives • Approach and Methodology • Evaluation Frame • Key Findings under evaluation themes • Conclusions and Recommendations

3. Evaluation Objectives • To assess the overall results and impact of the project at national and sub-national level • To assess the need for second phase of the project • To provide inputs for design of Phase 2

4. Approach and Methodology • Briefing meeting and participation at inaugural event of KIC initiative at Jaipur • Document perusal and secondary research • Field visits in 4 states: Tamil Nadu, Andhra Pradesh, Maharashtra, Orissa • Interviews with • SRG members • Media personnel • Civil Society Organisations • Visits to selected locations of Reachout meetings, and discussions with participants

5. Approach and Methodology • Interview visits also to Kolkata and Jaipur - NCU and CART, and telecon with Karnataka state partners • E-mail questionnaires to all state partners

6. Evaluation Scope • Five classical elements • Relevance • Efficiency • Results: • Effectiveness • Outcomes/Impacts • Sustainability • External challenges and constraints

7. Evaluation Frame: Definitions

8. Evaluation Frame: Relevance

9. Findings on Relevance • Project is highly relevant to the setting. • A large share of the population lives marginalised from India’s economic growth, and is engaged in informal employment, in agriculture and artisan trades. • Benefits of economic growth have not percolated to the weaker sections, for various reasons, incl: • Lack of awareness of trade and economic issues that directly or indirectly affect their livelihoods • Mainstreaming these stakeholders is a key challenge for India’s economic growth and social development. • Improving economic literacy at grassroots level will enable stakeholders to see the links between their livelihoods and economic developments, including those related to trade and globalisation.

10. Findings on Relevance • Sectors are well-chosen: • Agriculture and textiles are the two largest employment/ livelihood sustenance sectors in India. • Agriculture accounts for 23% of GDP, and 10% of exports • Textiles and Clothing is the second largest employment creating sector • More than 2 million households are engaged in handloom weaving. • Trade and globalisation especially WTO agreements have impacts on both sectors. • Market access constraints for agriculture exports • Elimination of market access preferences in handlooms • States: • Regional balance • The eight states selected for the project have sizeable agriculture sectors, and some have sizeable handloom textiles sectors. • Three states have very low export intensities in the sectors.

11. Findings on Relevance • State Partners are well-chosen: • Grassroots presence • Convening power and credibility • Some have a history of partnership with CUTS • However, varying levels of conversancy on sector-specific livelihood issues • Most have limited exposure to trade and globalisation issues, first exposure for some… • Limited budgetary resources

12. Findings on Relevance • State Consultative Mechanisms: • Mechanisms for government consultation with grassroots stakeholders either do not exist or are dysfunctional. • WTO Cells have the mandate to represent issues directly related to trade and globalisation but are not carrying out the mandate due to lack of resources and clarity • There is a need for enhanced advocacy based on • documentation of grassroots impacts/experiences, • increased articulation of concerns

13. Evaluation Frame: Efficiency

14. Project Activities • Project used a wide range of tools for its actions. • Capacity building- training of implementation partners and media persons • Reach out meetings with final beneficiaries and stakeholders, and state level workshops • Information dissemination: website, news letters, research reports. • Advocacy with state and centre government

15. Project Activities • National launch and training seminars • State Project launch meetings • State Reference Groups • Reachout meetings • Media workshops • State level workshops • Advocacy actions through TPCs, WTO Cells • Briefing papers • Research document • Newsletters • Advocacy document • Website

16. Launch and Preparatory Activities • National Launch and Training seminars • 1st Seminar, Jaipur 4 days • To provide training on globalisation/ WTO subjects, based on base line questionnaire analysis • 2nd Seminar, Kolkata 2 days • Originally to focus on skills to analyse and articulate sector level issues • Lack of continuity in participants • State Project launch meetings: one day panel discussion, 20-100 participants • State Reference Groups: 10-108 members • Original role - to serve as sounding board and monitor qualitative aspects. • However, no budgets for structured activities

17. Field Activities - Reach out meetings • Meetings • 5 outreach meetings planned per state • 3 states have held 7 meetings each • 3 states have held 4 meetings each • 20-350 participants (U.P, A.P, W.B with most) • No textiles coverage in Maharashtra, Orissa and Karnataka- conscious decision with NCU • Lack of product/cluster/issue specific focus especially in agriculture sector led to many generalised events, with little direct connection to livelihood issues for participants. • Non-homogenous groups attended most meetings. • Main issues identified in outreach meetings have a domestic dimension- remunerative pricing, cost of inputs, lack of extension services, exploitation by middlemen and money lenders, unfair competition from organised sector, etc. These were not, always, international trade aspects. • In some cases, there was good scope for advocacy actions addressed at local authorities…

18. Field Activities- Reach out meetings • In some locations, specific actions have been undertaken which enhance livelihood prospects for the beneficiaries: • Maharashtra: pomegranate crop protection, EurepGAP certification • Uttar Pradesh: Market interventions in mango; GI registration for chikan kari,etc. • Main issues identified in outreach meetings have a domestic dimension: remunerative pricing, cost of inputs, lack of extension services, exploitation by middlemen and money lenders, unfair competition from organised sector, etc.

19. Field Activities- Media Workshops • Originally two workshops planned, 1 per year • No media workshop in Orissa, 2 in A.P, U.P • Objectives: orientation and training of media persons on WTO issues, to ensure enhanced reporting • However, these seem to have been diluted, and several workshops were very generalised and unstructured, and lacked specific training content as such • Media persons’ interest in training is suspect, but there has been willingness to publish articles contributed by GRANITE partners. Good media coverage of GRANITE in all states. • Media workshop budgets underutilised, due to low participation by outstation persons

20. Field Activities - State Level Workshops • Originally two workshops, 1 per year • Objectives: better targeting, wider reach out and networking • Not held in 3 states: Tamil Nadu, Maharashtra, and Orissa; while U.P held two workshops. • Lack of clarity on objectives reported by many partners; also low credibility before govt officials • Focus of workshops not in line with the project document, and too general. • Sector-level workshops proposed in 2nd year. Not held in many states. • Sequencing of state level workshops not adhered to.

21. Field Activities- National Seminars • Two seminars, one at end of each year. First event in Nov 2005, Jaipur, took stock of lessons, issues and problems faced by state partners. • Second seminar: December 2006, Lucknow, this one… • Need to document collective learnings and develop advocacy messages for policy makers

22. Dissemination Products • Briefing papers • Two briefing papers produced, two more planned (December 2006?). • Not used as resource materials for reachout meetings or media workshops. • Poorly distributed in most states. • Newsletters • Quarterly issues, 2000 copies in English, 1000 in regional languages; i.e. 8 issues, 10,000 copies • 6 published in English, only 3 to 5 in states. • Poor levels of distribution (only 100 copies in English…) • Contents appreciated by recipients incl. SRG members • Research document- released at WTO Hong Kong meet • Conclusions are debatable. • Advocacy document- yet to be prepared • Website- useful information on project; updates needed.

23. Gender dimensions • Project management team had atleast one lady member (NCU and 7 states) • Special efforts to mobilize participation by women stakeholders in reachout meetings • Representation on SRGs

24. Advocacy Actions • State Trade Policy Councils/ WTO Cells • Idea of STPCs dropped in view of existing WTO cells in many states. • State of WTO cells highly unsatisfactory • Varying levels of interest in the project • National Trade Policy Council • Inter State Trade Council formed following active representation by CUTS, but not attributable to GRANITE as such. • Representations on the Foreign Trade Policy • Pro poor changes in the form of Focus Products and Focus Markets incorporated in 2006, again on CUTS representation, but not flowing from GRANITE activities as such.

25. Efficiency- Adequacy of Resources

26. Efficiency- Financial • Total Project budget: Rs 14.33 million • Budget for each state: Rs. 1 million • Budget for centralised expenditure by NCU: Rs. 6.36 million • 44% of total budget • Allocation for each activity specified in the TORs

27. Project Budget and Distribution

28. Project Budget and Distribution

29. Efficiency- Disbursements • Initial disbursements on time • Subsequent disbursement have been slow • Only 2 instalments had been received by some partners, with more than 6 month delays • Delays mainly due to non-completion of all previous period activities • Absence of flexibility to disburse according to state of completion under individual heads • Procedures can be improved in Phase 2, to provide for disbursement to continue all on-schedule actions

30. Efficiency- Cost-effectiveness • Overall, actions have been cost-effective • The project has influenced close to 5,000 beneficiaries, @ Rs 2860 per head; even a Rs. 100 / month increase in livelihood means/ incomes justifies the expenditure. • Unit costs for personnel and events are very reasonable. • However, budgets for advocacy networking and representation are concentrated on international travel – considered sub-optimal, inappropriately oriented.

31. Efficiency- Administration & Coordination • NCU has undertaken role of a Mother Unit, and taken a lead in some actions • There has been backstopping for state partners in several areas. • Some partners expressed that NCU should have provided more training content support and integrated the lessons from outreach meetings into learnings at project level. • Project financial and administrative reporting to donors has been timely

32. Factors affecting implementation • Internal Factors • Staff constraint/turnover • State partners lacking in-house expertise / sector conversancy • External Factors • Inadequate engagement with Government bodies and policy makers • Limited interest of local media • Elections in Tamil Nadu: delay in State level Workshops • Heavy rains affecting participation in State launch workshop in Maharashtra

33. Evaluation Frame: Effectiveness

34. Effectiveness- outcomes • Most target results have been achieved. • Out of 5 target outcomes, evaluators consider one to be an output and not an outcome. • Of the remaining four, there has been partial success in one, and substantial success in three, of which one is not considered attributable to the project itself, although achieved by CUTS. • Project needs more specific indicators for results, which are measurable at the level of beneficiaries.

35. Effectiveness- Target Outcomes

36. Effectiveness - Other outcomes • Capacity enhancement of partners • Increased articulation by grassroots stakeholders • Development results for beneficiaries: • Enhancement of livelihood opportunities

37. Evaluation Frame: Impact

38. Potential Impact • Realisation by grassroots stakeholders that their livelihoods have important linkages with domestic policies and international covenants. • Multiplier effect of the experience gained by partners in creating a stakeholders’ forum for analysis, articulation and advocacy of grassroots concerns

39. Evaluation Frame: Sustainability

40. Sustainability • Financial sustainability difficult for some partners, as GRANITE accounts for large share of their budgets • Concerns as to continuity of key persons after project support is withdrawn • Institutional structures created by the project- SRGs, news letters etc. can still continue with support from CUTS • Mainstreaming of agriculture and textiles in partner’s core activities

41. Key gains from the project • Reachout meetings acknowledged as an effective forum for grassroots stakeholders • Tangible and attributable benefits for stakeholders through increased awareness and understanding of trade issues • Increased reporting in local media on trade and WTO issues • Initiation of interface between grassroots stakeholders and the policy makers

42. Key gains from the project • Enhanced capacities of state partners in understanding, analysing, articulating and advocating on issues related to trade and globalisation/WTO issues • Strengthening of network of civil society organisations that interact and share research and capacities /skills on issues of globalisation

43. Conclusions and Recommendations • Project Emphasis: • Grassroots stakeholders are rather remote from trade and globalisation, and their livelihoods are linked more to domestic trade issues than WTO. Project title tends to mislead. • Amend the project emphasis to ‘Trade, Markets and Livelihoods’, which gives it a broader scope, including globalisation.

44. Conclusions and Recommendations • Sector Scope and coverage: • Agriculture and textiles are large, complex sectors with several products/segments, all not having similar pro poor dimensions. There are diverse issues relating to trade in each product/ segment. • Narrow the focus of the interventions to a few products common to most states, and a few cross cutting themes such as: • Market Structures • Remunerative Pricing and • Producer Empowerment

45. Conclusions and Recommendations • Sector Scope and coverage: • Project partners do not have adequate level of expertise or skills in both sectors, which skews the thrust of actions across various states. • Partners should have the flexibility to select only one sector, based on conversancy • More than one partner may be inducted in such cases • Narrow the skills gap by inducting sector specialists to support implementation partners

46. Conclusions and Recommendations • Delivery and execution: • Sequencing of events has not been followed in some states, resulting in sub-optimal use of project resources. Contents at many meetings were too general and not relevant or specific to local stakeholders • There should be rigid adherence to the logic of progression: outreach meetings - media workshops – state level workshops • Adequate prior preparation should precede field actions, enriched by SRG and external experts • One outreach meeting is inadequate to achieve any meaningful transformation at the grassroots level • There should be at least two outreach meetings per location, including a follow up at the end of the project

47. Conclusions and Recommendations • Delivery and execution: • Outreach meetings tend to have heterogeneous participants, covering several products and sectors, with few issues in common • Locations and themes should be selected to ensure homogeneity of stakeholders and also issues to be discussed in outreach meetings • There should be an integration of learnings from various meetings, facilitated by experts and the NCU • Media and government stakeholders workshops did not focus on capacity development specifically, nor was it in demand • Project should not fritter scarce resources on these groups considering their other sources of support, and focus only on advocacy • Advocacy should be bottom- up, focused at local levels of government as well

48. Conclusions and Recommendations • Delivery and execution: • Involvement of SRG resource persons was minimal, due to budget constraints • A small inner list of SRG persons should be inducted to act as a sounding board and for monitoring quality of delivery of the project.Budgets for their involvement should be included

49. Conclusions and Recommendations • Budget allocation and utilisation: • Overall, budgets should be enhanced for outreach meetings, intra-state travel and remuneration to key persons • Fungibility in budget items should be allowed within limits, based on specific justification

50. Conclusions and Recommendations • Capacity enhancement: • Focused training on sector-specific trade and globalisation issues could not be provided. • Formal training on sector-specific trade issues should be included in the project. Specialist resource persons should be inducted to bridge knowledge gaps among partners. Some examples already seen in Phase I. • Second outreach meeting should be used for identifying advocacy actions in consultation with stakeholders • SMART (specific, measurable, attributable, realistic and time-bound) indicators should be developed for project actions.