Water Implementation Projects In Rural North India: Evaluating implementation “on the ground”

1. Water Implementation Projects In Rural North India: Evaluating implementation “on the ground” Katharine Owens, University of Hartford, Politics and Government Marcia Hughes, University of Hartford, Center for Social Research

2. Overview of Presentation • Project efforts to date • Evaluation, implementation, and collaboration • Assumptions of this evaluation work and inherent challenges of project efforts • Evaluation model and methodology: Contextual Interaction Theory • Analyses, findings, and uses of the model • Recommendations and conclusion

3. Projects in Abheypur, India • University of Hartford student-faculty service-learning teams • Professional and student chapters of Engineers without Borders • Interdisciplinary projects: engineering, social science, and education/outreach projects focusing on water.

4. “On the ground” in Rural India Abheypur Delhi Gurgaon Abheypur Pathways

5. Abheypur, India Project Team Abheypur, India

6. The Projects • January 2008: Installation of a solar powered well pump and water storage system • August 2008: Extension of the water supply to a poor section of the village • January 2009: Implementation of a rainwater-harvesting system • October 2010: Installation of a potable water supply well and water storage tank for poorest community January 2010: Installation of a grey water drainage improvement project for the “Potter’s Village” • January 2011: Installation of a grey water drainage improvement project on main road of the village

7. Abheypur, Haryana October 2009 tank location • Well • Tanks • 2 tanks • Piped to girl’s school 2010 Soak Pit 2011 Soak Pit • Solar panels • 2 wells • 4 tanks • Rainwater harvesting

9. Main Road Drainage Problem

11. Gate Gutter: 6” wide by 6” Deep Brick Wall New Drain Driveway Continue Drain Across Road Drain Covered by Grate to Allow Vehicles & Carts to Pass New Drain Right-of-Way No Gutter – Grade so swale takes water along side of road Soak Pit Private Farm About 3% grade from west side of road to soak pit W N S E Design for Soak Pit Implemented in 2011

12. Potters’ Village Drainage Problem

13. BEFORE

14. Potters’ Village - Soak Pit Design Jan 2010 Two layers (6”) of bricks are used for soil structural support to keep soak pit open. Thus, excavated diameter is 1’ wider and longer than internal dimensions listed. Forebay Soak Pit 4’ 4’ Overflow to swale during for rain events Greywater Inflow 4’ Stones and rocks to reduce water velocity Brick Weir (Width = 3”) Forebay Depth = 1.5’ Soak Pit Depth = 4’ Forebay and Soak Pit covered with slate

15. AFTER

16. Implementation • When political ‘actors’ desire change in society, they develop and implement programs to address a perceived problem. • Researchers focus on the implementation stage of the policy process as a point where intentions and outcomes diverge • Two different processes: Developing a program/policy versus implementing a project/program/policy

17. Merging evaluation and implementation • Program evaluators seek to quantify and catalog the successes and failures of programs and policies. • Implementation policy analysts evaluates this stage in the process to better understand participants’ roles and any impediments to change. • We seek to infuse evaluation with implementation analysis methodology.

18. Merging the two • Evaluation researchers find it’s not enough to know whether outcomes are achieved, it’s important to understand how. • Using an implementation analysis tool in evaluation can answer questions about how people come together and work towards achieving a project (or fail to). • Scholars call “for responsive, contextual, flexible, adaptive, multidisciplinary, and mixed-methods approaches,” particularly when considering environmental programs and policies (Preskill, 2009, p. 99).

19. Collaboration is an imperative(Gadja, 2004) • Being solo does not work for most of the problems that need addressing • Need shared efforts to achieve goals that would not otherwise be attainable working independently. • More and more, collaboration is becoming the method for addressing complex social problems

20. Understanding collaboration Wostl et al. (2007) – In managing water resources in particular, a paradigm shift is needed, one that recognizes the importance of stakeholder involvement and collaboration. Water: communal and imperative

21. Understanding collaboration • One goal: To develop communities of practice. • Would we recognize if we were not working collaboratively with our Indian partners? • Should we trust our own conceptions, or evaluate the process to confirm our goals of collaboration?

22. What does collaboration look like?The Practice of Collaboration Gajda and Koliba (2007) describe key characteristics of interpersonal collaboration to be: • a shared purpose • dialogue • decision-making • action, and • evaluation (p. 29).

23. Water Implementation Projects in India: Inherent Challenges • Working across cultures • Across languages • Across disciplines (engineers and social scientists) • Across class (American middle-class and Indian impoverished villagers) • Caste distinctions remain present in the village • Gender distinctions (American women in positions of leadership, which may be atypical) • There is short period to build trust among partners, implement new projects, and manage and evaluate previous projects.

24. This evaluation work • We work to accomplish project implementation as well as outcome evaluation in India. • We find it important to engender true collaboration with India partners. • We use an implementation analysis theory to evaluate these water projects.

25. Contextual Interaction Theory • finds project participants and other stakeholders can influence implementation in critical ways. • analyzes collaboration between participants based on motivation, knowledge, and power, to shed light on how projects are implemented. • produces a prediction about interaction based on these core participant characteristics

26. Contextual interaction Theory • allows the emphasis on implementation (the how) as called for by evaluation researchers. • can provide evidence to those “on the ground” about their effectiveness in implementation. • allows large-n studies that enable researchers to find patterns in large data sets and enables multi-year studies.

27. Contextual Interaction Theory • The motivation, knowledge, and power of the implementer and target are the focus of the assessment. • Implementer: the student-faculty team, and a loose coalition including Navjoti, and the professional Engineers without Borders chapter • Target: local villagers, including those residing near the project and those taking an interest in the implementation of the project.

28. Research Questions • How does motivation, knowledge, and power among the different players affect implementation of the project? • What is the relationship between these characteristics among the people ‘on the ground’ and the outcomes of the implementation process? • How is the project plan or design implemented through collaboration processes?

29. Methods • Faculty-student trip to Abheypur, India in January 2010. • 14 interviews using a semi-structured interview instrument. • Questions investigate the core elements of the theory (motivation, knowledge, and power) • We asked slightly different questions of the American and Indian counterparts as reflected their roles in the process.

30. Independent variables • Motivation • Knowledge • Power

31. Dependent variables • Active cooperation • Passive cooperation • Forced cooperation • Opposition • Obstruction • Joint learning • No interaction • Policy learning

32. Analysis [1] M scale=((proportion)-.50) *2, scale of -1.0 to +1.0 [2] I and P scales= (proportion), scale of 0.00 to 1.00 [3] n/a : participant did not answer questions on this concept.

33. Analysis

34. Qualitative assessment • Implementers have some doubts about the work, their role, and the role of the target participants in the process. "…but you know sometimes I wonder maybe there might be actually better things we could be doing with our time." • "I think [with] this project we weren't communicating with them before we got there, they didn't know our plan.” • New understandings of caste subgroups and inter and intra group dynamics

35. Analysis • What is the potential for other scenarios to emerge? • The strong imbalance of power does not come into consideration when both actor groups are in favor of a project. • But how might this impact implementation if one actor was no longer in favor of implementation?

36. If the target lacked motivation

37. If the implementer lacked motivation

38. Evaluating our collaboration

39. Recommendations (February 2010) • Community-driven projects based on a formalized decision-making process. • establish a shared purpose/role among EWB interdisciplinary team • Always consider the caste/subcaste/gender/poverty issues and related power/resources, knowledge, and motivation among different groups

40. Recommendations (February 2010) • Make collaboration a goal • Are all stakeholders involved? • Is there a shared purpose and understanding of the project? • Is there equal decision-making? Has the community been involved and/or surveyed for their feedback? • Is there local leadership for the project (i.e., in the village)? • Expect and plan for disagreement and conflict.

41. Recommendations (February 2010) • Streamline the collaborative process • Develop project ideas with community leaders and Navjyoti. • Navjyoti can survey/scan community for input/feedback on project ideas. • Navjyoti can serve as link between EWB and village and district leadership (i.e., for communicating project design development, and planning). • Fine tune plan and develop budget-tasks and responsibilities (prior to starting project).

42. Recommendations (February 2010) • To make previous projects sustainable • Monitoring • Planning • modifications • Monitor systems to ensure proper use and satisfaction with • system(s), • community leadership, and • overall quality of water

43. Conclusions • It is both a goal and an obligation to work in a collaborative way with local groups in India. • Using the theory we can gain insight into not only the outcome of implementation (cooperation), but also provide context for how and why participants are cooperating.

44. Conclusions and Next Steps • Our analysis also sheds light on how the people involved in this project influence it. • It allowed us to set meaningful goals for collaboration and partnership • Next Steps: Analysis of 2011 data