Where do Operational

1. Where do Operational/Implementation Research questions come from? Soumya Swaminathan, MD Tuberculosis Research Centre Chennai, India

2. At the end of this session, you should be: More aware of the source of OR/IR questions Distinguish researchable and unresearchable implementation problems Know the criteria of a good OR/IR question Assess whether your research question is a good OR/IR question Finalize the research question which will be developed during the workshop

3. Link between Program Manager and Researcher OR is distinguished from other types of research by its focus on factors under the control of managers Successful OR requires close collaboration between researchers and managers. OR results should be immediately applicable to program problems. Need to think about inputs or interventions and outcomes � causal chain.

4. An ideal team� The manager�s role in OR is as, if not more important than, the role of the researcher The manager identifies the program problem to be researched, sets the research parameters, and is responsible for utilization of results The researcher is responsible for the integrity of the study Collaboration between managers and researchers is essential during all phases of the study

5. What is a Problem? OR begins with a problem to be solved A program problem is a discrepancy between actual and desired results Not all problems mean something is wrong with the program. Some problems are opportunities, e.g., �What is the best way to increase the size of this program to serve more people?�

6. Steps in analyzing the problem Clarify the viewpoints of all concerned and stakeholders List the problem Think of all the reasons why this could be so Example: Increasing defaulter Rate of TB patients Poor health services management? Social stigma with TB Negative attitudes of health workers towards TB patients?

7. Steps in analyzing the problem 2. Specify and describe the core problem Quantify the problem Describe it more elaborately Increasing defaulter Rate of TB patients How widespread is the observation? Which regions persistently? New areas which may be potential low compliant areas? Who is affected the most How severe is it � what are the consequences, e.g. increasing morbidity, deaths, waste of resources, development of multi-drug resistance?

8. Steps in analyzing the problem 3. Identify the factors that may have contributed to the problem, clarify the relationships between the problem and contributing factors Increasing defaulter Rate of TB patients Staff who are poorly trained because there are inadequate materials on TB health educations have little understanding of patients prescriptions and do not provide systematic advice and counselling to patients which results in patient not understanding treatment requirements and therefore a high default rate

9. Causal Chain � what is the model for expected change?

10. Thinking about the Solution or Intervention � Where do solutions come from? Program staff /Staff of other programs Clients Policies Literature review Researchers � Characteristics of a good solution: Under control of manager Has potential to make large improvement Effects can be measured Easy to implement - Affordable/sustainable

11. Identification of possible OR/IR questions Description of the health situation and , interventions � both in place and potential magnitude of the problem Distribution of health needs of the population Risk factors for some problems People's awareness of the problem Utilization patterns of services Cost-effectiveness of available and potential other interventions This information is required to formulate adequate policies, adapt or plan an intervention

12. Identification of possible IR questions 2. Information required to evaluate on-going interventions or access progress �..the need for making adjustments in the intervention coverage of priority health needs Coverage of target groups Acceptability of the services Quality of services Cost-effectiveness of the intervention Impact of the programme on health This information is required to assess progress and the need for adjustments

13. Identification of possible IR questions Analysis of possible causes for missed targets in order to find solutions Availability Acceptability Affordability Service delivery problems

14. Where do you get all this information? Local documentation � project progress report, theses, dissertations, seminar proceedings Programme progress or evaluation reports Medical literature including reviews which outline gaps in research Scientific meetings and conferences New ideas from previous research Funding agencies annual reports Questions asked by programme staff and even students Personal experience and observation

15. Is the Problem Researchable? Is there a discrepancy between actual and desired results? Do I need research to solve the problem ? Do I have enough time, money and qualified research staff ? In Operations Research the manager intervenes in inputs and processes to solve problems

16. What is a Researchable Problem? Non-Research � A discrepancy exists between desired and observed situation � We know why the discrepancy exists � We know the best solution Research A discrepancy exists between desired and observed situation We do not know why the discrepancy exists Different solutions exist but we do not know the best one

17. An OR/IR Question is a Program Problem Restated as a Question If I charge 1 dollar for a pap smear, will fewer women get pap smears? Will training community health workers increase vaccination coverage? Will a mass media campaign attract more women to deliver in hospitals? Can �loss to follow up� for early infant diagnosis be improved by using sms?

18. Do all problem situations require research? Problem situation: In District Y, (population 145,000) sanitary conditions are poor (5% of households have toilets) and diseases connected with poor sanitation such as hepatitis, gastroenteritis and worms are very common. The Department of Health has initiated a sanitary project that aims at increasing the percentage of households with toilets by 15% every year. The project provides materials and the population should provide labour. Two years later, less than half of the target was reached.

19. Do all problem situations require research? What is the discrepancy? What factors can explain this difference Service related factors? Failure to inform and involve the community? Bottleneck in the supply of materials? Training and effectiveness of sanitary inspectors? Population related factors? Lack of understanding of relationship between disease and sanitation? Poverty? Physical factors? Ecosystems? Hard soil? Area always flooded?

20. Criteria for Prioritizing IR questions Relevance How large or widespread is the problem Who is affected How severe is the problem? Potential for spread if not checked? Who considers this a problem? Burden to the health system? Economic impact on the population Avoidance of duplication Researched before? If yes, are there any major questions that deserve further research? Is my context so different that I cannot use results of previous research?

21. Criteria for Prioritizing IR questions 3. Timeliness (urgency of need for data) How urgently are the results needed by the policy makers, implementers and healthcare providers to change course, take on new interventions or stop what they are doing? 4. Political Acceptability Advisable to do IR of high interest and support of local/national authorities Increased likelihood for study results to be implemented Involvement of policy makers in the study

22. Criteria for Prioritizing IR questions 5. Feasibility Complexity of the research Adequate resources to do the study Is it feasible to conduct and report the findings in 12 to 36 months 6. Applicability of possible results or recommendations Likelihood of adoption of recommendations How would the findings be used to improve health and healthcare? Availability of resources for implementing the recommendations

23. Criteria for Prioritizing IR questions 7. Ethical acceptability How acceptable is the research to those who will be studied? Is the problem shared by the target group? Can informed consent be obtained from the research subjects Will the condition of the subjects be taken into account Will the results be shared with those who are being studied? Beneficial

24. Practice Stating an OR/IR Question Make a problem statement using the format: Will (increasing, decreasing, changing) this (input or process) improve (output, outcome or impact)?

25. Exercise Men in district Y have high levels of HIV. Condom use is also low. � Is this an OR problem? � Give two possible solutions

26. Main message OR/IR research questions should address the needs of the policy makers, programme managers and healthcare providers, not just satisfy the curiosity of academicians OR/IR questions should be identified through an analysis of the situation and evidence, not merely based on the "gut feeling" of the policy makers, programme managers or healthcare providers

27. Identify the OR Question We are having a cholera outbreak! Adolescents are having sex! What are ANC coverage rates in this population Will decreasing the frequency of supervision improve efficiency without reducing service quality?