Overview of operational research in MSF

1. Overview of operational research in MSF Myriam Henkens, MD, MPH International Medical Coordinator MSF London 1st of June, 2006 Good morning ladies and gentlemen To introduce the session on operational research, I was asked to present an overview of the MSF operational research activities.Good morning ladies and gentlemen To introduce the session on operational research, I was asked to present an overview of the MSF operational research activities.

2. MSF in 2004 in 78 countries OPD patients 9,858,426 IPD patients 327,529 Malaria patients 1,862,691 Patients in HIV programmes 113,899 Deliveries 73,223 Surgeries 43,911 TB patients 14,109 Vaccinated for measles 964,482 Vaccinated for meningitis 451,254 To set the scene, I will first give you some information on MSF operational activities based on aggregated data from 2004. As you can see from these figures, in 78 countries, MSF has been taking care of close to 10 MIO patients in outpatients departments and more than 300 000 have been hospitalised. Close to 2 Mio patients have been treated for malaria, and more than 100 000 have taken care of for HIV or AIDS. Less than 15000 TB cases have been treated and close to 1 mio have been vaccinated against measles, mainly in response to an outbreak. To set the scene, I will first give you some information on MSF operational activities based on aggregated data from 2004. As you can see from these figures, in 78 countries, MSF has been taking care of close to 10 MIO patients in outpatients departments and more than 300 000 have been hospitalised. Close to 2 Mio patients have been treated for malaria, and more than 100 000 have taken care of for HIV or AIDS. Less than 15000 TB cases have been treated and close to 1 mio have been vaccinated against measles, mainly in response to an outbreak.

3. MSF presence by contexts -2004 Looking at the context in which we are operating, most of our activities are taking place in unstable area, (during armed conflicts or in a post conflict period, or in situation of internal instability) Only 41 % of our projects are implemented in stable context. Looking at the context in which we are operating, most of our activities are taking place in unstable area, (during armed conflicts or in a post conflict period, or in situation of internal instability) Only 41 % of our projects are implemented in stable context.

4. Why does MSF do research? To understand the local situation, assess the severity of an emergency, and to focus activities To gather data in a organised way to strengthen advocacy To monitor programme outcomes and identify constraints To adapt treatment protocols to operational constraints The results of nutriotional survey allow usto quantify the needs and the number of persons in needs, to precise the intervention strategy (blanket feeding, therapeutic feeding in centres or at home, etc) and to better target the intervention to the most in need. Retrospective mortality data describes the gravity of a situation and are often used for advocacy purpose to drew attention ot an ofteh forgotten catastrophic situation. In was the case in Darfur in 2004 A example of monitoring of programmes outcomes will be given later today (in the case of TB treatment in Chechnya) Comparison of strategies have been done for example on the comparison between home delivery of TB treatment and DOT for TB, SAT is also currenlty studied in some programmes. The results of nutriotional survey allow usto quantify the needs and the number of persons in needs, to precise the intervention strategy (blanket feeding, therapeutic feeding in centres or at home, etc) and to better target the intervention to the most in need. Retrospective mortality data describes the gravity of a situation and are often used for advocacy purpose to drew attention ot an ofteh forgotten catastrophic situation. In was the case in Darfur in 2004 A example of monitoring of programmes outcomes will be given later today (in the case of TB treatment in Chechnya) Comparison of strategies have been done for example on the comparison between home delivery of TB treatment and DOT for TB, SAT is also currenlty studied in some programmes.

5. Why does MSF do research? (2) To verify the efficacy of new treatment protocols To describe treatment outcomes of specific patients To gather evidence systematically in order to contribute to new knowledge in a scientifically credible manner You will hear this afternoon a presentation on kala azar treatment. A mutlticentric analysis of the outcome of ARV treatment in children has been made using the data from 18 programmes in 9 countries A dexcription of the dynamic and duration of several measles outbreaks shows tha mass vaccination campign is still useful even when the outbreak has started. You will hear this afternoon a presentation on kala azar treatment. A mutlticentric analysis of the outcome of ARV treatment in children has been made using the data from 18 programmes in 9 countries A dexcription of the dynamic and duration of several measles outbreaks shows tha mass vaccination campign is still useful even when the outbreak has started.

6. How research priorities are set Focus on topics with impact on: Improving how we operate Policy and practice Based on: Field questions/suggestions Technical working groups suggestions Opportunities Yearly decision by medical directors in each section and jointly Research priorities usually focus on the impact the research results will have to the quality of the assistance and on improvement on policy and practices Research priorities usually focus on the impact the research results will have to the quality of the assistance and on improvement on policy and practices

7. What does MSF do? Overview of all projects Type of research Topics Countries This information is based on a complete list of operational research projects from the 5 operational sections for 2004 and for 4 out of 5 sections for 2006. It includes the international research projects financed by all 5 sections and mainly implemented by Epicentre. For the time being, we have an excell table listing briefly the topic, the title of the research project, the section in charge, the partners, the countries where the projects will take place, and the stage of a project. . A standard format, more complete is currently tested by some sections and on some topics. This information is based on a complete list of operational research projects from the 5 operational sections for 2004 and for 4 out of 5 sections for 2006. It includes the international research projects financed by all 5 sections and mainly implemented by Epicentre. For the time being, we have an excell table listing briefly the topic, the title of the research project, the section in charge, the partners, the countries where the projects will take place, and the stage of a project. . A standard format, more complete is currently tested by some sections and on some topics.

8. What does MSF do (overview)? To keep this list quite open and dynamic, we have also included projects called in the dream stage, meaning that the research question was found interesting, but that the feasibility of the research project was still under assessment The information on the current stage of implementation is not available for all sections External partners includes Epicentre, other research groups, academic institutions in several countries. There are some limitations to any analysis of these data: No comparison can be made between years, because the definition has obviously changed since 2004, Breakdown by sections is not shown, but the the definition of research most probably varies across individual MSF sections (from 166 research projects in one to 32 in another ) Some sections have establish specific research support position, and it has certainly contributed to a large boost in the number of projects undertaken - some adhoc projects are not listed in these lists, such as survey done in emergency response (to describe the situation or to document a problem for advocacy purpose e.g. To keep this list quite open and dynamic, we have also included projects called in the dream stage, meaning that the research question was found interesting, but that the feasibility of the research project was still under assessment The information on the current stage of implementation is not available for all sections External partners includes Epicentre, other research groups, academic institutions in several countries. There are some limitations to any analysis of these data: No comparison can be made between years, because the definition has obviously changed since 2004, Breakdown by sections is not shown, but the the definition of research most probably varies across individual MSF sections (from 166 research projects in one to 32 in another ) Some sections have establish specific research support position, and it has certainly contributed to a large boost in the number of projects undertaken - some adhoc projects are not listed in these lists, such as survey done in emergency response (to describe the situation or to document a problem for advocacy purpose e.g.

9. What do we mean by “research project”? we can see that the types of research projects we do are mainly with a clear operational focus, and again some limitations should be mentioned: Tools development category needs to be clarified in the future (lab tools, protocols and guidelines) Subjectivity (mine): consistency but some difficulties to understand what is exactly behind one line title. we can see that the types of research projects we do are mainly with a clear operational focus, and again some limitations should be mentioned: Tools development category needs to be clarified in the future (lab tools, protocols and guidelines) Subjectivity (mine): consistency but some difficulties to understand what is exactly behind one line title.

10. What topics? Others topics include: meningitis, AB resistance, nutrition, mental health and psychological care, emergency in general, surgery, water and sanitation, Chagas disease, violence, reproductive health, vaccination, migrants, access to health care, trachoma Large number of HIV/AIDS research projects, because it was from the start of the interventions and systematic data collection tools has been developed, allowing systematic data collection and more research to be carried out. Others topics include: meningitis, AB resistance, nutrition, mental health and psychological care, emergency in general, surgery, water and sanitation, Chagas disease, violence, reproductive health, vaccination, migrants, access to health care, trachoma Large number of HIV/AIDS research projects, because it was from the start of the interventions and systematic data collection tools has been developed, allowing systematic data collection and more research to be carried out.

11. Where does MSF do research? DISCUSS LIMITATIONS: List of countries available only for B and CH, F and international where relevant 28 countries from Uzbekistan to Honduras, but most of them in Africa (normal, that’s were the highest number of operational projects are located as well) 16 countries with one or two projects SEE Graph Data are incomplete and given at country level, so it is difficult (or rather impossible) to link them with the stability of the context DISCUSS LIMITATIONS: List of countries available only for B and CH, F and international where relevant 28 countries from Uzbekistan to Honduras, but most of them in Africa (normal, that’s were the highest number of operational projects are located as well) 16 countries with one or two projects SEE Graph Data are incomplete and given at country level, so it is difficult (or rather impossible) to link them with the stability of the context

12. How are research outcomes shared? Internal: report, within a section or between sections, via technical experts, training or guidelines, informally, lost? Externally: MSF website or own report publications Advocacy (to substantiate a message) Letter or view point sent to peer reviewed journal Peer reviewed article Shared with the population ? I have no clear information on the way the outcomes of a study are shared with the study population, but I suspect there is a lot of room for improvement in this field. I have no clear information on the way the outcomes of a study are shared with the study population, but I suspect there is a lot of room for improvement in this field.

13. External publication analysis Limitations – just peer reviewed articles [note pubmed does not index letters] – so not including very important grey literature – reports, briefing papers, press release FROM AN ANALYSIS OF PUBMED USING SEARCH STRING “MEDECINS SANS FRONTIERES” OR MSF OR “DOCTORS WITHOUT BORDERS” OR EPICENTRE by affiliation. It is a swift approx assessment – not all indexing by affiliation is complete in pubmed – therefore is likely to be an underestimate *Might be some overlap 2005/2006 data since aruond then journals started epub ahead of print pub and both dates listed in pubmed [ie a project published in 2005 as epub can listed again when print pub in 2006]. **Other (review/methods analysis/conference report) ***at least one author from epicentre (Epicentre research obviously unrelated to msf excluded) Cannot be linked directly to research agendas for each year obvioulsy, but gives an approx ideaLimitations – just peer reviewed articles [note pubmed does not index letters] – so not including very important grey literature – reports, briefing papers, press release FROM AN ANALYSIS OF PUBMED USING SEARCH STRING “MEDECINS SANS FRONTIERES” OR MSF OR “DOCTORS WITHOUT BORDERS” OR EPICENTRE by affiliation. It is a swift approx assessment – not all indexing by affiliation is complete in pubmed – therefore is likely to be an underestimate *Might be some overlap 2005/2006 data since aruond then journals started epub ahead of print pub and both dates listed in pubmed [ie a project published in 2005 as epub can listed again when print pub in 2006]. **Other (review/methods analysis/conference report) ***at least one author from epicentre (Epicentre research obviously unrelated to msf excluded) Cannot be linked directly to research agendas for each year obvioulsy, but gives an approx idea

14. Comments Wide and unclear definition of operational research that varies across MSF Creation of research posts has boosted numbers of projects undertaken within sections of MSF Not clear how many planned projects are completed and what use is made of data Much research not published externally No systematic internal library Even if the data on external publications are incomplete, we can see that major improvement could be done in this field, even if some information may not be suitable for publications, such as incomplete data due to the instability of the context, the quality of data collected especially in a retrospective manner, etc But we should remember that sometimes our data are the only data that are available on certain populations and topics – so should be made available with limitations made clear Some may argue that those data should be made available to researchers in some forms, and the debate is going on in MSF as well. XXX Even if the data on external publications are incomplete, we can see that major improvement could be done in this field, even if some information may not be suitable for publications, such as incomplete data due to the instability of the context, the quality of data collected especially in a retrospective manner, etc But we should remember that sometimes our data are the only data that are available on certain populations and topics – so should be made available with limitations made clear Some may argue that those data should be made available to researchers in some forms, and the debate is going on in MSF as well. XXX

15. Challenges Context: duration, security, logistic constraints, etc Resources: high turnover of staff, priorities for resource allocation (human, financial) Internal: field initiative versus research requirements, knowledge gap, collaboration between sections External: Ethical (MSF and in country), standards (for data collection and sharing, for informed consent, for publication, etc) Evaluation: effectiveness of operational research in programme improvement, policy changes and thus in improving patients’ lives: how to evaluate this? The ultimate proof of judging the effectiveness of operational research is whether it helps to improve the performance of programmes or influences policy changes The ultimate proof of judging the effectiveness of operational research is whether it helps to improve the performance of programmes or influences policy changes

16. How to improve within our operational constraints? Done or decided: Creation of an institutional ERB Creation of research support position in some sections Better overview of research agenda on some topics Prospective research agenda with standard format, including dissemination plan Improved collaboration with external partners **Principles of collaboration with external partners should be based on: Mutual trust and shared decision making in aspects linked to the study and its management Emphasis on getting research findings into policy and practice Sense of ownership by the MSF programmes, field teams and national partners Development of research capacity into MSF programmes **Principles of collaboration with external partners should be based on: Mutual trust and shared decision making in aspects linked to the study and its management Emphasis on getting research findings into policy and practice Sense of ownership by the MSF programmes, field teams and national partners Development of research capacity into MSF programmes

17. How to improve – future ideas Make better use of routine monitoring data Improve quality and research standards: Training External and internal research support Sharing protocols – internally and externally Supporting alternative approaches eg qualitative Improve use of data collected: Monitor outcomes of research in terms of dissemination Share data with involved populations Improve internal knowledge database Support: at research design stage, statistical review by a pool of experts Some initiatives are slowly taking place such as the common format for a research agenda that will include information on the completion of a project including the dissemination strategy and results, that will allow us to better monitor ourselves. There is also some discussions on the fact that publishing in a external environment, now encouraged should become mandatory. Support: at research design stage, statistical review by a pool of experts Some initiatives are slowly taking place such as the common format for a research agenda that will include information on the completion of a project including the dissemination strategy and results, that will allow us to better monitor ourselves. There is also some discussions on the fact that publishing in a external environment, now encouraged should become mandatory.

18. Conclusions Clarify the definition of operational research Establish coordination platform between sections Establish precise research standards (ERB) Prioritise research by allocating sufficient time within operational constraints, organise training and expert support Distinction between routine M&E and research M&E is a delicate case, since routine M&E may highlight a problem that is further clarified with retrospective data analysis needing a ERB approval. The sections have different strengths and weaknesses, and increased collaboration and exchange may improve the quality and outcomes of research for the whole MSF movement MSF does not function in a centralised way, and some sections use their country ethical review mechanisms, creating a potential challenge to have uniform quality standard in research projects Many time, there is little room for research in an operational project, or there is some conflict for resources allocation (human mainly) in operational projects. Specific resources should be allocated and mechanisms developed for increasing the feasibility of quality research projects in the field. Distinction between routine M&E and research M&E is a delicate case, since routine M&E may highlight a problem that is further clarified with retrospective data analysis needing a ERB approval. The sections have different strengths and weaknesses, and increased collaboration and exchange may improve the quality and outcomes of research for the whole MSF movement MSF does not function in a centralised way, and some sections use their country ethical review mechanisms, creating a potential challenge to have uniform quality standard in research projects Many time, there is little room for research in an operational project, or there is some conflict for resources allocation (human mainly) in operational projects. Specific resources should be allocated and mechanisms developed for increasing the feasibility of quality research projects in the field.

19. Thank you very much Thankyou very much and over to you – I’ve described a lot of the constraints we face and some suggestions for moving forwards and now i hope that the debate today will give some useful ideas and feedbackThankyou very much and over to you – I’ve described a lot of the constraints we face and some suggestions for moving forwards and now i hope that the debate today will give some useful ideas and feedback