WRNMMC SCIENTIFIC REVIEW PANEL Wendy B. Bernstein, MD COL MC USA, Retired

1. National Naval Medical CenterDirectorate for Professional EducationClinical Investigation Department WRNMMC SCIENTIFIC REVIEW PANEL Wendy B. Bernstein, MD COL MC USA, Retired

2. Disclosure Statement I have no financial support or conflict of interests to report. I am new at this job and don’t know all the rules yet.

3. Learning Objectives • Where did the idea of scientific review (SR) come from? • Where does SR fit in the protocol approval process ? • Understand the mechanics of the SR process • Understanding the key sections of the protocol • Common mistakes that slow the process

4. Origins of scientific review • Several protocol violations late 1990’s and early 2000’s, including several protocols at civilian institutions with Navy PIs • BUMED annual Inspector General review 2002 recommended separating IRB review into 2-step process • Scientific review • Institutional Review Board

5. Origins of scientific review • Two step review performed at most institutions • Considered gold standard • Helps to optimize benefit to risk ratio of research proposals • SRP and IRB instituted at all Navy MTF’s by Surgeon General in 2004

6. Origins of scientific review: Purpose • Ultimately to protect human and animal subjects protocols by ensuring scientific validity. • Investigators may not be objective about their own research. • Underestimate risks; overestimate benefits • Poorly designed research cannot be published • Poorly designed research in training will hamper academic progress.

7. Nursing Research Research Resources Simulation Center SR and the wiring diagram Commander Director for Graduate Education Clinical Investigations Department RESPONSIBLE CONDUCT OF RESEARCH SERVICES ADMINISTRATIVE SUPPORT SCIENTIFIC REVIEW PANEL INSTITUTIONAL REVIEW BOARD

8. The scientific review - investigator relationship must be collegial, respectful, and useful for all parties. Scientific Review Investigator

9. We are happy to help you at any step of the process. Feel free to bring us a draft or come with design questions. PI Scientific review

10. Scientific Review Committee: Structure • Chair • Up to two Vice-Chairs • Nine additional members • Reviewers are individuals with research experience

11. Scientific Review: Process • Your protocol is reviewed by the biostatistician first. • Expedited or Exempt research is reviewed by the Chair or his/her designee only. • If your protocol is going before the full IRB, then 3 individuals review the protocol, the chair, plus two additional reviewers. • This sounds like it will take forever!

12. Scientific Review: Speeding up the Process • Biostatistician 10 business days • Reviewers 1 and 2 10 business days • Chair 10 business days • To speed up the process, I am reviewing protocols concurrently with the statistician. This essentially cuts two weeks off the process. • If protocols need lots of work, I can forward my review immediately or if changes are minor, I can wait and summarize with the other two reviewers. • I make contact with the PI to let him know the process has started.

13. Scientific Review: Committee Meeting • You will be given a month to reply to our concerns with revisions or explanations. • New with the merger are live committee meetings with the Chair, Biostatistician, the two reviewers, investigators, anyone else who is interested. • ONLY projects going to the full IRB will be reviewed at a committee meeting. Meetings are held every two weeks. • Your project will be addressed at a meeting at least two weeks after you have received our comments. The meeting can be used to address the concerns, answer questions, or even get help with stats, design etc. The committee, unlike the IRS, is really there to help you.

14. Scientific Review: Committee Meeting • If the revisions are straightforward, the Chair will review revisions alone. • If the revisions are extensive, then the Chair may ask the original reviewers take another look at the protocol. This would apply if one of the reviewers was especially familiar with the research. For example, a radiologist looking at a protocol involving MRI in persons with PTSD.

15. Anyone can learn to write a protocol

16. Let’s write a protocol • What are you interested in? • What can you do in the amount of time available? • Are there projects in your department that you can build on? • Is an outside institution interested in collaboration? • Keep it simple!

17. My fantasy project • I have a theory that the physical exams recorded in AHLTA are inaccurate. • They are inaccurate because most docs and nurses cannot write their notes during clinic so they write them later and forget important findings. • Some people are so rushed they simply copy forward from a past note!

18. First things first: The correct format • The protocol must follow a specific format: http://www.bethesda.med.navy.mil/professional/research/Responsible_Conduct/Forms/index.aspx

19. Skip straight to Section 3 Objectives: • Start with your hypothesis: PE in AHLTA notes are inaccurate. • The main objective of this trial is to show that the physical exam section of AHLTA notes is inaccurate. • A secondary objectives might include: • Some parts of the exam are particularly susceptible to error. • Notes are written more than 24 hours after the clinic visit • Notes are copied forward from the last visit

20. The objective depends upon your study: The objective of your study might be to show that a new drug works better than an older drug. OR that a method of treatment works better than another Whatever your objective, begin to think about your endpoint now!

21. What is an endpoint?

22. Think about the endpoint as you write your objectives: Your endpoints are what you are measuring to meet your objectives. AHLTA study: Physical exam findings Drug study: Survival, improvement in x-ray or lab study, faster hospital discharge Methodology Study: better quality biopsy samples, anesthesia time, cost, complication rates

23. Section 4: Military Relevance • Explain the military relevance of your research. • For research to be relevant it does not have to relate only to active duty or war fighters. Old retirees with chronic health problems cost Uncle Sam a lot of money, so if you can find a better way to treat chronic disease…

24. Section 5: Background and significance Comprehensive review of the literature. Tell me which data bases you searched and what search terms you used. Read lots of papers. You might find one that has a design that will work for you.

25. DATABASE Literature Search • “A literature search using the Medline Ovid search engine was performed on 30JUN2007 from 1950 until the present using the search terms traumatic intracranial aneurysm, traumatic aneurysm, traumatic pseudoaneurysm. A total of 1143 entries returned. These results were initially screened for those that were English language entries leaving 944. Titles were then screened and an additional 855 entries were excluded secondary to relevancy leaving 89. Complete papers corresponding to each abstract were then obtained and read in detail. Papers were then rejected based on their relevance to vascular injury associated with military penetrating brain and spinal injury leaving 5 studies. These articles characterize an experience encompassing 75 cases of traumatic central nervous system arterial injury. Though valuable, they offer no reference to the rate of rupture or resolution, and limited reference to endovascular interventions. This study proposes to fill this significant gap.”

26. That looks like a pain in the neck • If your background section is well written and complete, I will not go looking for the specifics of how you searched data bases. • If on the other hand you state in the background, we searched the literature and found no articles on how to treat hypertension, I am going to be suspicious and ask you to show me your search methods. • If you tell me we searched the literature and found no articles on continuous infusion chemotherapy in small cell lung cancer, I might ask to see your search methods, especially if I turn up some papers.

27. Background The background section informs the reviewers why the research question is important. What is the current state of the art in this field What studies have been done in the past Where does your specific question fit in the big picture? What is different about your project How will your project add to the literature

28. Section 6 is the meat of your proposal • Plan • Selection of subjects • Inclusion Exclusion criteria • Study Design and Methodology • Statistical Consideration • Data Analysis • Reporting Adverse Events • Human Biological Specimens and Tissue

29. Section 6: The Plan • Brain storm with colleagues and spend some time figuring out how to do your project. • How will we show that AHLTA notes are inaccurate? • We must take into consideration that everyone makes mistakes doing a physical exam. • How will we separate the human error from AHLTA induced error? • Should we have one or two docs repeat the exam? • Should we just focus on one part of the PE? • Should we see how many hours after the visit, the AHLTA notes are written, document copy forward?

30. When you think you have things worked out, brainstorm some more • We are going to get consent from random patients coming into a clinic to undergo a second physical exam by two docs right after they complete their visit with the physician. • Patients will come from all physicians and NP seeing patients Mon - Fri. for a month. • Two board certified internists will perform the repeat exam. • The investigators will compare all 3 physical exams • The investigator will review past notes to look for copy forward effect and figure out how many hours after the visit, the note was written.

31. Anybody see problems? Have we worked out all the bugs?

32. Our Plan • We are going to get consent from random patients coming into a clinic to undergo a second physical exam right after they complete their visit with the physician. How many and from where? Sick folks? Exam details? Should the docs know this is happening? • Patients will come from all physicians and NP seeing patients Mon - Fri. for a month. Staff, trainees, follow ups, new consults? • Two board certified internists will perform the repeat exams. What if he finds something dangerous? Record findings on? • The investigator will compare the physical exams. Who is right? • The investigator will review past notes to look for copy forward effect and figure out how many hours after the visit, the note was written. All notes, just notes from that clinic, that physician? Record finding on?

33. Section 6: The Plan: Selection of Subjects • What type of patients? internal medicine, medicine subspecialties, surgery? • Should we avoid pregnant women, children, folks with mental handicap? • Should we approach patients in the waiting room, put up flyers, talk to the clinic docs first to get permission to approach their patients?

34. Section 6: Inclusion Exclusion Precisely define your subject population Try to avoid the defined vulnerable population such as pregnant women, children, prisoners. Are there certain medications or concurrent medical problems that will interfere? Think about how you will recruit subjects without coercion or the appearance of coercion.

35. Start to think about HIPAA and how you will protect your subject’s identity as well as your data.

36. For many of us, this is a… Section 6: Statistical Considerations

37. Section 6: Statistical Considerations Having a clear understanding of your objectives and endpoints will help the statistician model statistical analysis. Visit the statistician early and often during the design phase. Endpoints: Abnormalities on a standard physical exam Copy forward documented? Length of time between visit and note written

38. Now, back to Section 2: Abstract • A concise outline of your project • First is a paragraph or two that briefly summarizes your background section. Tell the reviewer enough to get him interested and keep him interested. • Purpose: State your objectives. • Study Design: Observational, Pilot, Randomized double blinded • Methodology and Technical Approach: Briefly out line what you will do. Include your endpoints and analysis. The reason to save this for last is that by the time you are done with the entire proposal you know the protocol like the back of your hand and you can clearly summarize its key features.

39. Avoiding common pitfalls will save you time and energy!

40. Common Pitfalls • The scientific reviewers are NOT all knowing. • Have your proposal reviewed by a more experienced colleague, especially for the science. Get your department head involved. • Spell out abbreviations with first use. • This is a formal document. Do not use slang. • Data are collected and patients are subjects • Make sure the references are correct. • PROOF READ

41. Common Pitfalls Section 2: Abstract

42. Common Pitfalls Section 3: Objectives • Make sure your objectives are very clear and you can accomplish them via the methods you describe. • Make sure any concepts you bring up in the objectives have been addressed in the background. • Make sure you have addressed every objective in your methods section. • Make sure your primary objective and primary endpoints align.

43. Common PitfallsSection 4: Military Relevance • Just remember that health care advances are important to active duty and retirees and their dependents.

44. Common PitfallsSection 5: Background • Too brief or sketchy • Incomplete • We are using a left handed smoke shifter to see if infantrymen have less corneal injury following blast exposure. Make sure you talk about the LHSS. • Review of the literature is not critical • There are several studies in the literature documenting the benefit of carotid endarterectomy (1,2,3). But the investigator forgets to tell you that one of those studies has major limitations.

45. Common PitfallsSection 6: Plan and Methods • Be obsessive with the details; leave nothing to the imagination. • Think carefully about Inclusion/Exclusion criteria • Don’t generalize too much, i.e. No short people will be included. Shortness is in the eye of the beholder. • Recruitment: Convince us you won’t coerce potential subjects, watch out for vulnerable populations. • Study Design: Again, be obsessive with the details. • How does the LHSS work? How will it be used? Who will calibrate it and make sure it is working? What kind of data do you get? • What type of forms are you using for data collection? Include them in sequentially lettered appendices.

46. Common PitfallsSection 6: Plan and Methods • Statistical consideration • Be sure to visit with the statisticians before you submit. Don’t plan on winging it once you see the data. • End points • Make sure they align with your objectives. If the primary objective is to show that AHLTA notes are inaccurate, then your endpoint is number of missed physical exam findings. • If your objective is to show that a new drug is a better treatment for PTSD, then you need an endpoint for measuring PTSD, such as a validated survey tool. • If your objective is to show a new drug prolongs survival, then survival is your endpoint. • Don’t forget to show us your data collection sheets!

47. What about collaboration with civilian institutions ? You have been asked by Dr. Scholl to collaborate with him to test a new gel combat boot insert. He provides you with an approved protocol from the biggest most prestigious university in the universe. This is a complicated study with some fancy high tech molecular biology analysis of the blood to look for evidence of stress hormone levels while gellin’ So, I don’t have to write another protocol do I?

48. Here is the deal: We think it is great that you have this opportunity. But, the protocol will not be rubber stamped. You must submit a military adaptation of the protocol. You may either refer the reader to the appropriate pages and sections of the sponsor’s protocol or you may cut and paste as needed. Be sure to address military unique points. If you think an aspect of the sponsor’s protocol will be hard to understand or controversial, then address that in the military adaptation. We read them both, carefully.

49. Here is the deal continued: Usually we do not find major problems that change the science, but we do find errors that are relevant to the methods. If we find problems we ask you to ask the sponsor to address our concerns at the time of an amendment. If you are working on something like this, get the protocol to us asap and we can review it along with the other scientific review committees that are relevant to the project.

50. The Review Template