1 / 80

The Critical Appraisal of Journal Articles -The Critical Steps in EBM-

The Critical Appraisal of Journal Articles -The Critical Steps in EBM-. Long-Yau Lin MD, ScD Chung Shan Medical University National Health Research Institute. What this talk is not: Disclaimers. 1. Not to tell you to write better English. 2. Not a comprehensive research course.

dalejohnson
Download Presentation

The Critical Appraisal of Journal Articles -The Critical Steps in EBM-

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Critical Appraisal of Journal Articles -The Critical Steps in EBM- Long-Yau Lin MD, ScD Chung Shan Medical University National Health Research Institute

  2. What this talk is not: Disclaimers • 1. Not to tell you to write better English. • 2. Not a comprehensive research course. • 3. Not to criticize any specific proposal. • 4. The speaker do not pretend to know everything. • 5. The speaker may be wrong. • 6. Do not speak for any setting.

  3. Ethical standards for research involving human subjects • Nuremburg Codes. • Helsinki Declaration. • Informed consent. • IRB.

  4. 1949:The Nuremberg Code • 美國的法官於1947年在所謂的 「醫師的審判」（Doctors' Trial）中訂定了紐倫堡條約（Nuremberg Code），為關於醫學研究倫理上最重要的文件。 紐倫堡條約一共有十條，第一條即開宗明義地說出：「受試者的自願同意是絕對必要。」 (The voluntary consent of the human subject is absolutely essential.) 。此後，任何臨床醫學實驗在進行以前都必須獲得受試者的同意。所謂之「同意」係指受試者必須處在沒有任何壓力、脅迫、利誘、哄騙的情形下，並且不受隱瞞，實驗者必須明白告知受試者實驗的目的、對個人的影響與研究成果對社會可能之預期貢獻等。 • Art. 1: The voluntary consent of the human subject is absolutely essential. • Art. 9: During the course of the experiment the human subject should be at liberty to bring the experiment to an end if he has reached the physical or mental state where continuation of the experiment seems to he to be impossible.

  5. 1964: Helsinki Declaration • 世界醫學協會(World Medical Association)經過幾年的研議，在1964年於芬蘭首都赫爾新基所召開的第十八屆總會中通過確認人體試驗的倫理規範 ”Recommendations guiding physicians in biomedical research involving human subjects”，通稱為「赫爾新基宣言」 • 1975, 29th Tokyo Revision • 1983, 35th Venice Revision • 1989, 41th Hong Kong Revision • 1996, 48th Somerset West Revision • 2000, 52nd Edinburgh Revision

  6. Helsinki Declaration • Introduction • The health of my patients will be my first consideration. • Medical progress is based on research which ultimately must rest in part in experimentation involving human subjects. • Clinical research vs. Non-clinical research. • Basic Principles（1）需事先徵求受試人在自由意志情況下之自願同意，且此受試人必須具有同意之法律能力；（2）受試人對於實驗所涉及之內容，有一定程度之瞭解；（3）實驗本身設計的目的是為人類社會之福祉；（4）進行人體實驗前必須先有實驗室及動物實驗依據；（5）盡力避免對人體身心的傷害，一旦實驗進行中發現對人體有害，應立即停止；（6）必須在合法機關監督下，由具備資格者進行實驗；且必須事先擬好補償措施； • Medical Research combined with clinical care (Clinical research) • Non-therapeutic biomedical research involving human subjects(Non-clinical biomedical research). • 試驗計畫書(protocol)應經一獨立委員會審查“specially appointed committee independent of the investigator and the sponsor” （通則第4條） ─ 人體試驗委員會：美國─”Institutional Review Board”, 台灣─ “Ethics Committee” • 赫爾辛基宣言之精神 • 自主：受試驗者是在被充分告知相關訊息後，自由決定要參加的。 • 有益：參加試驗的風險相對於可能有的好處是可以接受的。受試驗者參加試驗後並不會犧牲其權利，仍會受到已證明有效的最佳照顧。 • 赫爾辛基宣言之功能- 道德勸說, 效果…

  7. 知情同意（Helsinki Declaration通則第 11, 12, 13, 14,15,16條） • 人體試驗受試者之告知後同意 • 告知之範圍：受試者應被告知試驗目的、方法、實驗之預期利益及潛在風險；並應被告知其有不加入以及任意退出之自由(赫爾辛基宣言通則第13條） • 告知之態度：醫師應避免病患對醫師之依賴關係而「不得不」同意（consent under duress) (赫爾辛基宣言通則第14條） • 受試者為無行為能力人時，應取得其法定代理人之同意；受試者雖然為未成年人，但只要具備意思能力，也應一併取得其本人同意(赫爾辛基宣言通則第16條）

  8. Q1: Why was the study done? • Clarify your purposes for evaluation first. • What is the question to be answered? • Is it a practical issue or a theoretical issue? • Is the problem open-ended and non-biasing?

  9. Q2: What are prior hypotheses? • Marshal your external knowledge relevant to this article. • What do you know about this topic-first hand knowledge as well as the results of other research? • How important is this problem to your more general concern? • How much influence should the credibility of the author and the journal have on your judgment? • Is the research sponsored? • Is the precise aim described?

  10. Publication bias • Suppose 1,200 trials small trials are in progress. In 1,000 trials there is no difference between therapies, and in 200 trials, there is a true difference. 50 of the 1,000 trials where no difference exists will show a misleading ? significant? difference by chance alone ( reflecting the alpha risk of 0.05) and only 50 of the 200 trials where a true difference exists will result in a correct significant result ( reflecting the false negative risk of small trials). Assuming that only those small trials will be published where the results are significant, a mere 100 of them will reach the literature- and in half of cases the results will be misleading!

  11. Robert C.,Young ‘s article

  12. Nelson’s critique

  13. Robert C., Young’s response

  14. Credibility of authors and journal • Extended work shifts and the Risk of motor vehicles crashes among Interns • 3429 subjects enrolled--- 40 subjects submitted “ The Diary” at the end of study • Lauras k. Boger. Harvard Work Hours, Health and Safety Group • “ The New England Journal of Medicine January 13, 2005 Vol. 372, No. 2

  15. Clopidogrel versus Aspirin and Esomeprazol to Prevent Recurrent Ulcer Bleeding • N Engl Med 2005; 352: 238-44 • Francis K.L Chan et. el. • Prince of Wales Hospital in Hong Kong • First endpoint: recurrent ulcer bleeding • Second endpoint: lower gastrointestinal bleeding

  16. Equivalent study • Any comparisons of side effects, cost analyses and quality of life analyses should base on the the equivalent comparisons. • Equivalent comparison: superior or not inferior.

  17. Q2-continued • Are definitions sufficiently specific? • Are the definitions fruitful? • Are circular definitions used? • What are included in the literature review? • Did the author thoroughly consider what might be found and the corresponding implication prior to gathering the data?

  18. Circular definition ( Begging the question) 國民健康局: Less prenatal visits is associated with the occurrences of preterm birth

  19. Q3: How would you label the study in epidemiology? • Is it a descriptive study, survey, case control study, cohort study, experimental study, meta-analysis or a surveillance?

  20. Inappropriate study design-(I) example • Study design: Before -after observational study. • Results: Mortality decreased from 27.8% before to 19.9% after surfactant therapy was introduced. • Jeffrey D. Horbar et al. Decreasing Mortality Associated with the Introduction of Surfactant Therapy: An Observational Study of Neonates Weighing 601to 1300 Grams at Birth. Pediatrics 92(2):191-196,1992

  21. Inappropriate study design(II)

  22. Another example of secular trend • Mckeown: The role of medicine: Dream, Mirage or Nemesis? London, Nuffield Provincial Hospital Trust, 1976, • The inventory of antibiotics has not contributed much in the decline of mortality in infectious diseases. Mortality of infectious disease 1945

  23. Prevalence of hysterectomy?

  24. Prevention and Treatment of Nosocomial Sepsis in the NICUReese Clark (original article) journal of perinatology 2004;24:446-453 Effective strategies to prevent nosocomial sepsis are, 1. Continuous monitoring. 2. Surveillance of infection rates and distribution of pathogens. 3. Strategic nursery design and staffing. 4. Emphasis on staff accountability for incidence of infection.

  25. Q4: What is the study subjects? • Was the assignment of patients to treatments randomized? • Were all patients who entered the trial properly accounted for and attributed at its conclusion? • Was follow-up completed? • Were patients analyzed in the groups to which they were randomized? • Is sample size scientifically decided?

  26. Inappropriate comparison • A survey was based upon 160 rehospitalized patients whith Crohn’s colitis or ileocolitis. A total of 100 (63%) underwent major operation. Of these 58(58%) required re-operation. Of these 31/58(53.5%) required a third, 15/31( 48.4%) a fourth surgical intervention, etc. • Conclusion: The re-operation rate diminished with each succeeding operation from 58% after the first, to 47% after the fourth.

  27. Inbalance in base-line • Operation for morbid obesity , 496 operations for 434 patients • Conslusion: Best weight loss occurs in the heaviest patients • Ann Surg 182: 405

  28. Archie error • Again, this cases is an operation outcome of morbid obesity. The conclusion is the heavier the patient was pre-operatively, the smaller was the excess weight shed post-operatively. The equation to pre-operative weight was y=0.19 x+ 11.3. The slope of the linear regression line was significantly different from zero according to a t- test. • Surgery 90:446

  29. Population versus Sample • A population is a collection ( set) of data that describes some phenomenon of interesting you. • A sample is a subset of data from population

  30. Subjects precluded • Neurologic and Developmental Disability at Six Years of Age after extremely Preterm Birth • Neil Marlone D.M. EPIcure Study Group UK • The New England Journal of Medicine • January 6, 2005, Vol. 352,No.3

  31. Q5: What are the comparison subjects? • Was more than one group studied so that a comparison can be made? • Is the clinical trial agent compared with gold standard/placebo? • Are realistic comparative doses being used?

  32. Inadequate or no control • You always need a control group for analytic study, otherwise it is only a descriptive study. • Example: • 70% of motor cycle traffic accident mortality didn’t use helmet. So, helmet is protective of head injury in motor cycle traffic accident. It is all right in certain circumstances, however, if 90% of motor cyclists did not use helmet, then it will be a different story.

  33. Association of Viral Factors with Non-Familial Breast Cancer in Taiwan by Comparison with Non-Cancerous, Fibroadenoma, and Thyroid Tumor Tissues • Journal of Medical Virology 75:276-281 (2005)

  34. Q6: Could there have been bias in the selection of study subjects and comparison subject? • What is target population? What is the actual population? What is study sample? Is it a representative sample? • Did the subjects randomly selected? • What are exclusion/inclusion criteria? • Are there any standard selection process? Are there any rationale of the process? • How did the sample size been determined? • Are there any missing data? • What kind of errors are present? • How important are consequences of these errors for your purpose? • What is left out? • Did the data ( demographic-age or ethnicity, disease prevalence, economic status-SES or access to care, patient references) been adjusted?

  35. Q7: Could there have been bias in the collection of information? • Who did the observation? • What is the measurement scale? Quality or Quantity? • Any repeated measurement? • Are operational definitions used and are they adequate? • Is the researcher trying to measure more precisely than the instrument allows? • Is some measure of reliability given? • What is really being measured? • Were the patients, health workers, and study personnel “ blind” to the treatment? • Were the groups similar at the start of the trial? • Aside from the experimental intervention, were the groups treated equally? • Are steps to avoid bias clearly described? • Was the duration of the cohort or experimental study appropriate?

  36. Q8: What provision was made to minimize confounding? • A confounder is an extrinsic factor that bis associated with the predictor variable and is a cause of the outcome variable • Restriction, matching or stratification?

  37. Q9: What ws/were statistic method used? What was the measure of association and stability in the association? • Are the statistic tests appropriate? • How does/do the result(s) presented? • Relative risk or Odds ratio? • 95％confidence interval been presented? • Are results statistically significant or clinically significant? • Are all subjects accounted for?

  38. Harmony mean • A guy had a journey of round trip to Tonhai university, he decided to finish the journey by a speed of 60 miles / hour. However, his speed was 30 miles/ hour in the first half journey, what speed he should used in the second half to meet the predetermined speed.

  39. The idea of “at least” -I • Suppose transmission of HIV virus through each single blood transfusion is 1%. • What is the probability of getting HIV virus with 10 times blood transfusion? • 10% ? • What is the probability of having HIV infection with 101 times blood transfusion?

  40. The idea of “at least”-II • The answer is:

  41. Inappropriate table • Basic requirements for tables: Simplicity, Consistent, Compactness, Clarity, Readability • Table/text ratio in a manuscript: Approximately 1 table per 1000 words 1 page ( A4, double spaced) equals 250 to 300 words Best ratio is: less than 1 table per 4 typing pages 60 characters per column for two-column page, 120 characters width ( for tables)

  42. Incorrect figure presentation Annual delivery statistic at St. Peter’s Medical Center

  43. It should be looked like this Annual delivery statistic at St.. Peter’s Medical Center

  44. Reliability or Validity The true values of sperm count for 5 men are: 10,000. 11,000. 12,000. 13,000. 14,000. The readings of A measurement method are: 9,000. 10,000. 11,000. 12,000. 1,3000. The readings of B measurement method are: 900. 1,000. 1,100. 1,200. 1,300 The correlation coefficient of method A and true value is 1.0 The correlation coefficient of method A and B method is 1.0 However, B method is not a “valid” method.

More Related