1 / 93

Critical appraisal of the medical literature

Critical appraisal of the medical literature. Partini Pudjiastuti, Sudigdo Sastroasmoro Child Health Department Faculty of Medicine University of Indonesia. Population & Sample. Sudigdo Sastroasmoro sudigdo_s@yahoo.com.

talasi
Download Presentation

Critical appraisal of the medical literature

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Critical appraisal of the medical literature Partini Pudjiastuti, Sudigdo Sastroasmoro Child Health Department Faculty of Medicine University of Indonesia

  2. Population & Sample Sudigdo Sastroasmoro sudigdo_s@yahoo.com

  3. Populationis a large group of study subjects (human, animals, tissues, blood specimens, medical records, etc) with defined characteristics [“Population is a group of study subjects defined by the researcher as population”] • Sampleis a subset of population which will be directly investigated. Sample should be (or assumed to be) representative to the population; otherwise all statistical analyses will be invalid • All investigations arealwaysperformed in the sample, and the results will be applied to the population

  4. Avoid using ambiguous terms • Sample population • Sampled population • Populasi sampel

  5. Target population= domain = population in which the results of the study will be applied. Usually character-ized by demographic & clinicalcharacteristics; e.g. normal infants, teens with epilepsy, post-menopausal women with osteoporosis. • Accessible population = subset of target population which can be accessed by the investigator. Frame: time& place. Example: teens with epilepsy in RSCM, 2000-2005; women with osteoporosis, 2002 RSGS • Intended sample = subjects who meet eligibility criteria and selected to be included in the study • Actual study subjects = subjects who actually completed the participation in the study

  6. Usually based on practical purposes Accessible population (+ time, place) Appropriate sampling technique Actual study subjects Intended Sample Subjects completed the study [Subjects selected for study] [Non-response, drop outs, withdrawals, loss to follow-up] Target population = DOMAIN (demographic, clinical)

  7. External validity II: Does AP represent TP? Target population Accessible population [External validity I: Does IS represent AP?} Actual study subjects Intended Sample [Internal validity: does ASS represent IS?]

  8. Sampling methods A. Probability sampling • Simple random sampling (r. table, computer) • Stratified random sampling • Systematic sampling • Cluster sampling • Others: two stage cluster sampling, etc B. Non-probability sampling • Consecutive sampling • Convenience sampling • Judgmental sampling

  9. Note • All statistical analyses (inferences) are based on random sampling • Whether or not a sample is representative to the population depends on whether or not it resembles the results if it were done by random sampling

  10. IMPORTANT!!! Statistical significancevs.clinical importance • Negligible clinical difference may be statistically very significant if the number of subjects >>>. e.g., difference in reduction of cholesterol level of 3 mg/dl, n1=n2 = 10,000; p = 0.00002 • Large clinical difference may be statistically non-significant if the no of subjects <<<, e.g. 30% difference in cure rate, if n1 = n2 = 10, p = 0.74

  11. x = 220 x = 217 Clinical importance vs. statistical significance Cholesterol level, mg/dl Standard treatment x = 300 mg/dl R n=10000 Clinical n=10000 New treatment x = 300 mg/dl t = df = 9998 p = 0.00002 Statistical

  12. Clinical Statistical Clinical importance vs. statistical significance Cured Died Standard Rx 0 10 (100%) New Rx 3 7 (70%) Absolute risk reduction = 30% Fischer exact test: p = 0.211

  13. Correlation between abdominal circumference and total cholesterol level in middle-aged men • N = 200 • R = 0.22, p = 0.031 • Conclusion: There was a significant correlation between abdominal circumference and total cholesterol level in the subjects studied. Measuring abdominal circumference may predict the cholesterol level in middle-aged healthy men.

  14. How important is important? • Two percent mortality reduction is probably not important in your clinic • In a community prevention, a simple measure that reduce 2% severe morbidity is probably important. • Low dose aspirin reduces 2% cardiac events in 5 years (without aspirin 400 cardiac events per 10,000, with aspirin 200 cardiac events) • Requires judgment

  15. How to infer? • Can the results of the study (in sample) be applied in the accessible or target population? • Hypothesis testing & confidence interval

  16. Statistic and Parameter • An observed value drawn from the sample is called a statistic(cf.statistics, the science) • The corresponding value in population is called a parameter • We measure, analyze, etc statistics and translate them as parameters

  17. Proportion Percentage Mean Median Mode Difference in proportion/mean OR RR Sensitivity Specificity Kappa LR NNT Examples of statistics:

  18. There are 2 ways in inferring statistic into parameter: • Hypothesis testing p value • Estimation:confidence interval (CI) P Value & CI tell the same concept in different ways

  19. P value • Determines the probability that the observed results are caused solely by chance (probability to obtain the observed results if Ho were true)

  20. Group Success Failure Total C 30 (60%) 20 (40%) 50 E 40 (80%) 10 (20%) 50 X2= ; df = 1; p = 0.0432

  21. Group Success Failure Total C 30 (60%) 20 (40%) 50 E 40 (40%) 10 (20%) 50 X2= ; df = 1; p = 0.0432 If drugs E and C were equally effective, we still can have the above result (difference of success rate of 20%) but the probability is small (4.32%) If drugs E and C were equally effective, the probability that the result is merely caused by chance is 4.32% If we define in advance that p<0.05 is significant, than the result is called statistically significant

  22. Similar interpretation applies to ALL hypothesis testing: t-test, Anova, non-parametric tests, Pearson correlation, multivariate tests, etc: If null-hypothesis null were true, the probability of obtaining the result was ……. (example 0,02 or 2%, etc)

  23. Confidence Intervals • Estimate the range of values (parameter) in the population using a statistic in the sample (as point estimate)

  24. If the observed result in the sample is X, what is the figure in the population? X X P X S CI A statistic (point estimate)

  25. Most commonly used CI: • CI 90% corresponds to p 0.10 • CI 95% corresponds to p 0.05 • CI 99% corresponds to p 0.01 Note: • p value  only for analytical studies • CI  for descriptive and analytical studies

  26. How to calculate CI General Formula: CI = p  Z x SE • p = point of estimate, a value drawn from sample (a statistic) • Z = standard normal deviate for , if  = 0.05  Z = 1.96 (~ 95% CI)

  27. Example 1 • 100 FKUI students  60 females (p=0.6) • What is the proportion of females in Indonesian FK students? (assuming FKUI represents FK in Indonesia)

  28. Example pq = SE(p) n . . 0 6 x 0 4 = ± % . . 95 CI 0 6 1 96 100 = ± . . 0 6 1 96 X0.5/10 = ± . . = . ; 0 . 0 6 0 1 0 5 7

  29. Example 2: CI of the mean • 100 newborn babies, mean BW = 3000 (SD = 400) grams, what is 95% CI? 95% CI = x  1.96 x SEM

  30. Examples 3: CI of difference between proportions (p1-p2) • 50 patients with drug A, 30 cured (p1=0.6) • 50 patients with drug B, 40 cured (p2=0.8)

  31. Example 4: CI for difference between 2 means Mean systolic BP: 50 smokers = 146.4 (SD 18.5) mmHg 50 non-smokers = 140.4 (SD 16.8) mmHg x1-x2 = 6.0 mmHg 95% CI(x1-x2) = (x1-x2)  1.96 x SE (x1-x2) SE(x1-x2) = S x V(1/n1 + 1/n2)

  32. Example 4: CI for difference between 2 means

  33. Other commonly supplied CI • Relative risk (RR) • Odds ratio (OR) • Sensitivity, specificity (Se, Sp) • Likelihood ratio (LR) • Relative risk reduction (RRR) • Number needed to treat (NNT)

  34. Suggested CI presentation: • 95%CI: 1.5 to 4.5 • 95%CI: -2.5 to 4.3 • 95%CI: -12 to -6 • Not recommended: 3 +1.5 • Not recommended: -9+ -3

  35. In contrast to CI for proportion, mean, diff. between proportions/means, where the values of CI are symmetrical around point estimate, CI’s for RR, OR, LR, NNT are asymmetrical because the calculations involve logarithm

  36. Examples • RR = 5.6 (95% CI 1.2 ; 23.7) • OR = 12.8 (95% CI 3.6 ; 44,2) • NNT = 12 (95% CI 9 ; 26)

  37. If p value <0.05, then 95% CI: exclude 0 (for difference), because if A=B then A-B = 0  p>0.05 • exclude 1 (for ratio), because if A=B then A/B = 1,  p>0.05 • For small number of subjects, computer calculated CI may not meet this rule due to correction for continuity automatically done by the computer

  38. Concluding remarks • In every study sample should (assumed to) be representative to the population. Otherwise all statistical calculations are not valid • p values (hypothesis testing) gives you the probability that the result in the sample is merely caused by chance, it does not give the magnitude and direction of the difference • Confidence interval (estimation) indicates estimate of value in the population given one result in the sample, it gives the magnitude and direction of the difference

  39. Concluding remarks • p value alone tends to equate statistical significance and clinical importance • CI avoids this confusion because it provides estimate of clinical values and exclude statistical significance • whenever applicable, supply CI especially for the main results of study • in critical appraisal of study results, focus should be on CI rather than on p value.

  40. 1 • The ultimate goal of clinical research is the use of evidence in source population

  41. 2 • The best non-probabiity sampling is consecuitve sampling

  42. 3 • P value refers to the probability of getting the observed result when the Ho were false

  43. 4 • The mean difference of 2 measurements is 20 mmHg, with 95% CI 15 to 25 mmHg. The p value should be “statistically significant”

  44. 5 • Confidence intervals give more information than p value

  45. 6 • It is possible to have a study with good internal validity but poor external validity

  46. 7 • If the odds ratio is 5, then the 95% CI may have values from 3 to 11

  47. 8 • It is possible to have a significant difference even when the clinical difference is not important, but clinically important difference always statistically significant

  48. 9 • Appropriate sampling method is mandatory to ensure generalization

  49. 10 • Clinical epidemiology may include animal studies

  50. 11 • The more wide the confidence interval, the more precise the result of any study

More Related