1 / 42

Validity and Reliability Dr. Voranuch Wangsuphachart Dept. of Social & Environmental Medicine

Learn the concepts, importance, and strategies to assess and enhance validity and reliability in medical research. Understand the major types of bias. Presented by Dr. Voranuch Wangsuphachart.

thomascharles
Download Presentation

Validity and Reliability Dr. Voranuch Wangsuphachart Dept. of Social & Environmental Medicine

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validity and Reliability • Dr. Voranuch Wangsuphachart • Dept. of Social & Environmental Medicine • Faculty of Tropical Medicine • Mahodil University • 420/6 Rajvithi Road • Bangkok 10400, THAILAND • E-mail

  2. Objective: at the end of lecture, student would be able to: 1. Know concepts & definition of validity & reliability 2. List importance and impact of validity & reliability 3. Specify strategies to assess validity & reliability 4. List strategies to enhance validity & reliability 5. Describe major types of bias

  3. Contents: 1. Validity 2. Reliability • Definition and synonyms • Important points • Accessing validity & reliability • Strategies to enhance validity & reliability 3. Major types of bias

  4. Medical or epidemiological study, major consideration is to obtain: Valid measurement Reliable measurement on the exposure factors and outcomes of interest in the study population “WITHOUT BIAS and ERRORS”or to minimize them to the least as possible

  5. To achieve a high standard quality study: Ensure right answers to study questions Good the study design Valid and reliable the measurements . Control for any possible bias Good cooperation between * research group and * study population

  6. X11, x12, x13 X31, x32, x33 X21, x22, x23 X61, x62, x63 X41, x42, x43 X51, x52, x53 Screening for fasting blood cholesterol profile among people x1 x2 x3 x6 x4 x5

  7. X11 X21 X31 X41 X51 X61 Screening for fasting blood cholesterol profile among people x1 x2 x3 x5 x6 x4

  8. Instrument or Research Tool • “equipment hard ware” • a red blood cell counter • a PH meter • an electronic weighing machine • “paper ware” • a questionnaire • a weekly diet diary • “people ware” • observers/investigators • technicians

  9. instrument tool measurement without bias or error minimize bias How good is the instrument or tool? • true value truth • measurement • valid/accurate • precise/reliable

  10. What is accuracy & precision? • What do you think of first when talking about validity & reliability? • What is the different between validity & reliability? • Why are validity & reliability important in conducting any medical research - both in laboratory & field setting?

  11. DEFINITION : A precise measurement in one that has nearly the same value each time it is measured SYNONYM • reliability • repeatability • reproducibility • consistency • agreement PRECISION

  12. IMPORTANT POINTS • precision depends on: • sample size • efficiency of the study • VIP influence on the power of a study • precision, reliability and consistency affected byRANDOM ERROR

  13. ASSESSING PRECISION • Using S.D.sVariance (s2) • Using Coefficient of variation = S.D. X • Using Kappa statistic • Using Cronbach's alpha

  14. Strategies for enhancing precision 1. standardizing measurement methods • preparing study protocols • preparing operations manual • writing specific guidelines or instructions for making each measurement • serving as basis for describing methods when results are reported

  15. Strategies for enhancing precision • preparing operations manual • write down precisely : - how to prepare environment and subject - how to carry out and record interview - how to calibrate instrument

  16. Strategies for enhancing precision • writing specific guidelines or instructions for making the measurement uniform performance over the duration of study

  17. Strategies for enhancing precision 2. Training and certifying the observers • improving consistency of measurement techniques (several observers) • performing pilot study • to test the power of techniques specified in operations manual

  18. Strategies for enhancing precision 3. Refining the instruments • writing or spelling out questionnaires and interviews to increase clarity 4. Automating the instruments • using automatic mechanical devices

  19. Strategies for enhancing precision 5. Repeating the measurement • impact of random error of any source can be reduced by • repeating measurement • using mean of the two or more readings

  20. DEFINITION : The degree to which the results of a measurement correspond to the true state or truth SYNONYM: validity conformity ACCURACY

  21. IMPORTANT POINTS • accuracy is a function of “SYSTEMATIC RROR” • VIP influence on theinternal and external validity of the study • the greater the systematic error, the less accurate the variable

  22. IMPORTANT POINTS • It is attributed to: • Methodological aspect of study design or analysis • Selection of subject • Quality of information obtained • Confounding • Effect Modification • Misclassification

  23. ASSESSING ACCURACY Comparison with reference techniques Gold standards

  24. Strategies for enhancing accuracy 1. Standardizing measurement methods 2. Training and certifying the observers 3. Refining the instruments 4. Automating the instruments 5. Making informal measures 6. Blinding 7. Calibrating the instrument

  25. MAJOR TYPES OF BIAS • Observer bias • Subject bias • Instrument bias • Information bias • Selection bias

  26. MAJOR TYPES OF BIAS • Observer bias consistent distortion in reporting measurement by observer - more intensive measurements in certain subjects - ask questions about specific exposures several times of cases but only once of controls

  27. MAJOR TYPES OF BIAS • Observer bias Ex. a tendency to underestimate blood pressure in cases known to be receiving treatment Ex. a more persistent search of medical records for a history of smoking cigarettes in patients known to have lung cancer

  28. MAJOR TYPES OF BIAS • Subject bias consistent distortion of measurement bystudy subject - selective recall or reporting of an event respondent bias or recall bias

  29. MAJOR TYPES OF BIAS • Instrument bias - may result from faulty function of a mechanical instrument - may result from inappropriate use of technique or tool to objective of measurement leading questions on questionnaire

  30. MAJOR TYPES OF BIAS • Information bias a distortion in the estimate of effect or variable due to: * measurement error * misclassification of subjects on measurement variable * invalid measurement

  31. MAJOR TYPES OF BIAS • Information bias * incorrect diagnostic criteria * inadequacies in previously recorded data * unequal diagnostic surveillance among exposure study groups in follow up studies

  32. MAJOR TYPES OF BIAS • Selection bias a distortion in the estimate of effect resulting from how subjects are selected for study population “self-selection bias”

  33. MAJOR TYPES OF BIAS Selection bias can result from: - choice of groups to be compared (in all types of studies) - choice of sampling frame - loss to follow up or NON RESPONSE during data collection (in follow-up studies)

  34. MAJOR TYPES OF BIAS Selection bias can result from: - selective surveillance/diagnostic surveillance varies with exposure status - more intensive measurements in certain subjects

  35. SUMMARY 1. Reliability : Precision, Reproducibility Random Error 2. Validity : Accuracy , Conformity Systematic Error Bias

  36. MAJOR TYPES OF BIAS • Observer Bias • Subject BiasRecall Bias Respondent Bias • Instrument Bias • Information Bias • Selection Bias

  37. Reliability Validity Reliabilityand validity of measurement The degree to which a variable actually represents what it is supposed to represent Definition The degree to which a variable has nearly the same value when measured several times Best way to assess Comparison among repeated measures Comparison with a reference standard

  38. Reliability Validity Value to study Increase power to detect effects Increase validity of conclusions Threatened by Random error (variance) contributed by : Systematic error (Bias) contributed by : The observer The subject The instrument The observer The subject The instrument

  39. . . . . . . . . . . . . . . . . . . . . . Illustration of the difference between Precision and Accuracy

  40. . . . . . . . . . . . . . . . . . . . . good precision poor precision good precision poor precision poor accuracy good accuracy good accuracy poor accuracy Illustration of the difference between Precision and Accuracy

  41. DIFFERENCES BETWEEN VALIDITY AND RELIABILITY Frequency A C Unreliable Invalid B D Measurement True value

  42. DIFFERENCES BETWEEN VALIDITY AND RELIABILITY Frequency A C A- Valid and reliable B- Valid but not reliable C- Not valid but reliable D- Not valid and not reliable Unreliable Invalid B D Measurement True value

More Related