1 / 21

Measures of Reliability and Multivariate Analysis

Measures of Reliability and Multivariate Analysis. Objectives. To discuss measures of reliability To discuss multivariate analysis. Kappa Statistic. p o = observed probability of concordance between the two surveys p e = expected probability of concordance between the two surveys

ulysses-soto
Download Presentation

Measures of Reliability and Multivariate Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measures of Reliabilityand Multivariate Analysis

  2. Objectives • To discuss measures of reliability • To discuss multivariate analysis

  3. Kappa Statistic • po = observed probability of concordance between the two surveys • pe = expected probability of concordance between the two surveys • The standard error of the Kappa statistic is calculated by: • To test the hypothesis Ho:k=0 vs. H1:k>0, use the test statistic:

  4. Percent agreement

  5. Figure 1. Association of average faculty performance rating (from 1, bottom 20%, to 5, top 20%) and absolute rank on the National Resident Matching Program (NRMP) list (r = 0.19; P =.11).

  6. Table 1. Discrepancy Between the DIS and SCAN for the Lifetime Occurrence of Depressive Disorder in the Baltimore ECA Follow-up*

  7. Table 1-Comparison of WHO and ADA diagnostic categories for undiagnosed diabetes From:   Lee: Diabetes Care, Volume 23(2).February 2000.181-186

  8. Kappa score calculation

  9. Kappa and prevalence • You have two populations of 1000 each being compared by interview A and interviewer B • One has a prevalence of 5 and the other has a prevalence of 40 • Sensitivities and specificities for observer A and B are .90 and .90 and .80 and .95 respectively • Create a table for each population

  10. Prevalence and Kappa • Kappa tends toward 0 as the prevalence approaches either 0 or 1 • High values of kappa can be obtained when the marginals of the contingency table are unbalanced. (tends to be higher when the positivity prevalence is different between A and B

  11. Kappa scores for table 8-18

More Related