1 / 7

IOA

IOA. Three reasons for IOA: Consistency (i.e., reliable recording) Inconsistent data doesn’t reflect actual performance Intervention’s role in changing Bx difficult to evaluate (Don’t know if variation is due to actual performance or who took data) Minimize biases

clarice
Download Presentation

IOA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IOA • Three reasons for IOA: • Consistency (i.e., reliable recording) • Inconsistent data doesn’t reflect actual performance • Intervention’s role in changing Bx difficult to evaluate (Don’t know if variation is due to actual performance or who took data) • Minimize biases • Single observers may change definitions • Expectations of improvement • IOA ensures TB is well defined

  2. Agreement vs. Accuracy • Agreement: evaluation of how well data from separate observers correspond • Accuracy: evaluation of how well an observer’s data reflect actual performance • Accurate but low IOA • Poor accuracy but high IOA

  3. Methods of IOA • Frequency Ratio • Compares totals of observers • Can be used for frequency, intervals of Bx, or duration Frequency Ratio = small/large X 100 • Frequency ratios limited: Don’t know if observers agree on any one instance of Bx

  4. Methods of IOA (Cont.) • Point-by-Point Agreement • Assess agreement on each instance of Bx • Use when there are discrete opportunities for Bx (e.g., present-absent; correct-incorrect, etc.) Point-by-Point IOA = A/A + D X 100 • Considerations: • How should agreements be scored?

  5. Methods of IOA (Cont.) • IOA for frequency-within-interval recording • Subtract # of disagreements from total # of intervals • For each interval that was scored as a disagreement, divide the smaller frequency by the larger frequency • Add the sum of the quotients to the number of agreement intervals • Divide by the total # of intervals

  6. Base Rates and Chance • Base rate of Bx contributes to level of agreement • If Bx occurs frequently, agreement on occurrences will be higher (inflated) • “Chance” – level of agreement expected by randomly marking occurrences for a given # of intervals • How high chance agreement is depends on what counts as an agreement • Vary definition of agreement • Report reliability separately for occurrence and nonoccurrence intervals • Plot both primary and secondary observers’ data • Use correlational statistics

  7. Sources of Bias • Reactivity • Bx may change when people realize they’re being monitored • Observer Drift • Observers may change the way they define Bx over time • Observer Expectancies & Feedback • Expectancies alone not likely to influence observations • Expectancies combined with feedback do influence observations • Complexity of the Observations

More Related