How Much is Enough? An Examination of Behavioral Sampling

1. How Much is Enough? An Examination of Behavioral Sampling Maureen A. Conroy, Jennifer M. Asmus, Crystal N. Ladwig, & William H. Brown Presented at the 2004 Association for Behavior Analysis Annual Convention Boston, MA

2. Introduction • Obtaining a representative sample of data is critical (Moore, 1998) • Major limitation of many descriptive observation procedures is the potential for inaccurate or insufficient sampling of behavior (Freeman et al., 2000) • Total amount of observational time can influence the conclusions (Hindahl & Schwibbe, 1997; Karweit & Slavin, 1978)

3. Introduction • Estimates of time required to obtain a representative sample have varied • Arrington (1989) suggested a minimum of 24 five minute observations • Karweit & Slavin (1978) suggested 20 days of observation • Moore (1998) suggested that 16 - 20 eight minute observations is necessary • Clearly, there is no consensus regarding the amount of observation time required to obtain a representative sample • Odom & Ogawa (1992) reviewed 51 studies and found the total observation time varied from 6 minutes to 42 hours

4. Introduction • Factors that influence the amount of observation time needed to obtain a representative sample • Research time & budget constraints (Gay, 1992) • Frequency or duration of target behavior (Doll & Elliot, 1994; Mudford, Beale, & Singh, 1990) • Age of target population (Merrell, 1996) • Examination of contextual factors (Greenwood et al., 1985) • Researchers professional judgment (Foster & Cone, 1986)

5. Introduction • Further research is needed to examine the length of observation time needed to obtain a representative sample of behavior in order to guide researchers and practitioners • Further research is needed to examine the variables that influence the length of observation time needed

6. Purpose of Study • Overall Purpose • To examine the sampling of target behaviors across 10 hours of descriptive observation to determine the amount of time needed to obtain a representative sample • Specific Research Questions: • How many observation samples are needed to obtain a representative sample of behavior in natural settings? • Is the amount of observation time needed to obtain a representative sample in natural settings influenced by the type or base rate of the behavior?

7. Method • Participants • 3 children diagnosed with Autism Spectrum Disorders (ASD) • Ages 5 - 7 years old • Setting • General education classroom (K - 1st grade) (3 participants) • Home setting (1 participant) • Activities included structured activities with demands, such as academics (e.g., math, reading), homework, group work, individual, seatwork, 1:1 activities, and a variety of materials

8. Method • Dependent Measures • Disruption • Verbal talk or making loud noises out of context, throwing objects or work materials, destroying objects or work materials, eloping, failure to complete an instruction • Stereotypy • Hand/arm flapping or clenching, rocking/running back and forth, chair spinning, twirling hair, repetitive vocalizations. • Engagement • Appropriately engaging in task, such as, manipulating object or work materials, facing instructor or task, engaging in a social activity, or requesting information related to the task

9. Method • Observation Sampling Procedures • Observations were conducted for 15 - 20 minute sessions for a total of approximately 10 hours of data per participant across a 5 week period • Data was collected using a direct, continuous, sequential recording system during naturally occurring activities using HP Jornada palmtop computers • Data analysis was conducted using MOOSES (Tapp, 2002) • Visual representation of data • Trend line analysis at 5 hours of observation and 10 hours of observation

10. Method • IOA • IOA was calculated using A/A+D x 100 • IOA calculated on 29% - 40% of sessions • Mean occurrence only IOA = 89% (83%-94%)

11. Base Rates Per Minute Results

12. Stereotypy

13. Disruption

14. Engagement

15. Stereotypy Base rates of stereotypy ranged from low - moderate across participants Stereotypy was relatively stable with the exception of WTS Trend between 5 & 10 hours of observation are similar regardless of participant and different base rates Results

16. Disruption Base rates of disruption was very low across participants Variability of disruption was relatively low Trend between 5 & 10 hours of observation are similar regardless of participants Results

17. Engagement Base rate of engagement was moderate to high across participants Variability of engagement was relatively high Trend between 5 & 10 hours of observation varied depending on the participant WTS & WTH similar trends across 5 & 10 hours Within participant similarities JAH & DMS different trends across 5 & 10 hours Between participant dissimilarities Results

18. Data presented is from a small sample Observations occurred in natural settings during naturally occurring activities; therefore, the changing context may contribute to the variability of the data Number of daily sessions varied across days Limitations

19. For disruption and stereotypy 5 hours of observation may be sufficient to obtain a representative sample Low to moderate rates of behavior = 5 hrs For engagement 5 hours of observation may be sufficient for some participants to obtain a representative sample Variability of the data may influence the amount of observation needed Engagement for JAH & DMS were more variable in comparison to WTH & WTS Conclusions

20. Research Recommendations 5 hrs of observation time may be indicated for behaviors that occur at a low rate and are stable (e.g., disruption) < 5 hrs of observation time may be indicated for higher rate, less stable behaviors (e.g., engagement) Further research is needed to examine the generality of findings Further research directions Examination of the minimal amount of time needed to obtain a representative sample (i.e., Generalization Theory, Brennan, 2001) Examination of sampling procedures that may influence findings Future Directions & Recommendations

21. Practical Recommendations Minimize variability in observations by controlling for the time of day, activities/materials, length of session, and other contextual factors as indicated Observe during times when the rate of target behavior is most consistent across observations Continue observations until a stable behavior pattern is present rather than a predetermined limit Consider base rate of behavior and adults data collection accordingly (especially if evaluating behavior with different rates) Future Directions and Recommendations

22. Development of this paper was supported with funding from the U.S. Department of Education, Office of Special Education and Rehabilitative Services (No. H32D990024). The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the U.S. Department of Education.