Procedures for Observation in Quantitative Research

1. Procedures for Observation in Quantitative Research Defining Observational Variables Recording Observations Selecting an Observation Recording Procedure Selecting and Training Observers Reducing Observer Effects

2. Defining Observational Variables • Important to define the variables that are to be observed • Types of Observational Variables: • Descriptive observational variables: require little inference on the part of the observer (e.g., low-inference variables, yield reliable data) • Inferential observational variables: require the observer to make an inference from behavior to a construct that is presumed to underly the behavior • Evaluative observational variables: require not only an inference from behavior on part of the observer but also an evaluative judgment

3. Recording Observations • To ensure accurate recording, observers should be required to record data on only one observational variable at a time • Four major types of recording observations: • Duration recording (stop watch, time-on/off-task) • Frequency-count recording (occurrence of behavior) • Interval recording (behavior at given intervals) • Continuous recording (particular setting, such as a classroom)

4. Selecting Observation Recording Procedure • Standard observation forms (effective, prior use and reliability) • Use of audiotape and videotape recorders (advantages and disadvantages) • Use of computers and other electronic devices

5. Selecting and Training Observers • Advantage of using other individuals is that it allows for control of the observer bias that can occur when the same individual who designs the research study and frames its hypotheses also does the observing • Determining Observer Agreement: • Criterion-related observer reliability (observer’s score vs. expert observer/researcher’s score) • Intra-observer reliability (consistent with own observational coding) • Inter-observer reliability (pairs of observers)

6. Reducing Observer Effects • Observer effect is any action by the observer that has a negative effect on the validity or reliability of the data they collect • Effect of the observer on the observed (e.g., classroom, teaching, etc.) • Observer personal bias (bias toward certain racial/ethnic groups, etc.) • Rating errors (response set: error of leniency, error of central tendency, halo effect) • Observer contamination • Observer omission

7. Procedures for Observation in Qualitative Research The Purpose of Observation in Qualitative Research Identifying the Observers and Their Role Preparing for Observation Determining the Focus of Observation Gaining Entry into the Field Setting Recording Observations Dealing with Observer Effects Analyzing Qualitative Observational Data

8. Differences b/w Qualitative and Quantitative Research Qualitative Research: • Observers do not seek to remain neutral or “objective” about the phenomena being observed (include own feelings and experiences in interpreting their observations) • Focus is much more emergent (observers are free to shift their attention to new phenomena as new research questions emerge) • Focus is generally much wider compared to quantitative research (i.e., holistic perspective)

9. Purpose of Observation • Two common methods of data collection in qualitative research: interviews and analysis of documents – involve words uttered or written by the participants in the natural setting. • This information is limited by participants’ knowledge, memory, and ability to convey information clearly and accurately and, also, by how they wish to be perceived by outsiders such as the researchers. • Observation, allows researchers to formulate their own version of what is occurring, independent of the participants • “Triangulation” of data: inclusion of different sources of data to verify information obtained by other methods • Examples: ethnography, cultural studies, cognitive psychology (Chapter 15, GBG)

10. Identifying the Observers and their Role • Complete observer: researcher maintains a posture of detachment from the setting being studied • Complete participant: researcher studies a setting in which he/she already is a member or becomes converted to genuine membership during the course of the research • Observer-participant role: researcher acts primarily as an observer, entering the setting only to gather data and interacting only casually and non-directly with individuals or groups while engaged in observation • Participant-observer role: researcher observes and interacts closely enough with individuals to establish a meaningful identify within their group; however, the researcher does not engage in activities that are at the core of the group’s identity.

11. Preparing for Observation • Take courses in qualitative methods • Serve as an apprentice to an expert in the type of observation being planned • Observation skills are complex and subtle (takes practice and time) • By working with an expert, a novice observer can develop an understanding of how to focus his/her observations and to shift the focus across the three states (descriptive, focused, and selected)

12. Determining the Focus of Observation Three Stages (Spradley, 1980) Descriptive stage: when observations tend to be unfocused and general in scope, providing a base from which the observers can branch out in many directions. Focused stage: when the observers have identified features of the phenomena under study that are of greatest interest and begin to direct their attention to collecting deeper information about this narrower range of features. Selected stage: when research questions or problems emerge, and the observer’s focus shifts to refining and depending their understanding of the specific elements that have emerged as theoretically or empirically most essential.

13. Gaining Entry into the Field Setting • No strict rules about how to enter a field setting to make observations • Need to develop a procedure based on the characteristics of the field setting and its members and on where you intend to situate yourself along the continuum of complete participant to complete observer • Consult with expert qualitative researchers and by reading reports of their studies

14. Recording Observations • Take field notes using different methods (e.g., laptops, notepads; dictate notes into an audiotape recorder • Identify methods that are not intrusive to the observed • Examples: tablet, notebook, toilet paper, inside a matchbook cover • If researcher is unable to take notes in the field setting, you will need to remember what occurred and take notes thereafter • Field notes should be descriptive and reflective (detailed and concrete)

15. Dealing with Observer Effects • Qualitative researcher operates on the premise that observations should be independent of the particular individual who is making them • Effort is made to minimize observer bias and to control for possible effects of the observer on what is observed • As observers, they consider their biases and personal reactions to be part of the “scene” that is observed • Don’t overestimate or underestimate their effects on what is being observed • Consider observers’ attention to the effects of observations on themselves • Observer’s personal predisposition or biases (establish procedures for validating and verifying data analyses to reduce distortions) • Observer incompetence (insufficient preparation to do the type of data collection)

16. Analyzing Qualitative Observational Data • Extensive field notes and visual data that serve as a record of their observations • Takes time • Use software programs (text software, etc.) • Data need to be analyzed, interpreted, and reported • Chapter 14 (GBG) describes methods for analyzing

17. Nonreactive Observation • Unobtrusive Measures • Validity • Reliability • Ethnical consideration • Qualitative observation of Material Culture

18. Content Analysis of Documents • Content Analysis: defined as “a research technique for the objective, systematic, and quantitative description of the manifest content of communication.” • Raw material for content analysis can be any type of document or other communication medium (e.g., written documents, visual media, audio media, combinations of media, etc.) • Analysis of Document and Records • Documents (e.g., written communications that are prepared for personal rather than official reasons) – personal letters, diaries, draft of articles • Records (e.g., written communications that have an official purpose) – legal contracts, commission reports, tax statements, newspaper articles