Chapter Nineteen

Descriptive Methods • Do not involve manipulation of an independent variable • When we use descriptive methods, we can only speculate about causation that may be involved. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Archival and Previously Recorded Sources of Data • Refers to use of data recorded by other individuals for other purposes • (e.g. public health and census data) • The General Social Survey (GSS) is an archival source that can be accessed online • http://www.icpsr.umich.edu/gss/home.htm Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Archival and Previously Recorded Sources of Data • Potential Problems • You will not know exactly who left the data you are investigating. • The participants may have been selective in what they chose to write. • This problem is also known as selective deposit. • Archival and previously recorded sources of data may not survive long enough for you to make use of them. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Comparisons with the Experimental Method • We are not able to exercise any control with regard to gathering these data and cannot make any statements regarding cause-and-effect. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Observational Techniques • Case studies • Involves intense observation and recording of behavior of a single (perhaps two) participant(s) over an extended period of time. • There are no guidelines for conducting a case study and the procedures employed, behaviors observed, and reports produced may vary substantially. • Frequently used in clinical settings to help formulate ideas and hypotheses for further research. • Although case studies often provide interesting data, their results may be applicable only to the individual participants who was observed. That is, the researcher should not generalize beyond the individual participant who was observed. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Observational Techniques • Naturalistic Observation • Involves seeking answers to research questions by observing behavior in the real world. • The first goal of naturalistic observation is to describe behavior as it occurs in the natural setting without the artificiality of the laboratory. • The second goal of naturalistic observation is to describe the variables that are present and the relations among them. Smith/Davis (c) 2005 Prentice Hall

Psychological Detective • Why should the researcher be concealed or unobtrusive in a study using naturalistic observation? • The reactance or reactivity effect • Refers to biasing of the participants’ responses because they know they are being observed. • The reactivity effect is also known as the Hawthorne effect because of the location of the original study. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Participant Observation • The researcher becomes part of the group being studied. • Often used when the goal of the research project is to learn something about a specific culture or socioeconomic group. • Ethnography is a form of participant observation based on the anthropological tradition of research. • “Observer as participant” refers to a researcher who primarily observes a situation but who interacts with the others (Glesne, 1999). • “Participant as observer” refers to the researcher who becomes a part of the culture by working and interacting extensively with the others (Glesne, 1999). Smith/Davis (c) 2005 Prentice Hall

Psychological Detective • What are the drawbacks and weaknesses of the participant observer technique? • An extended period of time may be necessary before the participant observer is accepted as a member of the group that is under study. • Cannot make cause-and-effect statements. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Clinical perspective • Schein (1987) argued convincingly that the clinical perspective or model is not a subcategory of participant observation because: • A client typically chooses the clinician, whereas the participant observer chooses the others to be studied. • Unlike participant observers, clinicians cannot be unobtrusive because they have been asked to participate in the situation. • Although the participant observer can remain passive, clinicians must intervene in the situation. • The participant observer’s goal is understanding, whereas the clinician’s goal is helping. • Participant observers validate their findings by replication while clinicians validate their findings by being able to predict the results of a given intervention. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Choosing Behaviors and Recording Techniques. • Time sampling • Involves making observations at different time periods in order to obtain a more representative sampling of the behavior of interest. • Selection of time periods may be determined randomly or in a more systematic manner. • The use of time sampling may apply to the same or different participants. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Choosing Behaviors and Recording Techniques. • Situation sampling • Involves observing the same behavior in several different situations. This techniques offers the researcher two advantages: • By sampling behavior in several different situations, you are able to determine whether the behavior in question changes as a function of the context in which you observed it. • You are likely to observe different participants in the different situations and because different individuals are observed, your ability to generalize any behavioral consistencies across the various situations is increased. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Deciding how to present the results of your research project. • Qualitative presentation of results: • Report consists of a description of the behavior in question (a narrative record) and the conclusions prompted by this description. • Quantitative or numerical presentation of results: • Need to know how behavior under investigation is going to be measured and how these measurements will be analyzed. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Using More than One Observer • There are two main reasons for using more than one observer: • One observer may miss or overlook a bit of behavior. • There may be some disagreement concerning exactly what was seen and how it should be rated or categorized. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • Using More than One Observer • When two individuals observe the same behavior, it is possible to see how well their observations agree. • The extent to which the observers agree is called interobserver reliability. • Low interobserver reliability indicates that the observers disagree about the behavior(s) they observed. • High interobserver reliability indicates agreement. Smith/Davis (c) 2005 Prentice Hall

Descriptive Methods • The formula for calculating interobserver reliability is as follows: • 85% agreement is generally considered to be an acceptable minimum level for interobserver reliability. Smith/Davis (c) 2005 Prentice Hall

Qualitative Research • Qualitative research • “is defined as an inquiry process of understanding a social or human problem, based on building a complex, holistic picture, formed with words, reporting detailed views of informants, and conducted in a natural setting” (Creswell, 1994, p. 2) • The qualitative research style is much less formal and impersonal, and the reader of a qualitative research report can expect to find such additions as “definitions that evolved during a study” (Creswell, 1994, p. 7) Smith/Davis (c) 2005 Prentice Hall

Qualitative Research • The qualitative researcher believes that a full description of human behavior includes people’s feelings in addition to what they are doing and how they are doing it. • The qualitative researcher is committed to studying particular people in specific settings. • Qualitative researchers prefer to use inductive logic. • Qualitative research begins with guiding hypotheses reflecting a global issue of interest (Marshall & Rossman, 1989). • Qualitative researchers typically analyze their data “simultaneously with data collection, data interpretation, and narrative reporting writing” (Creswell, 1994, p. 153). Smith/Davis (c) 2005 Prentice Hall

Correlational Studies • A correlational study involves the measurement and determination of the relation between two variables. • In terms of control, empirical measurement, and statistical an analysis, a correlational study is likely to be more rigorous than one of the descriptive methods. • Researchers use correlational studies when data on two variables are available, but we can only measure, rather than manipulate, either variable. • Although we can determine the degree of relation that exists between these two variables, we are not able to offer a cause-and-effect statement concerning these two variables. Smith/Davis (c) 2005 Prentice Hall

Ex Post Facto Studies • Ex post facto – is a Latin phrase meaning “after the fact.” • When we conduct an ex post facto study, we are using an IV “after the fact” – it has already varied before we arrived on the scene. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Surveys typically request our opinion on some topic or issue that is of interest to the researcher. • Types of Surveys • Descriptive – seeks to determine what percentage of the population has a certain characteristic, holds a certain opinion, or engages in a particular behavior. • Examples: • The Gallup Poll • Nielsen television ratings • Analytic –seeks to determine what the relevant variables are and how they might be related. • Questions need to be chosen carefully. • Pilot testing is generally necessary to determine the type of questions that will be used in the final survey instrument. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Step 1 – decide what type of instrument to use. How will the information be gathered? • Will you use a mail survey, conduct a phone interview, or conduct the survey in person? Will you use trained interviewers? • Step 2 – identify the types of questions to use. • Yes-No Questions • The respondent answers yes or no to the items. • Forced Alternative Questions • The respondent must select between two alternative responses. • Multiple-Choice Questions • The respondent must select the most suitable response from among several alternatives. • Likert-Type Scales • The individual answers a question by selecting a response alternative from a designated scale. A typical scale might be the following: (5) strongly agree, (4) agree, (3) undecided, (2) disagree, or (1) strongly disagree. • Open-Ended Questions • A question is asked to which the respondent must construct his or her own answer. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Step 3 – Write the items: They should be clear, short, and specific. • Use familiar vocabulary and be at the reading level of the individuals you intend to test. • Step 4 – Pilot-test and seek opinions from knowledgeable others. • It is important to ask others, especially professionals who have expertise in your area of research interest, to review your items. They may be able to detect biases and unintended wordings that you had not considered. • Step 5 – determine the relevant demographic data to be collected. • Includes items such as age, sex, annual income, size of community, academic major, and academic classification. • Step 6 – determine administration procedures and develop instructions. • Instructions must be clear, concise, easy to follow, and thoroughly rehearsed (for in-person or phone interviews). • How will informed consent be obtained? Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Survey Administration Procedures • Mail Surveys • Advantages • The researcher does not have to be present while the survey is being completed. • Surveys can be sent to a much larger number of participants than a single researcher could ever hope to contact in person. • Disadvantages • The researcher cannot be sure who actually completes the survey. • There is no guarantee the respondent answered the questions in the same order in which they appeared in the survey. • Low return rate. Smith/Davis (c) 2005 Prentice Hall

Psychological Detective • Assume that you are planning to conduct a mail survey project. You are concerned about the possibility of having a low response rate and want to do everything to ensure the return of your surveys. What can you do to increase your response rate? • The initial mailing should include a letter that clearly summarizes the nature and importance of the research project, how the respondents were selected, and the fact that all responses are confidential. You should include a prepaid envelope for the return of the completed survey. • It may be necessary to send an additional mailing(s) to respondents. Because the original survey may have been misplaced or lost, it is important to include a replacement. One extra mailing may not be sufficient; you may find it necessary to send two or three requests before you achieve an acceptable response rate. These extra mailings are typically sent at two- to three-week intervals. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Personal Interviews • Advantages • When a trained interviewer administers a survey in a respondent’s home, the response rate climbs dramatically. • The trained interviewer is able to clarify ambiguous questions, making sure that all questions are answered in the proper sequence, and generally assisting with any problems that the respondents may experience. • Disadvantages • The potential for considerable expenditure of time and money exists. • People not always at home. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Telephone Interviews • Advantages • It is easy to create random samples with random-digit dialing. • 95% of households in the U.S. have telephones. • Computer technology allows responses to be entered directly as they are made. • Disadvantages • Incoming calls may be screened and potential respondents lost . • It is easy to say no to a phone interviewer. • Cannot use visual aids to clarify questions. • Cannot evaluate nonverbal cues such as facial expressions, gestures, and posture. • It is more difficult to establish rapport. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Tests and Inventories • Tests and inventories are designed to assess a specific attribute, ability, or characteristic possessed by the individual being tested. • Characteristics of Good Tests and Inventories • Validity – a test or inventory has validity when it actually measures what it is supposed to measure. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Validity (ways to establish) • Content validity – indicates that the test items actually represent the type of material they are supposed to test. A panel of expert judges is often used to assess the content validity of test items. Their degree of agreement is known as interrater reliability. • Concurrent validity – is established when a score on a test or inventory agrees with another measure (e.g. aggression score on a test agrees with a clinician’s assessment). • Criterion validity – is established when a test score compares favorably with a predicted future outcome (e.g. SAT score and college success). Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Reliability - refers to the extent that the test or inventory is consistent in its evaluation of the same individuals over repeated administrations. • The greater the similarity between scores produced by the same individuals on repeated administrations, the greater the reliability of the test or inventory. Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Reliability (assessment) • Test-retest procedure • The test is simply given a second time and scores from the two tests are compared; the greater the similarity, the higher the reliability • Split-Half technique • Involves dividing a test or inventory into two halves or subtests and then administering them to the same individuals on different occasions or by administering the entire test and then splitting it into two halves. Smith/Davis (c) 2005 Prentice Hall

Psychological Detective • On the surface, the test-retest procedure appears to be quite straightforward and reasonable. However, there may be a problem with establishing reliability in this manner. What is it? • The main problem with test-rest concerns the fact that the participants are repeatedly administered the same test or inventory. Having already taken the test or inventory may result in the individuals remembering the questions and answers the next time the instrument is administered (thus allowing potential bias). Smith/Davis (c) 2005 Prentice Hall

Surveys, Questionnaires, Tests, and Inventories • Types of Tests and Inventories • Achievement tests • Are given when an evaluation of an individual’s level of mastery or competence is desired. • The score that distinguishes passing from failing determines the minimum level of achievement that must be attained. • Aptitude tests • Are used to assess an individual’s ability or skill in a particular situation or job. • Personality tests or inventories • Measure specific aspects of an individual’s motivational state, interpersonal capability, or personality. Smith/Davis (c) 2005 Prentice Hall

Chapter Nineteen