Data Collection, Measurement, & Data Quality in Quantitative and Qualitative Research

1. Data Collection, Measurement, & Data Quality in Quantitative and Qualitative Research

2. Data Collection Methods • Without appropriate data collection methods, the validity of research conclusions is easily challenged

3. Data Collection Methods • Using New Data • Collect own data for the study

4. Data Collection Methods • Using Existing Data • Historical research • Use records and other documents from the past • Secondary analysis • Use of data gathered in a previous study

5. Key Dimensions of Data Collection Methods • Structure • The data collection should be very structured and consistent • Quantifiability • Able to be analyzed statistically • Obtrusiveness • Degree to which people are aware that they are being studied • Objectivity • Try to be as objective as possible

6. Data Collection Quantitative Research

7. Types of Data Collection • Self-Reports • Observation • Biophysiologic Measures

8. Types of Data Collection • Self-Reports • Interviews • Questionnaires • Scales • Vignettes • Projective techniques • Q-sorts

9. Types of Data Collection: Self-Reports Interviews and Questionnaires (Structured) • Participant's responses to questions by researcher • Data is usually collected by means of a formal, written document (instrument) • Uses an interview schedule for questions that are asked orally (face to face or via phone) • Uses a questionnaire when participants complete the instrument themselves

10. Types of Data Collection: Self-Reports Interviews and Questionnaires (Structured) • Closed-ended questions (fixed alternative questions) • Response alternatives are specified by the researcher • Ensures comparability of responses • Facilitates analysis • Easy to administer • More efficient time use • Difficult to develop • Could lead to overlooking something important

11. Types of Data Collection: Self-Reports Interviews and Questionnaires (Structured) • Open-ended questions • Allows participants to respond to questions in their own words • Allows for richer, fuller information

12. Types of Data Collection: Self-Reports Interviews and Questionnaires (Structured) • Instrument Construction • Develop outline of content of research • Design questions • Pretest • Trial run to determine if instrument is free of biases, errors, etc

13. Types of Data Collection: Self-Reports • Interviews Vs. Questionnaires • Advantages of questionnaires • Less costly • Require less time and effort to administer • Can be completely anonymous • No biases relating to the researcher being present

14. Types of Data Collection: Self-Reports • Interviews Vs. Questionnaires • Advantages of Interviews • Response rate is higher in face to face interviews • Effective for those that can not complete questionnaires (children, blind, ESL, elderly) • Questions are less likely to be misinterpreted than questionnaires • Interviews can produce additional information through observation

15. Types of Data Collection: Self-Reports • Interviews Vs. Questionnaires • Interviews are considered to be superior to questionnaires

16. Types of Data Collection: Self-Reports Types of Self-Reports (Structured) • Composite Scales (social - psychological) • Vignettes • Projective techniques • Q sorts

17. Types of Data Collection: Self-Reports Composite Scales (social - psychological) • Scale: assigns a numeric score to people to place them on a continuum with respect to attributes being measured

18. Types of Data Collection: Self-Reports Composite Scales (social - psychological) • Likert scale • Semantic Differential scale • Visual Analog scale

19. Types of Data Collection: Self-Reports Composite Scales (social - psychological) • Likert scale (summated rating scales) • Consists of several declarative statements that express a viewpoint • Participant indicates the degree to which they agree to disagree • Able to summate the scores allowing for discrimination among people with different viewpoints

20. Types of Data Collection: Self-Reports Composite Scales (social - psychological) Example Likert Scale: AU nursing students are very well prepared for working within the current healthcare system Strongly agree Agree Neutral Disagree Strongly disagree

21. Types of Data Collection: Self-Reports Composite Scales (social - psychological) • Semantic Differential • Participants rate a concept on a series of bipolar adjectives • Can measure any concept • Visual Analog Scale • The scale is a straight line with anchors which are the extreme limits of the experience or feeling • Measures subjective experiences

22. Types of Data Collection: Self-Reports • Semantic Differential Example AU nursing graduates are: Competent Incompetent Intelligent Dim • Visual Analog Scale Example On a scale of 0 to 10 how would you rate your pain if 10 was the worst pain you have even experienced and 0 was no pain

23. Advantages of Scales • Scales allow researchers to efficiently quantify the strength and intensities of individual characteristics • Discriminates among people with different attitudes, fears, motives, perceptions, personality traits, needs • Good for group and individual comparisons • Can be implemented either verbally or in writing

24. Disadvantages of Scales Response set biases • Social Desirability Response Set Bias • Participants give answers that are common social views • Extreme Response Set Bias • Participants express attitudes or feelings in the extreme (always, never) • Acquiescence Response Set Bias • Participants agree with all statements (yea-sayers or nay-sayers)

25. Disadvantages of Scales • Ways to Reduce Response Set Biases • Counterbalancing: positively and negatively worded statements • Developing sensitively worded questions • Creating a permissive, nonjudgmental atmosphere • Guaranteeing confidentiality

26. Types of Data Collection: Self-Reports Vignettes • Brief description of events or situations to which participants are asked to react • Information about perceptions, opinions, or knowledge • Questions post vignettes may be open-ended or close-ended • Economical to administer • May contain response biases

27. Types of Data Collection: Self-Reports Projective Techniques • Verbal self reports to obtain psychological measurements • Seek minimal participants’ conscious cooperation • Ambiguous or unstructured stimuli elicits participants needs, motives, attitudes, personality traits i.e. Inkblot test, word association, role playing, drawing • Useful in children, hearing or speech impaired

28. Types of Data Collection: Self-Reports Q Sorts • Uses a set of card with words, phrases or statements • Participant sorts cards along a bipolar dimension (agree/disagree)

29. Advantages of Self-Reporting Methods • Most common method of data collection used by nurses • Reveal information that is difficult to obtain by other means • Can gather retrospective and prospective data • Can measure psychological characteristics

30. Disadvantages of Self-Reporting Methods • Questionable validity and accuracy • Biases

31. Types of Data Collection: Observation • Observational Methods • An alternative to self-reports • Can be used to gather information such as characteristics, condition of individuals, verbal communication, nonverbal communication, activities, environmental conditions

32. Types of Data Collection: Observation Observational Methods • Researcher has flexibility in the following areas: • The focus of observation • What events are to be observed • Concealment • Duration of observation • Method of recording observations

33. Types of Data Collection: Observation Observational Methods (structured) • Categories and checklists • Rating Scales

34. Types of Data Collection: Observation Categories and Checklists • Category system: • attempts to designate information in a systematic, quantitative manner • Clear definition of behaviors and characteristics to be observed is necessary • Lists all behaviors or activities the observer wants to observe and records occurrences • Checklist: • instrument to record observations • Rating Scales: • Are tools that require the observer to rate some phenomena along a descriptive continuum

35. Types of Data Collection: Observation • Observational Sampling • Time sampling • Selection of time periods for observations • Event sampling • Selects behaviors or events for observation

36. Evaluation of Observational Methods • Advantages • Provides depth and variety of information • Some problems are better suited to observation • Disadvantages • Potential ethical issues • Lack of consent to be observed • Participants reaction to be observed • Biases • Faulty inferences

37. Types of Data Collection Biophysiologic

38. Types of Data Collection: Biophysiologic Types of Biophysiologic Measures • In vivo • Measures performed directly within or on living organisms • i.e. blood pressure, temperature • In vitro • Data gathered from participants by extracting some biophysiologic material from them for lab analysis • i.e. blood work, microbiologic measures, cytology and histological measures

39. Advantages of Biophysiologic Measures • Are relatively accurate and precise • Are objective • Provide valid measures of targeted variables • Equipment is readily available

40. Disadvantages of Biophysiologic Measures • Measuring tool may affect variables it is attempting to measure • Interferences may create artifact • Energy must often be applied to the organism when taking measurements

41. Measurement and Assessment of Data

42. Measurement • Involves rules for assigning numeric values to qualities • Determines how much of an attribute is present • Quantification • Communicates the amount in numbers

43. Advantages of Measurement • Removes guesswork in gathering information • Tends to be objective • Obtains precise information • Can differentiate among people who possess different degrees of an attribute • Common language

44. Errors of Measurement • Always the potential for error in all tools • Extraneous factors affect measurement and distort results • Obtained score – is observed score • True score – true score if no errors • Error of measurement – the different between the true and obtained scores

45. Factors Contributing to Errors of Measurement • Situational contaminants • People’s awareness of observer, environmental factors • Response set biases • Transitory personal factors • Fatigue, mood, hunger (temporary) • Administration variations • Alterations in data collection methods • Item sampling • Errors introduced as a result of sampling

46. Reliability of Measuring Instruments Reliability • Refers to the consistency with which an instrument measures the attribute • The less variation in repeat measures the higher its reliability

47. Reliability of Measuring Instruments Reliability • Aspects of reliability • Stability • Internal consistency • Equivalence

48. Reliability of Measuring Instruments • Stability • The extent to which the same scores are obtained when the instrument is used with the same people on separate occasions • To assess stability: Test-retest reliability • researcher administers the same measure to a sample of people on two occasions and then compares the scores

49. Reliability of Measuring Instruments • Internal Consistency • Reliable to the extent that all its subparts measure the same characteristic • To assess internal consistency: Split-half technique • the items comprising the test or scale are split into two groups and scored, compute reliability coefficient

50. Reliability of Measuring Instruments • Equivalence • Determines the consistency or equivalence of the instrument by different observers or raters • To assess equivalence – interrater (interobserver) reliability • Has two or more trained observers make simultaneous, independent observations, compete reliability coefficient