Data collection (1) Quantitative: Structure, Process and Outcome

1. Data collection (1) Quantitative:Structure, Process and Outcome Clare Robertson HSRU is funded by the Chief Scientist Office of the Scottish Government Health Directorates. The author accepts full responsibility for this talk.

2. Outline • What to measure – structure, process, outcome • Studies of structure, process and outcome • Outcome measurements • Developing a new scale

3. Purpose of HSR • To provide information regarding health care delivery ‘Health services research must provide the evidence by which health services activities are judged’ Crombie IK (1996)

4. Research process • Identify the research question eg Is keyhole surgery better than open surgery for hernia repair? • Decide upon research design eg randomised controlled trial • Choose what to measure eg recurrence of hernia • Decide how to measure eg physical examination at 1 year

5. Health care system

6. Deciding what to measure I • Structure the physical features of health care eg premises, number of staff, range and type of equipment, medical records • Process interaction between health professional and patient eg examinations, investigations, referrals, interactions

7. Deciding what to measure II • Intermediate (surrogate) outcome describes measures which may be either process or short-term outcomes which are closely related to the outcome of interest and highly predictive of it eg measuring blood pressure when evaluating treatments for stroke

8. Deciding what to measure III • Outcome changes in a patient’s current and future health status that can be attributed to antecedent health care(the resulting changes in the health of that patient) eg quality of life measures, survival, cure rate

9. EXAMPLE- MRSA MRSA (sometimes referred to as the superbug) methicillin-resistant Staphylococcus aureus. MRSA infections are more difficult to treat due to the antibiotic-resistance of the bacteria. • septicaemia (blood poisoning), • septic shock (widespread infection of the blood that leads to a fall in blood pressure and organ failure), • severe joint problems (septic arthritis), • Bone marrow infection (osteomyelitis), • internal abscesses anywhere within the body, • inflammation of the tissues that surround the brain and spinal cord (meningitis), • lung infection (pneumonia), and  • infection of the heart lining (endocarditis).

10. Example - MRSA • Those in hospital are more likely to develop MRSA infections because they often have an entry point for the bacteria to get into their body, such as a surgical wound, a catheter, or an intravenous tube. • MRSA infections are diagnosed by testing blood, urine or a sample of tissue from the infected area for the presence of MRSA bacteria. • MRSA is usually passed on by human contact, often from the skin of the hands.

11. MRSA- prevention • Organisation • Regular cleaning of hospitals • Health care professionals • Handwashing -fast-acting, special antiseptic solutions (alcohol rubs or gels) before examination of patients • disposable gloves when they have physical contact with open wounds

12. MRSA- prevention • Patients • Keep hands and body clean • wash your hands after using the toilet (hand wipe) • wash your hands or clean them with a hand-wipe before and after eating • Visitors handwashing when enter patient areas

13. Structural studies of care I When appropriate • exploratory studies in new areas • ensuring minimum standard of care • structure is expected to be strongly related to outcome Methods • observation survey • questionnaire survey

14. Structural studies of care II Advantages • easy to conduct • valuable when strong link between structure and outcome Disadvantage • link between structure and outcome is tenuous

15. Process studies of care I When appropriate • process expected to be strongly related to outcome (intermediate outcome) • quality of care • mechanisms- (behavioural) Methods • observation of process • self report instruments • case note review • routine data sources

16. Process studies of care II Advantages • easy to conduct (?behavioural change) • stronger links between process and outcome Disadvantages • no guarantee that changes in process will improve outcome

17. Outcome studies of care I When appropriate • any evaluation Methods • self administered outcome measure • interview administered outcome measure • case note review • routine data sources • trial report forms = web based(clinical outcomes)

18. Outcome studies of care II Advantages • ‘gold standard’ for evaluation Disadvantages • difficult to measure: validity reliability timing of evaluation accessibility feasibility • problems of attribution • valuation of costs

19. Difficulties associated with outcome measurement • Consider • How specific an outcome should be measured? • Whose values should be used? • When should assessment take place? • How will it be assessed? • Subjective experience of health varies across individuals and time What would you measure to assess the effects of the intervention on the ‘health’ of patients?

20. What type of outcome? • critical events • condition/disease specific • clinical • biomedical/biological markers • symptom • functional • educational • economic How general?

21. What type of outcome? How general? (continued) • client group specific • general • health profiles • health indices using indirect valuation • health indices using direct valuation • patient satisfaction, health related knowledge and behaviour

22. Examples: Condition specific outcome measurement I • clinical • biomedical/biological markers eg peak flow rate, blood pressure • symptoms eg wheeze,

23. Examples: Condition specific outcome measurement II • functional eg disturbed nights in previous month • educational eg knowledge about asthma drugs • economic eg time and travel to clinic

24. Examples: Client group specific II Hospital Anxiety and Depression Scale (HADS) • anxiety • 7 items (4 point scale) • depression • 7 items (4 point scale)

25. Examples: General outcome measurement I Health profile: eg SF-36 (36 items in 8 dimensions) • General health perceptions (5 items) • Physical function (10 items) • Role limitation physical (4 items) • Role limitation emotional (3 items) • Mental health (5 items) • Pain (2 items) • Social functioning (2 items) • Energy (4 items)

26. Examples: General outcome measurement II Health indices • single value for the health status of each patient surveyed • either indirect valuation or direct valuation

27. Examples: Health indices I Indirect valuation: • EUROQOL-5D (EQ-5D) • 5 dimensions represented by one value • mobility • self care • usual activities • pain • anxiety • Visual analogue scale (VAS)

28. Examples: Health indices II Direct valuation: enables patients to value their health status directly without reference to components of that health status eg time trade off standard gamble magnitude estimate

29. Example • Compare two different “types” of knee replacement surgery in patients • What outcomes might you be interested in?

30. Developing a new scale • Identify any suitable published scales • Devise new items • Select items • Avoid bias • Combine items to form a scale • Assess reliability • Assess validity

31. Devising new items • Sources • literature review • clinical observation • expert opinion

32. Devising new items • Defining responses • binary • categorical • ordinal • interval • continuous

33. Selecting items • Interpretability • reading level Flesch score • jargon how long have you had hypertension? • value laden words do you often go to your doctor with trivial problems? • positive and negative wording not, rarely, never I rarely feel well agree/disagree

34. Assessing validity I • Face validity appear to be assessing desired qualities • Content validity sampling all relevant and important areas • Criterion validity correlated with gold standard • concurrently - new simple scale and SF36 • predictive - prescribing behaviour scale and actual prescribing

35. Assessing validity II • Construct validity scale correlated with other measures of underlying construct or measures independent of construct time trade off and SF36 time trade off and sleep patterns

36. Assessing reliability • Influence of observers • Intra observer • Inter observer • Test-retest reliability • Internal consistency

37. Avoiding bias I • Social desirability • unwilling to report things they see as socially unacceptable • Acquiescent • agree with statements regardless of what the content is • disguise intent of scale • use subtle items • random response technique

38. Example Imagine you want to evaluate the implementation of a shared care scheme with guidelines for those newly diagnosed with asthma (cough and wheeze) Usual care = see general practitioner only as per usual care Shared care = see general practitioner and go to asthma clinic (3 monthly in 1st year/ 6 monthly yr 2 and 3) to see consultant – re use of asthma drugs, asthma control GP given guidelines re referral and treatment

39. Questions What would you measure (& how & when)? Structure, process and outcome? Consider: advantages/disadvantages

40. Further reading • Bowling A. Research methods in health: Investigating health and health services, 2nd edition. Open University Press, 2002. • McDowell and Newell (1995). Measuring health. A guide to rating scales and questionnaires. Oxford University Press. • Wilkin, Hallam and Doggett (1992). Measures of need and outcome for primary health care. Oxford University Press.

41. Next time on HSR Everything you ever wanted to know about questionnaires (and possible a whole lot more)!