1 / 21

Data Extraction Workshop

Data Extraction Workshop. Lindsay Shepard 30 March 2011 PhD 7113 Systematic Reviews College of Social Work, University of Utah. Overview. The Purpose The Process The Contents Data Extraction Forms Example Data Extraction Forms Example Data Extraction Class Workshop.

kipp
Download Presentation

Data Extraction Workshop

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Extraction Workshop Lindsay Shepard 30 March 2011 PhD 7113 Systematic Reviews College of Social Work, University of Utah

  2. Overview • The Purpose • The Process • The Contents • Data Extraction Forms • Example Data Extraction Forms • Example Data Extraction • Class Workshop

  3. The Purpose of Data Extraction • “to describe the study in general, to extract the findings from each study in a consistent manner to enable later synthesis, and to extract information to enable quality appraisal so that the findings can be interpreted. Ideally this should be undertaken in such a way as to require minimal reference to the original papers at data synthesis stage.” (Social Care Institute for Excellence, 2006)

  4. The Rationale for Data Extraction Processes • “The findings of a systematic review depend critically on decisions relating to which studies are included, and on decisions relating to which data from these studies are presented and analysed. Methods used for these decisions must be transparent, and they should be chosen to minimize biases and human error” (Cochrane Handbook, Section 7.1, emphasis added)

  5. The Process of Data Extraction • Plan data collection in advance • Data – “any information about (or deriving from) a study, including details of methods, participants, setting, context, interventions, outcomes, results, publications and investigators” (Cochrane Handbook, Section 7.3.1) • Develop an extraction strategy • Replicable • Transparent • Use resources • Checklists and Review Group guidelines • Topical systematic reviews and primary studies • Data extraction forms • Pilot test method • Duplicate data extraction

  6. Data Collection Checklist, Cochrane Handbook, Table 7.3.a Items without parentheses should normally be collected in all reviews Items in square brackets may be relevant to some reviews and not others • *Full description required for standard items in the ‘Risk of bias’ tool (see Chapter 8, Section 8.5).

  7. Method Collection • “Different research methods can influence study outcomes by introducing different biases into results. Basic study design characteristics should be collected” (Cochrane Handbook, Section 7.3.2) • Randomized? • Cluster or cross-over design? • Duration of the study? • Non-randomized studies included? (see Cochrane Handbook, Section 13.4) • “Information should also be collected to facilitate assessments of the risk of bias in each included study using the tool described in Chapter 8 (Section 8.5)” (Cochrane Handbook, Section 7.3.2) • Sequence generation? • Allocation sequence concealment? • Blinding? • Incomplete outcome data? • Selective outcome reporting?, etc.

  8. Participant and Setting Collection • “Typically, aspects that should be collected are those that could (or are believed to) affect presence or magnitude of an intervention effect and those that could help users assess applicability” (Cochrane Handbook, Section 7.3.3) • Participant characteristics? • Total number? • Sub-group analyses? (Age, Sex, Ethnicity, Country, Co-morbidity, etc.) • Study setting? • Diagnostic criteria?

  9. Intervention Collection • “Again, details are required for aspects that could affect presence or magnitude of effect, or that could help users assess applicability. Where feasible, information should be sought (and presented in the review) that is sufficient for replication of the interventions under study” (Cochrane Handbook, Section 7.3.4) • Number of intervention groups? • Routes of delivery? (Oral or intravenous, Surgical technique, etc.) • Dosage? (Amount, Intensity, Frequency, etc.) • Timing? (Between diagnosis and treatment, etc.) • Length of treatment? • Contents of the intervention? • Who delivered or administered? • Format and timing of delivery? • Intervention integrity/fidelity/compliance?

  10. Outcome Measure Collection • “Review authors should decide in advance whether they will collect information about all outcomes measured in a study, or about only those outcomes of (pre-specified) interest in the review. Because we recommend in Section 7.3.6 that results should only be collected for pre-specified outcomes, we also suggest that only the outcomes listed in the protocol be described in detail” (Cochrane Handbook, Section 7.3.5) • Definition? (Diagnostic method, scale, threshold, or behavior) • Timing? • Unit of measurement? • Scale interpretation? (Upper/Lower limits; Favorable outcomes, etc.) • Economic outcomes? (See Cochrane Handbook, Section 15.4.2) • Patient-reported outcomes? (See Cochrane Handbook, Chapter 17) • Adverse outcomes? (See Cochrane Handbook, Chapter 14) • Method of collection? • Number of outcomes reported?

  11. Results Collection • “Other outcomes should not be extracted unless the protocol is modified to add them, and this modification should be reported in the review…Review authors should be alert to the possibility of important, unexpected findings, particularly serious adverse effects” (Cochrane Handbook, Section 7.3.6) • Number of participants allocated to each intervention? • Variation in outcome measure/time/summary statistic reporting? • Content for meta-analysis? (see Cochrane Handbook, Section 7.7) • Unit of analysis? • Type of outcome? (Dichotomous, Continuous, etc.) • Summary data for each intervention group? (Mean, Standard Deviation, Standard Error, Confidence Interval, Test Statistic, etc.) • Missing participants? (CONSORT Flow diagram) • Estimate of effect? (Effect size, confidence interval, and p-value) • Sub-group analyses? • Number of outcomes reported?

  12. Additional Information for Collection • “Other information will be required from each report of a study, including the citation, contact details for the authors of the study and any other details of sources of additional information about it” (Cochrane Handbook, Section 7.3.7) • Eligibility for inclusion? • Page numbers of content/extraction? • Funding source or potential conflicts of interest? • Key conclusions? (Direction of effect) • References to other relevant studies? • Study quality notes? (Ethical approval, sample size calculation, etc.) • Correspondence required? • Comments/Notes (Date of extraction, dates of revisions, name of extractor, etc.)

  13. Rationale for Data Extraction Forms • “The data collection form serves several important functions (Meade 1997)… • the form is linked directly to the review question and criteria for assessing eligibility of studies, and provides a clear summary of these that can be applied to identified study reports… • the data collection form is the historical record of the multitude of decisions (and changes to decisions) that occur throughout the review process… • the form is the source of data for inclusion in an analysis” (Cochrane Handbook, Section 7.5.1) • “Given the important functions of data collection forms, ample time and thought should be invested in their design” (Cochrane Handbook, Section 7.5.1, emphasis added)

  14. Data Extraction Form Types Electronic Advantages convenience or preference; data extraction can be undertaken almost anywhere; easier to create and implement (no need for computer programming or specialist software); provides a permanent record of all manipulations and modifications (providing these manipulations and modifications are not erased); and simple comparison of forms completed by different review authors convenience or preference; combines data extraction and data entry into one step; forms may be programmed (e.g. using Microsoft Access) to ‘lead’ the author through the data collection process, for example, by posing questions that depend on answers to previous questions; data from reviews involving large numbers of studies are more easily stored, sorted and retrieved; allows simple conversions at the time of data extraction; rapid comparison of forms completed by different review authors; and environmental considerations Paper Advantages Cochrane Handbook, Section 7.5.2

  15. Data Extraction Form Examples • Sample Data Extraction Forms: • Berkeley Systematic Reviews Group: http://www.medepi.net/meta/forms.html • Cochrane Cystic Fibrosis and Genetic Disorders Group: http://cfgd.cochrane.org/Files/Study%20selection%20and%20%20extraction%20form.doc • Petticrew and Roberts Systematic Reviews in the Social Sciences: A Practical Guide: http://onlinelibrary.wiley.com/doi/10.1002/9780470754887.app4/pdf • Cochrane Schizophrenia Group: http://www.cochrane-net.org/openlearning/Other/Example_from_a_review_in_schizophrenia.pdf

  16. Data Extraction Form Development • “If review authors plan to develop their own electronic forms using spreadsheet or database programs, we recommend that • (i) a paper form is designed first, and piloted using more than one author and several study reports; • (ii) the data entry is structured in a logical manner with coding of responses as consistent and straightforward as possible; • (iii) compatibility of output with RevMan is checked; and • (iv) mechanisms are considered for recording, assessing and correcting data entry errors” (Cochrane Handbook, Section 7.5.2)

  17. Example Data Extraction • Systematic Review Research Question: Amongst women in polygamous marriages, as compared to women in monogamous marriages in the same population, what is the prevalence of mental health issues? • See Paper Data Extraction Exercise Example • See Electronic Data Extraction Form Example

  18. Workshop Task • For your Systematic Review Research Question: • Begin to develop a data extraction form • For one paper in your review, begin to pilot your data extraction • Consider how to revise your form or process to make the most of your time • How to balance detail with usefulness?(i.e., too little versus too much information)

  19. Writing the Protocol • “The methods section of both the protocol and the review should detail: • the data categories that are to be collected; • how verification of extracted data from each report will be verified (e.g. extraction by two review authors, independently); • whether data extraction is undertaken by content area experts, methodologists, or both; • piloting, training and existence of coding instructions for the data collection form; • how data are extracted from multiple reports of the same study; and • how disagreements are handled if more than one author extracts data from each report” (Cochrane Handbook, Section 7.6.6)

  20. References • Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org • Social Care Institute for Excellence (2006). The conduct of systematic research reviews for SCIE knowledge reviews. Available from http://www.scie.org.uk/publications/researchresources/rr01.pdf

More Related