150 likes | 164 Views
Explore a collaborative approach to managing data for longitudinal studies to address existing deficiencies and improve efficiency. Learn about established steps, automated tests, and the use of STATA programs for better data management.
E N D
Collaborative Data Management for Longitudinal Studies Stephen Brehm [coauthors: L. Philip Schumm & Ronald A. Thisted] University of Chicago (Supported by National Institute on Aging Grant P01 AG18911-01A1)
Agenda 1. Background on Study 2. Problem – Data Management Deficiencies 3. Solution – Collaborative Data Management 4. STATA Programs – maketest & makedata
Background on Study • NIH-funded Longitudinal Study • Loneliness & Health • Thousands of Measures • Loneliness • Depression • 230 subjects • Repeated Yearly
Problem – Data Management Deficiencies • Code Not Modular …Difficult to manage the data cleaning code …Limited code reuse from year to year …Difficult to collaborate among interns • No Established Set of Data Cleaning Steps …Difficult for research assistants (turn-over) …Inconsistent data cleaning techniques …Data cleaning code difficult to read
Problem – Data Management Deficiencies Research Assistant Research Assistant Research Assistant Core File Set Research Assistant Research Assistant
Solution – Collaborative Data Management • Process • Established Steps • File System Layout • Automated Tests • Collaboration • Concepts • Module • Batch • “Data Certification” • STATA Programs • maketest • makedata
Solution – Collaborative Data Management • Process • Established Steps • File System Layout • Automated Tests • Collaboration • Concepts • Module Ex:loneliness • Batch • “Data Certification” • STATA Programs • maketest • makedata
Solution – Collaborative Data Management • Process • Established Steps • File System Layout • Automated Tests • Collaboration • Concepts • Module Ex:loneliness • Batch Ex:yr1, yr2, yr3 • “Data Certification” • STATA Programs • maketest • makedata
Solution – Collaborative Data Management Set of Files for Each Module acquire-[module].do & fix-[module].do test-[module].do derive-[module].do label-[module].do Year-Specific 60% Code Reuse – Files Shared Between Years Acquire & Fix Test Derive Label
STATA Program – maketest • Purpose: • Auto-generation of Data Certifying Tests • Functionality: • Tests Variable Type • Checks Consistency of Value Labels • Verifies Existence of Variable
STATA Program – maketest • Syntax: • maketest [varlist] using, [REQuire(varlist) append replace] • Example: • maketest using filename.do, replace • Options: • using: specifies file to write • REQ: requires presence of variables in list • append: add to existing test .do file • replace: overwrite existing .do file
STATA Program – makedata “Bringing it all together”
STATA Program – makedata • Syntax: • makedata [namelist], Pattern(string) [replace clear Noisily Batch(namelist) TESTonly] • Example: • makedata ats, p("acquire-*.do") b(yr1) clear replace • Options: • p: pattern – file naming convention • replace: overwrite existing data file • clear: clear current data in memory • Noisily: full output (default = summary) • b: batch – year, wave, center • TESTonly: only run tests step
Other Applications • Beyond Longitudinal Data • Teaching Data Cleaning with STATA • Contact Information • Stephen Brehm: sbrehm@uchicago.edu • L. Philip Schumm: pschumm@uchicago.edu • Ronald A. Thisted: thisted@health.bsd.uchicago.edu • Supported by National Institute on Aging Grant P01 AG18911-01A1