Survey Design with a Focus on Model Validity

Overview of Presentation • Study Environment • Design Approach • Findings

McHenry Lake Kane DuPage Cook Kendall Will Grundy Study Environment • Geography • 8 Counties • 4,000 square miles • Population • 2,940,007 households • Diversity • Residents • Languages • Travel Modes • Political Environment • Changing Technology • Telephone Service • Travel Demand Modeling

Inventory Objectives • Document demographic and travel behavior characteristics of the region • Support update of the regional travel demand model • Serve as inputs for the development of a new regional travel demand model  Model Validity

Design Phase Objectives 1. Confirm inventory contents 2. Validate budget assumptions • Establish “typical” conditions • Identify unique conditions, remedies, and associated costs • Evaluate most effective survey administration options 3. Vet recommendations

Design Approach Community Outreach Passive Recruitment Traditional Pilot White Papers Peer Review Final Study Design

Passive Recruitment • Objectives • Low-Cost Recruitment Mode • Contact Non-Telephone HH • Action • Address-based Sample • Mailed 1000 Letters with Brochure and Mail/Web Recruitment Survey • 50% received $2 incentive

Passive Recruitment • Results • 53 HH returned surveys (6%) • 37% were non-telephone households • 83% received an incentive • No statistical differences from phone recruits • 38 non-participating HH debriefed • Only 28% recalled packet • 39% said would have participated if we called them

Passive Recruitment • Full Study • Advance Mailings Included in Full Study • Mail to Active and Passive Sample • Capture of Cell-Only HH • Lower-cost Recruitment • Mailed 10,000 letters to Date • 225 passive recruits (~10% of all recruits) • 4% non-landline HH • 1-person/1-worker HH • Higher reported use of Auto

Community Outreach • Objective • Identify and ameliorate participation issues • Action • Held 4 community group meetings

Community Outreach • Results • Identified Design Improvements • Identified Activities to Increase Participation • Full Study • Design – Reflected Community Group Input • Lower Participation among Target Groups • Phase 1: Advance Mailings, Incentives, Tailored Interviewing, Multiplicity Sampling • Phase 2: Community Groups • Phase 3: In-Person Interviewing

Traditional Pilot • Objectives • Establish “typical” conditions • Evaluate respondent reaction • Action • Three Pilot Test Areas • Three Diary Types • Two Survey Lengths • Incentives Test

Traditional Pilot

White Papers • Objectives • Recommend Final Study Design • Recommend Inventory Elements • Action • Identified Primary Data Elements for Pilot • Analyzed Pilot Data and Secondary Data • Presented & Discussed Recommendations

White Papers • Results • Inventory Contents • Sampling • Efficient Data Collection • Maximizing Participation www.nustats.com/chicago

White Papers • Full Study • Identified data elements • Support Model Update • Develop New Model • Sampling based on density and transit access • Phone/Mail/Web Options

Peer Review Panel • Objectives • Vet Design Recommendations • National and Local “Experts” • Action • Presented Pilot Results and White Papers • Obtained Local Stakeholder Input • Full Day Discussion of Each Design Element

Final Project Design • 11,600 HH • Mix of 1-day and 2-day HH • Year-long Data Collection Effort • 300 HH 7-day GPS Component • 1,000 SP Follow-up Surveys

Findings • In-depth Design Phase Was Worth It • If Budget Constrained, Secondary Data Helpful • Peer Review Panel Input Was Invaluable • Mix of Local and National • Focus on Model Validity • Driving Question for Evaluation • Identify Data Elements for Current and Future Model Updates

THANK YOU! Questions or Comments?

Survey Design with a Focus on Model Validity