110 likes | 274 Views
Data Collection Strategies An Overview. How Do We Assess ?. Data Collection Strategies. BEFORE collecting data, your first question should be. Has someone already collected the information from my sample of interest?. Personal interview Focus group Telephone poll Telephone interview
E N D
Data Collection Strategies BEFORE collecting data, your first question should be . . . . Has someone already collected the information from my sample of interest?
Personal interview Focus group Telephone poll Telephone interview Classroom assessment Paper/pencil survey Web survey Database/Warehouse Existing Reports Naturalistic observation Data Collection Strategies Indirect Contact Direct Contact
Assessment Instruments • Standardized • “Build Your Own” When selecting assessment tools you have a choice …
Proven item pool Reliability / Validity Clarity of directions Scoring Benchmark comparisons Data analysis Reporting formats Cost Processing Delay Item pool Inappropriate norms Existing Standardized Instruments Pros Cons
“Build Your Own” Instruments Pros Cons • Relevant item pool • Stakeholder involvement • Theory/Model based • Lower Cost • Processing control • Data analysis control • Report format • Long development time • Local technical expertise • Unknown reliability/validity • Length • Poorly worded items
Evaluating Existing Instruments • Does the title convey the content to student? • Review content of EVERY item by team. • Are items relevant to ALL students?
Evaluating Existing Instruments • Does manual report reliability ? • Does manual report validity evidence? • How appropriate is normative/benchmark data?
Evaluating Existing Instruments • Are results timely? • Is the cost worth the benefit?
Sources for Assessment Instruments Instrument Sources