110 likes | 340 Views
Data Collection Strategies An Overview. How Do We Assess ?. Data Collection Strategies. BEFORE collecting data, your first question should be . . . . Has someone already collected the information from my sample of interest?. Personal interview Focus group Telephone poll Telephone interview
E N D
Data Collection Strategies BEFORE collecting data, your first question should be . . . . Has someone already collected the information from my sample of interest?
Personal interview Focus group Telephone poll Telephone interview Classroom assessment Paper/pencil survey Web survey Database/Warehouse Existing Reports Naturalistic observation Data Collection Strategies Indirect Contact Direct Contact
Assessment Instruments • Standardized • “Build Your Own” When selecting assessment tools you have a choice …
Proven item pool Reliability / Validity Clarity of directions Scoring Benchmark comparisons Data analysis Reporting formats Cost Processing Delay Item pool Inappropriate norms Existing Standardized Instruments Pros Cons
“Build Your Own” Instruments Pros Cons • Relevant item pool • Stakeholder involvement • Theory/Model based • Lower Cost • Processing control • Data analysis control • Report format • Long development time • Local technical expertise • Unknown reliability/validity • Length • Poorly worded items
Evaluating Existing Instruments • Does the title convey the content to student? • Review content of EVERY item by team. • Are items relevant to ALL students?
Evaluating Existing Instruments • Does manual report reliability ? • Does manual report validity evidence? • How appropriate is normative/benchmark data?
Evaluating Existing Instruments • Are results timely? • Is the cost worth the benefit?
Sources for Assessment Instruments Instrument Sources