September session 1: Research Questions & Selecting Measures

1. September session 1: Research Questions & Selecting Measures The first steps toward getting your EA research project underway

2. Your EA Research Project • As an EA, you will be evaluating the impact of your SLC project • Why? Because you need to know if it works • Typical SLC projects do outreach to a group of students with a particular goal in mind, say, getting middle school students excited about computing. If you don’t evaluate it, you won’t know if it is effective!

3. Evaluation • Evaluation is the systematic acquisition and assessment of information to provide useful feedback about some object 1. Research Question 2. What data can be collected 3. What data will be collected, when, how & by whom 4. Findings & Interpretations

4. Research Questions • Questions require answers- • They give you a way to evaluate evidence • Clear, open-ended questions require research and critical thinking

5. STEP 1: Identify Topic • What are you interested in? • What is your SLC doing? • What are you curious about? Example: an EA, is curious if doing a pair programming activity with students and parents will make parents think that computer science is a viable option for their kids

6. STEP 2: Brainstorm • Start asking questions about the SLC outreach • What are the goals of the SLC outreach • How will you know that the goals are being met? Example: EAstarts thinking about how he can answer his question; do parents enjoy the activity? Do they think computing is a viable career for their kids? Is there a larger effect on the kids attitudes when doing pair programming with their parents, versus a group of similar kids doing pair programming with peers?

7. Step 3: Hypothesize • What do you expect to happen? • What do you think the outcome will be? • Or what is the intended outcome? • Are there several expected outcomes? • Prioritize • What’s feasible in your timeframe? For our example: The ability to get groups of parents and a comparable group of kids will determine the questions and research design

8. Step 4: Design your Research • What measurement tool(s) will you use? • Student survey, knowledge test, interviews, observation • Teacher survey, interviews • Parents survey, interviews • What will be the design? Quasi-experimental: • Pre/post assessment (before activity/after activity) [repeated measures design] • Post assessment only Qualitative- interviews, focus groups, open ended items on a survey [mixed-methods]; observations (documented) Experimental: require control group

9. Common Research Designs • Descriptive • Goal is to observe & describe what happened • Methods used: • Surveys about opinions, attitudes, behaviors (called self-report) • Observation Example- Collect teacher observation of classroom environment (mood of students) following an SLC activity

10. Common Research Designs • Quasi-Experimental (because you have no ‘control’ group) • Goal is to determine cause, i.e. see if your SLC outreach (called an “intervention”) creates the outcomes you hypothesize • Methods used: • Surveys about opinions, attitudes, behaviors (called self-report) • Tests of knowledge • Can do pre/post design (before and after intervention) Example- Collect student attitudes (survey) about robots before an SLC activity and after- works best for long-term interventions, not one time workshop

11. Common Research Designs • Experimental (has ‘control’ group) • Goal is to determine cause, i.e. see if your SLC outreach (called an “intervention”) creates the outcomes you hypothesize • Control group for comparison, who has no ‘intervention’ • Methods used: • Surveys about opinions, attitudes, behaviors (called self-report) • Tests of knowledge • Can do pre/post design (before and after intervention) Example- Collect student attitudes (survey) about robots before an SLC activity and after- works best for long-term interventions, not one time workshop; collect same survey from a classroom of students in same school and same grade who didn’t have the SLC activity

12. Qualitative • Interviews • Protocol- the questions you will ask • Semi-structured vs Structured • Focus Groups • Small groups of 8-10 people • Observations • Identify what you will record • Documentation (form, pictures, recordings, etc.)

13. What will you do? • Report back in the EA session what your initial thoughts are • Your write up is your IRB 