Topics

1. Topics • Review • Action Research • Research designs • Data collection and analysis • Quality of research • Research report • Final project

2. Topics • Schoolwide action research • Case study videos • Guest speaker • Quiz • Articles relating to common core

3. Role and Purpose of Action Research • To identify and solve specific classroom or school problems • To improve educational practices • To help to make decisions at a local site • To empower teachers and other educational practitioners • To promote professional growth

4. What is Action Research? • Systematic inquiry conducted by educators into their own practices with the purpose to gather information about how their school operates, how they teach, and how their students learn • Research done by teachers for themselves • Systematic inquiry into one’s own practices • Research then has increased utility, effectiveness

5. Review of Terms • Research: a systematic approach to finding answers to questions. • Research Design/Plan: a plan for gathering data for answering specific research questions. • Action Research: a systematic approach to finding answers to questions done by practitioners for themselves to improve educational practices.

6. Characteristics of Action Research • Practice-oriented • Conducted by practitioners done for themselves • Situation/context specific • Focused on solving problems at a local setting • Reflective • Researcher as participant • Systematic • Cyclical/on-going

7. Process of Action Research • Identify and define research problem and questions • Design research study to collect data bearing on questions • Conduct the research • Analyze the data • Interpret the data in light of the research questions

8. Action Research Model… Stringer’s Interacting Spiral

9. Action Research Model… Lewin’s Action Research Spiral Identifying a General or Initial Idea Reconnaissance or Fact Finding Take First Action Step Planning Evaluate Amended Plan Take Second Action Step…

10. Action Research Model… Calhoun’s Action Research Cycle 1 Select Area 5 Take Action 2 Collect Data 4 Analyze and Interpret Data 3 Organize Data

11. Action Research Model… Bachman’s Action Research Spiral Plan Reflect Act and Observe Revised Plan Reflect Act and Observe

12. Action Research Model… Riel’s Action Research Model

13. Action Research Model… Piggot-Irvine’s Action Research Model

14. Action Research Model… Hendrick’s Action Research Process

15. Action Research Model Mertler’s AR four stages:

16. With a Partner… Analyze the models of Action Research and identify a context (a building situation or issue) you think would be appropriate for each. For example, Stringer’s model could apply to a principal’s ongoing observation (Look)about safety issues when children are dropped off for school. Thinking about the issue and exploring possibilities, several new procedures are created, tried out with models, shared as a possibility with stakeholders, and reviewed with appropriate officials (Think), and finally the new procedure is put into place(Act). Create a personal graphic of an Action Research Model you would implement in your building. Share with group.

17. Major Research Methods Quantitative research methods Require numerical data Utilize deductive reasoning (‘top-down’ approach) Qualitative research methods Require narrative data Utilize inductive reasoning (‘bottom-up’ approach) Mixed methods Studies that combine both quantitative and qualitative data Many individuals consider action research studies to be most similar to mixed-methods research (than purely quantitative or qualitative research)

18. Quantiative Research Design Descriptive designs purpose is to describe and make interpretations of current status of individuals, objects, conditions, or events. Survey research Correlational designs purpose is to measure and describe statistical relationship between two or more variables Typically use correlation coefficient Group comparison designs attempt to investigate cause-and-effect relationships by comparing two or more groups that differ on some characteristic Experimental and non-experimental designs Single-subject designs

19. Quantitative research designs Group comparison designs Pre-experimental (no control group or randomization) One-shot case study One-group pretest-posttest design True experimental (control group and randomization) Posttest-only control-group design Pretest-posttest control-group design Quasi-experimental (control group but no randomization) Non-equivalent control group design Time-series designs Causal-comparative designs Ex post facto = ‘after the fact’ Presumed cause has already occurred (prior to study)

20. Qualitative Research Designs Case study particular individual, event, or program is studied Ethnography in-depth study of a group Phenomenological study studies of individual perceptions of a particular situation Grounded theory Qualitative studies that attempt to discover a theory

21. Quantitative Data Collection Techniques Quantitative data are numerical Variety of techniques: Surveys, questionnaires, rating scales—verbal or written administration of set of questions or statements to sample of people Closed-response Likert and Likert-type scales Checklists Follow guidelines and suggestions for developing instruments Tests and other formal instruments

22. Quality of Quantitative Data Validity—extent to which you actually measured what you intended to measure Must be appropriate and accurate for your purposes Seen as a unitary concept, combining content, concurrent, predictive, and construct validity (focus should be on evidence) Reliability—refers to consistency of collected data Determined by correlating results with themselves or with another quantitative measure Three methods: Test-retest reliability Equivalent forms reliability Internal consistency reliability ‘A valid test is always reliable, but a reliable test is not always valid.’

23. Rigor of Quantitative Study • Can you trust the conclusions of the study? • Internal Validity: The extent to which the outcomes of the study result from the variables manipulated, measured or selected rather than from other variables not systematically managed. • External Validity: the extent to which the findings of a particular study can be generalized to people or situations other than those observed in the study.

24. Best Practices in improving rigor of quantitative studies • Control Group: a group of subjects whose selection and treatment are exactly the same as those of the experimental group except that the control group does not receive the experimental treatment. • Random Assignment: a method for assigning subjects to control and experimental groups. Not to be confused with random selection (a method for selecting a sample of subjects from a population). • Pretests: When random assignment is impossible or undesirable, pretests can be used to examine prior existing differences between groups and to statistically adjust for these differences. • Instrumentation: select valid and reliable instruments to minimize measurement errors. • Sample size: Large sample size from representative sample is desired. • Effect size: report effect size. • Replication: perhaps the best way to ensure internal and external validity

25. Qualitative Data Collection Techniques • Qualitative data are narrative • Variety of techniques: • Observations—carefully watching and systematically recording what you see and hear • Structured/semi-structured/unstructured observations • Recorded using field notes, videotapes • Should include observer’s comments in the field notes • Interviews—directly asking people questions (as opposed to watching them); conversations between researcher and participants • Prepare an interview guide (may be specific or general) • Several types: • Structured/semi-structured/unstructured interviews • Individual, phone, Focus group

26. Qualitative Data Collection Techniques Journals—means of gathering data to provide insight into workings of a classroom Types of data journals: Student journals Teacher journals Class journals Existing documents and records— schools are filled with existing sources of data… Classroom artifacts—as are classrooms!

27. Quality of Qualitative Data Validity of research data—extent to which data collected accurately measure what they purport to measure Validity of qualitative data…concern lies with trustworthiness of data; Trustworthiness of data is enhanced through: Triangulation—use of multiple data sources, multiple data-collection methods, and multiple teacher-researchers (if possible) Member checking—sharing data and analyses with participants to check for accuracy Prolonged engagement & persistent observation—more time spent ‘in the field,’ more you get to know participants, culture, behaviors, etc.

28. Rigor of Qualitative Study • Can we trust the conclusions of the study? • Four trustworthiness criteria (Guba & Lincoln, 1989) • Credibility: related to the “true” picture of the phenomenon. • Transferability: related to whether the findings can be transferred to other situations. (think description of setting/sample) • Dependability: relates to the consistency between the data and the findings. (audit trail) • Confirmability: involves the strategies used to limit bias in the research, specifically the neutrality of the data not the researcher (peer debriefing)

29. Best Practices in Improving Rigor of Qualitative Studies Determination of rigor often depends on intended audience for sharing results Broader dissemination—should be more concerned with generalizability Narrower dessimination—may be no generalizable results Methods of providing rigor in action research Triangulation of data Member checking Peer debriefing Prolonged engagement and persistent observation Thick description of the setting and study Make available an audit trail (a recode of data analyzed in the study) Repetition of the cycle

30. Research StudiesPair-Share Review slides on research designs Which design have you used with your study? Rigor of designs Review slides on data collection strategies What data collections have you used? Quality of data Be the critical friend

31. Final project presentation

32. Analyze Quantitative Data • Descriptive statistics • Descriptive study • Describe student make-up at your school • Describe student performance at your school • Group comparison • Compare performance on tests by different student subgroups (gender, ethnicity, FRL, grade level, etc.) • Relationship / correlational • Describe relationship between Plan and ACT • Describe relationship between performance on math and reading

33. Review Analyze quantitative data Descriptive Group comparison (disaggregation and trend analysis Correlational Analyze qualitative data

34. Group Activity You are assisting the district supt. to analyze the ISAT data for one building. Use the data to create a report on Student demographics (who are we?) Interested in Gender, low income by grade level Please note strengths and challenges in student make-up Student performance (how did we do?) Interested in ISAT reading and math by grade level, gender, low income Please note strengths and challenges in student learning Is there any relationship between performance on reading and writing? If there is, describe the relationship. Develop an action plan based on your analyses

35. How to analyze Descriptive study Nominal/categorical data • Frequency table • Frequency chart (e.g. bar chart) Interval/ratio data • Frequency table • Frequency chart (e.g. histogram) • Measures of central tendency (mean, median, mode) • Measures of dispersion (range, standard deviation)

36. How to analyzeGroup comparison study - different groups Disaggregation analysis • Break down data by subgroups • Compute averages and standard deviations for each group as well as for the total group To gauge the magnitude of the difference • Compute average performance difference between groups • Divide average performance difference by the standard deviation for the total group • Use Cohen’s (1988, 1992) criterion: |.2| = small, |.5| = medium, |.8| = large Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (second ed.). Lawrence Erlbaum Associates. Cohen, J (1992). "A power primer". Psychological Bulletin112: 155–159. doi:10.1037/0033-2909.112.1.155

37. How to analyzeGroup comparison study – same group over time Pre-Post analysis (Trend analysis) • Compute learning gain for each individual • Compute the average gain and the standard deviations of the gains for the group To gauge the magnitude of the gain • Divide the average gain by the standard deviation of the gain for the group • Use Cohen’s (1988, 1992) criterion: |.2| = small, |.5| = medium, |.8| = large

38. How to analyzeCorrelational study Correlational analysis • Derive a Scatterplot for two variables • Compute the correlation coefficient To gauge the magnitude of correlation • Use Cohen’s (1988, 1992) criterion: |.1| - |.3| small, |.3| - |.5| medium, over |.5| large

39. How to Analyze Qualitative Data Transcribing the interviews Reading the transcripts to tentatively identify categories of responses Testing the tentative categories by classifying responses in the first hour of the interviews Using final categories to code all responses Tallying coded response Report and display the coded responses Peer debrief Member check 40

40. Group Activity Analyze qualitative data

41. Research Ethics Resect the rights of and protect human subjects involved in a research study Permissions from students, parents, teachers, others May require formal permission Informed consent form (See sample) Ways to maintain confidentiality and anonymity of study participants Report aggregate data Use fictitious names for individual students, schools, etc.

42. Standard Format of Research Articles • Abstract • Introduction: Context, Research Problem, Review of Literature • Methods • Results • Discussion • References

43. Introduction • Background - the reasons the author(s) conducted the study; theoretical framework • Statement of Purpose - the goal of the research (the destination); the problem statement • Hypotheses - “educated guesses” about relationships or differences

44. Methodology • Participants (sample) - who the subjects are, how obtained/selected • Materials (equipment, apparatus, measuring instruments) - what was used, quality of measuring instruments • Procedures - how study was conducted; what subjects did or what was done to them

45. Results • Summary of the analyses of numerical or narrative data used: • In text • In tables • In figures

46. Discussion/Conclusions • Non-technical interpretation of results • Linking results to original purposes and research questions • Why the results turned out the way they did • Identifying the study’s limitations • Formulating future plans of action

47. Descriptive Statistics • Methods used to obtain indices that characterize or summarize data collected • Focus is on the sample(s) at hand • Simple description of: • Individuals • Collection of individuals • Used as basis for inferential statistics

48. Basic Elements: Hypotheses • Hypothesis: a tentative statement (“educated guess”) about the expected relationship between two or more variables. • State expected relationship or difference • Be worthy of being tested • Be testable • Be brief and clear

49. Basic Elements: Variables • Variable: what is measured or varied. An attribute or characteristic of a person (or object) that can change from person to person. • Independent • Dependent • Control • Intervening

50. Classification of Variables • Independent Variable:a variable that is manipulated, measured or selected by the researcher in order to observe its relation to the subject's "response". An antecedent condition. • Dependent Variable:the variable that is observed and measured in response to an independent variable.