Session 7 Introduction to Research and Evaluation

1. Session 7 Introduction to Research and Evaluation Topic 1: Research Questions and Hypothesis Testing And Topic 2: Introduction to Statistics

2. For Tonight

3. Today • Review the contents of the proposal • Topics tonight • Finish the research questions • Types of data review • Hypothesis Testing • Intro to Stats

4. The phases of a research project • Problem statement • Purpose • Hypothesis development / research question(s) • Population / Sample type • Results reporting (data) • Statistical testing • Conclusions Recommendations

5. Parts of the Research Report • Chapter 1 • Chapter 2 • Chapter 3 • Chapter 4 • Chapter 5 • References • Appendix

6. Components of Chapter 1 • Introduction • Background of the study • Problem statement • Significance of study • Overview of methodology • Delimitations of study • Definitions of key terms • Conclusion (optional)

7. Characteristics of Components in Chapter 1 • Introduction – 1 paragraph – 3 pages • Gets attention - gradually • Brief vs. reflective opening • Background – 2-5 pages • History of problem, etc. • Professional vs. practical use • Be careful of personal intrusions

8. Characteristics of Components in Chapter 1 • Problem Statement – ½ page • States problem as clearly as possible • Significance of study – 1 pgh. to 1 page • Answers: “Why did you bother to conduct the study?” • Be careful of promising too much

9. Ways to Convey Significance • Problem has intrinsic importance, affecting organizations or people • Previous studies have produced mixed results • Your study examines problem in different setting • Meaningful results can be used by practitioners • Unique population • Different methods used

10. Characteristics of Components • Delimitations – as needed • Not flaws • Establishes the boundaries – can study be generalized? • Consider: • sample • Setting • time period • methods

11. Stating the Problem • Developing a hypothesis: • Methods: estimation and hypothesis testing. • Estimation, the sample is used to estimate a parameter and a confidence interval about the estimate is constructed. • Parameter: numerical quantity measuring some aspect • Confidence Interval: range of values that estimates a parameter for a high proportion of the time • Hypothesis Testing: the most common use • Hypothesis: an intelligent guess or assumption that guides the design of the study • Null hypothesis: there is no difference or there is no effect • Alternative hypothesis: there is a difference or there is an effect • Hypotheses: more than hypothesis, which are related to the population

12. TYPES OF DATA

13. Variables • Two categories: • Independent • Variables in an experiment or study which are not easily to be manipulated without changing the participants. • Age, gender, year, classroom teacher, any personal background data, etc • Dependent • Variables which are changed in an experiment • Hours of sleep, amount of food, time given to complete an activity, curriculum, instructional method, etc.

15. Variables • A variable: any measured characteristic or attribute that differs for different subjects. • Two types: • Quantitative: sometimes called "categorical variables.“ • measured on one of three scales: • Ordinal: first second or third choice (most of the children preferred red popsicles, and grape was the second choice) • Interval: direct time periods between two events ( time it takes a child to respond to a question) • Ratio scale: compares the number of times one event happens in comparison to another event. (example: the number of time a black card is pulled in comparison to the number of times a red card is pulled) • Qualitative: • measured on a nominal scale.

16. Types of Data • Nominal Data  -- Data that describe the presence or absence of some characteristic or attribute; data that name a characteristic without any regard to the value of the characteristic; also referred to as categorical data. Male = 1 Female = 2, blue, green, etc • Ordinal Data -- Measurement based on the rank order of concepts or variables; differences among ranks need not be equal. • interval data -- Measurement based on numerical scores or values in which the distance between any two adjacent, or contiguous, data points is equal; scale without a meaningful or true zero • Ratio Data  -- Order and magnitude…. Measurement for which intervals between data points are equal; a true zero exists; if the score is zero, there is a complete absence of the variable.

17. Four levels: • nominal: assigning items to groups or categories • Examples: Classroom, color, size • Ordinal: ordered in the sense that higher numbers represent higher values • Examples 1= freshmen, 2= sophomore • Interval: one unit on the scale represents the same magnitude on the trait or characteristic being measured across the whole range of the scale. • Interval scales do not have a "true" zero point, • it is not possible to make statements about how many times higher one score is than another. • Ratio: represents the same magnitude on the trait or characteristic being measured across the whole range of the scale. • DO have true zero points

18. Nominal level of measurement • Assigns a number to represent a group (gender; geography) • Numbers represent qualitative differences (good-bad) • No order to numbers • Statistics -- mode, percentages, chi-square

19. Ordinal level of measurement • Things are rank-ordered -- >, < • Numbers are not assigned arbitrarily • Assume a continuum • Examples -- classification (fr, soph, jr, sr), levels of education, Likert scales • Statistics--median (preferred), mode, percentage, percentile rank, chi-square, rank correlation.

20. Interval level of measurement • Equal units of measurement • Arbitrary zero point--does not indicate absence of the property • Example -- degrees, Likert-type scales (treatment), numerical grades • Statistics -- frequencies, percentages, mode, mean, SD, t test, F test, product moment correlation

21. Ratio level of measurement • Absolute zero • Interval scale • Examples -- distance, weight • Statistics -- all statistical determinations

22. Never married Lower middle Class Divorced Age Separated Middle class Widowed Weight Religious Affiliations Height Political Affiliations Distance freshmen Which are these?

23. Which are these? • Never married • Lower middle Class • Divorced • Age • Separated • Middle class • Widowed • Weight • Religious Affiliations • Height • Political Affiliations • Distance • freshmen • Minutes N I/R O N N I/R O I/R N I/R O N I/R N

30. Key Point • Statistical Significance must be distinguished from practical significance • Even a small difference in a large sample might be significant if the sample is large • No p-value of a .0001 means that 1 in 10000 times the difference observed will occur by chance (no real difference between groups)

31. Example Hypothesis • There will be no significant difference in the EOC scores for schools that use CAERT and those that don’t. • The EOC exam scores for schools using Caert and those that don’t will not be significantly different. • The EOC exam scores for schools using Caert and those that don’t will be significantly different.

36. Statistics for Teachers

37. Statistics“If you can assign a number to it, you can measure it”Dr. W. Edward Demming • Statistics • refers to calculated quantities regardless of whether or not they are from a sample • is defined as a numerical quantity • Often used incorrectly to refer to a range of techniques and procedures for analyzing data, interpreting data, displaying data, and making decisions based on data. Because that is the basic learning outcomes of a statistics course.

40. What is the mean medium and the mode in this example?

45. Descriptive statistics • Descriptive statistics • summarize a collection of data in a clear and understandable way. • Example: Scores of 500 children on all parts of a standardized test. • Methods: numerical and graphical. • Numerical: more precise- uses numbers as accurate measure • mean the arithmetic average which is calculated by adding a the scores or totals and then dividing by the number of scores. • standard deviation. These statistics convey information about the average degree of shyness and the degree to which people differ in shyness. • Graphical: better for identifying patterns • stem and leaf display : a graphical method of displaying data to show how several data are aligned on a graph • box plot. Graphical method to show what data are included. The box stretches from the 25th percentile to the the 75th percentile • historgrams. • Since the numerical and graphical approaches compliment each other, it is wise to use both.

46. Inferential statistics

47. For choosing a statistical test variables fall into 2 groups • Continuous variables are numeric values that can be ordered sequentially, and that do not naturally fall into discrete ranges. • Examples include:  weight, number of seconds it takes to perform a task, number of words on a user interface • Categorical variable values cannot be sequentially ordered or differentiated from each other using a mathematical method. • Examples include:  gender, ethnicity, software user interfaces

48. Tools for Measuring • Measurement is the assignment of numbers to objects or events in a systematic fashion. • Four levels: • nominal: assigning items to groups or categories • Examples: Classroom, color, size • Ordinal: ordered in the sense that higher numbers represent higher values • Examples 1= freshmen, 2= sophomore • Interval: one unit on the scale represents the same magnitude on the trait or characteristic being measured across the whole range of the scale. • Interval scales do not have a "true" zero point, • it is not possible to make statements about how many times higher one score is than another. • Ratio: represents the same magnitude on the trait or characteristic being measured across the whole range of the scale. • DO have true zero points

49. Data Analysis • Explaining and interpreting the data: • Data are plural • You are looking at more than one number or group of numbers; subject-verb agreement is important when writing. • Central Tendency: measures of the location of the middle or the center of the whole data base for a variable or group of variables • Frequency: the number of times a number appears • Mean: the arithmetic average • Mode: the number that appears most often • Median: the number in the middle when numbers are arranged by value • Skew: A distribution is skewed if one of its tails is longer than the other. Data may be skewed positively or negatively. • Standard deviation: the amount of variance between each sigma

50. Inferential statistics • Inferential statistics • Infers or implies something about population from a sample. • Population: A total group • Sample: A few from the whole group • Representative sample: a sample that is equally propionate to the population • Random Sample: a sample that is chosen strictly by chance is not “hand-picked” • Probability: the percentage of change that an event will occur