Defining and Measuring Concepts

1. Defining and Measuring Concepts Variables, Validity, Reliability, and Descriptive Statistics

2. Defining Concepts • A concept is defined by clearly describing it’s measurable properties and specifying the units of analysis to which the concept applies. • A concept may be an abstraction. • However, the operational definition of a concept should be clear and unambiguous. • An operational definition should lend itself to empirical measurement.

3. Example • Ideology – might be measured by considering people’s relative liberalism/conservatism. Is this the only dimension of ideology? • What is liberalism/conservatism? • Self-identification as liberal or conservative? • Economic liberal/conservative - Individual policy preferences relative to government spending, taxation, and size? • Social liberal/conservative - Attitudes about moral issues? • Market liberal/conservative - Attitudes about market approaches to economic allocations? • Change liberal/conservative – Someone who views change as potentially good.

4. In order to operationalize any of these different concepts of liberal/conservative, one needs to identify associated empirical properties. • Operational definition might start with an inventory of features associated with the property. • Example: • Economic liberals – Favor a greater role for the central government in creating a just society; favor more government spending, higher taxes, favor government intervention in the economy;…. • Economic conservatives – Believe that justice lies in allowing people to rise by their own merits; government should have less of a role; favor less government spending; lower taxes; favor less government intervention in the economy. • The operationalization of a concept should not be ambiguous. For example, the preceding definitions are ambibuous. • Using the definitions above, conservatives should be in favor of abortion (a free market allowing choice), less military spending, less government spending on crime, etc. • In contrast, liberals should be in favor of restricting abortion, more military spending, and more government spending on crime, etc.

5. Measurement Error • Two types of measurement error can occur. • Systematic – This type of measurement error induces systematic bias into our characterization of a research concept. • Example: Self-Identified Liberalism/Conservatism is often systematically biased. • Random – This type of measurement error does not induce systematic bias, but does make measurement less precise. • Example: Economic Liberalism/Conservatism might exhibit random measurement error when the variables used to characterize it are difficult for survey respondents to understand. As a result, their responses exhibit a pattern of random error.

6. R (respondent) favors more federal government involvement in education. Is R, therefore, a liberal? George W. Bush and many Republicans favored more government involvement in education (No Child Left Behind). R might favor more federal government involvement in education because of some perceived weakness in the current educational system, rather than because R is a liberal.

7. Reliability and Validity • Reliability – The reliability of a measure of a concept is the extent to which the measure gives the same reading every time it is taken. • Does the measure consistently “hit” the same place on the target? Reliability can be thought of as the “scatter” around the place it typically hits. • Validity – The validity of a measure of a concept is the extent to which the measure gives an accurate reading of the concept. • Does the measure hit the center of the target, on average? • A measure can be valid without being reliable. Similarly, a measure can be reliable without being valid. • Both types of measurement error can be bad. An unreliable measure leads to an inability to draw inferences consistently. We can’t reach strong conclusions for our explanations. • An invalid measure leads us to draw poor inferences. In other words, our explanations are wrong.

8. Evaluating Reliability • Test-Retest – Apply the measure to the same object multiple times. Evaluate whether the target is hit both times. In other words, look at the scatter around the target. • Alternative-Form – Split the empirical constructs into parts. Then see if the parts give the same answer. • Split-Half Method – A variant of test-retest where a scale is split into two parts. • Cronbach’s Alpha- This is often used to evaluate the reliability of the elements of a scale consisting of responses to a set of questions. See http://en.wikipedia.org/wiki/Cronbach's_alpha for a mathematical definition.

9. Evaluating Validity • Face Validity – Does the measure of the concept seem to measure what it is intended to measure. Face validity is an intuitive evaluation. Are there good reasons to think it does or does not? • Construct Validity – Does the measure of the concept have the relationship to other concepts which we would expect it to have? • I coded every presidential sentence from Public Papers of the Presidents relating to 9 policy dimensions as liberal or conservative. • I then compared this to another commonly used measure of presidential liberalism, Presidential ADA Scores.

10. Variables and Their Measurement • A variable is an empirical measure which varies across the units of analysis. For example, if the unit of analysis is the individual, then a variable would vary across individuals. Etc. for groups, states, nations. • A variable must have at least two possible values, and often many more. • Variables are used to provide operational definitions for concepts.

12. Levels of Measurement: • Nominal – the values of the variable do not have order or mathematical meaning. Example: Gender (male/female); Region (North, South, East, West); Race (white/black/Asian, etc.) • Ordinal – the values of the variable have intrinsic order or ranking, but the distance between units does not have a consistent meaning. Example: Approve of President’s Job Performance (Strongly Approve, Approve, Don’t Know, Disapprove, Strongly Disapprove); Support for Abortion Rights (Strongly Support, Weakly Support, No Opinion, Weakly Oppose, Strongly Oppose) • Interval – the values of the variable have precise numerical meaning. Example: Age, Income, Years/Months/Days of Education, etc.

13. Additive Indices • An index is an additive combination of ordinal variables, all measured at the same level and identically coded. • An example is a Likert scale, which is an additive index of 5 or 7 valued ordinal measures.

15. Describing Variables • Central tendency of a variable • Mode – a variable’s most common value • Median – the value of a variable which divides the cases right down the middle • Mean – the average value of a variable. The arithmetic average of the variable. • Dispersion of a variable • The spread of cases for a variable, often calculated relative to the central tendency. • Commonly measured using the statistical variance, standard deviation, or a coefficient of dispersion.

16. Frequency Distribution • A frequency distribution is a mapping of the number of cases in each bin of a nominal or ordinal variable onto each bin. • Commonly a frequency distribution is reported either as a table or graphically in what is called a histogram.