Lesson 2 Main Test Theories: The Classical Test Theory (CTT)

1. Lesson 2Main Test Theories: The Classical Test Theory (CTT)

2. Test theories • They allow to set one functional relationship between: • Observable variables (from empirical scores obtained by subjects in tests or in their items); and • Unobservable variables (true scores or the skill level of the subjects in the construct that is measuring).

3. Test theories • Main test theories: • Classical Test Theory (CTT). • Item Response Theory (IRT). • Generalizability Theory (GT).

4. 1. Classical Test Theory (CTT) • Spearman (1904, 1907, 1910, 1913) • Functional relationship between empirical or observed scores (X), true scores (T) and scores due to error (E). • Linear model: X = T + E

5. 1. Classical Test Theory (CTT) • The actions of one subject responding to a test at a particular time are affected by many factors difficult to control. • That implies that the obtained score (empirical) doesn’t match with their true score. • It will be necessary to estimate the true score based on assumptions of the model.

6. 1. Classical Test Theory (CTT) • The error term includes all random errors that are affecting empirical scores. They can come from several sources: • The subject (emotional state, fatigue, stress, etc..). • The test (due to their items and type of format). • Characteristics of the applicators. • Environmental conditions. • Instructions. • Etc. • We should try to control them through the study of reliability (lesson 5).

7. 1.1. Classical Test Theory (CTT) Model assumptions A. The true score (T) is the mathematical expectation of the empirical score (X). • If we pass an infinite number of times the same test to a person (assuming that the applications are independent, so the score obtained in one application do not influence the others), the mean of all observed scores (X) would be the real score of the subject. T = E(X)

8. 1.1. Classical Test Theory (CTT) Model assumptions B. The correlation between true scores of 'n' participants in a test and measurement errors is equal to 0. • There is not relationship between measurement errors and true scores. rte = 0

9. 1.1. Classical Test Theory (CTT) Model assumptions C. The correlation between measurement errors (re1e2) that affect scores in two different tests (X1 y X2) is equal to 0. • There is no reason to assume that measurement errors committed in one test will influence, positively or negatively, the measurement errors in another test if tests are applied correctly. re1e2 = 0

10. 1.2. Classical Test Theory (CTT) Model deductions • 1. when • 2. • 3. • 4. E = X – T • 5. cov (T, E) = 0 • 6. cov (X, T) =var (T)= • 7. • 8.

11. 2. Item Response Theory (IRT) • Lord (1952, 1953). • The probability that one person emits a specific response to an item depends on the skill level of the person in the construct and on item characteristics (difficulty, discrimination, pseudochance). • The IRT provides a number of models that assume a functional relationship between the values ​​of the variable that items measure (skill level of the subjects in the measured construct) and the likelihood that the subjects hit each item, depending on their skill level. • This function is called: Item Characteristic Curve.

12. 3. Generalizability Theory (GT) • Cronbach, Glesser, Nanda & Rajaratnam (1972) • It represents a way to try to systematize and classify the error as a function of the possible sources that cause it. • It takes into account all possible sources of error (due to individual factors, situational characteristics of the evaluator, and instrumental variables) and tries to differentiate by applying the classical procedures of analysis of variance (ANOVA).