The Efficient Exam

1. The Efficient Exam Shlomo Yitzhaki Hebrew University

2. Talk’s Structure • Characterization of Grades in Exams • Monotonic correlation • Properties of Gini’s Mean Difference • Properties of the Efficient Exam

3. Characterization of grades • Grades are an Ordinal Variable • It is as if we are measuring height of people standing behind a screen • We ask who has the X centimeter and those that are taller than X respond positively. • Height is the number of positive answers. • It is impossible to plot a cumulative distribution of grades

4. Characterization of Grades • If cumulative distributions of two groups intersect, then there are two alternative legitimate exams that will result in contradicting ranking of average grades. • Hence, one can improve her country performance in international exams like PISA, by pointing out the alternative exam.

5. Monotonic Correlation • It is assumed that we are examining a uni-dimensional ability • Otherwise we have to examine whether the correlation is monotonic. • The method to do that is based on plotting Concentration curves (A variant of Lorenz curve for two variables). • The Method is already published (Economics Letters, 2012).

6. Properties of GMD • Gini’s Mean Difference can be decomposed in a way that makes the decomposition of the variance a special case. • This way one can find the implicit assumptions behind the variance. • Properties described in a 540 pages book • Entitled “The Gini Methodology” by Springer Statistics N. Y. 2013.

7. GMD vs. Variance • Variance = cov(X, X) • GMD = 4 cov(X, F(X)) • Note that F is uniform [0, 1]. • Gini covarince cov(X, F(Y)), cov(Y,F(X)) • They don’t have to have the same sign. • Known in economics as “Index number problem.

8. Properties of GMD • ANOVA Translates into ANOGI • Two correlation coefficients between two variables two Gini Covariance, two regression coefficients, mixed GMD-OLS regression, etc.. • If the two correlations between two variables are equal, then we get an identical decomposition of the variance of a sum of random variabes

9. Properties of the Efficient exam • Because of the limited number of questions, There is “Binning” • Main proposition: To maximize between-group variability, the distribution of grades in the “efficient exam” should be Uniform. • No proof is presented in this talk.

10. A sketch of the proof • The proof is based on the proposition that the distribution of the cumultive distribution is uniform [0, 1]. • Using Lorenz curve then the question is what is the optimal size of a “bin” • Two stages: Every “bin” should be positive. Mid-point is optimal

11. Transvariation • Two possible ways to rank groups: • According to average grade • According to transvariation: The probability of a randomly selected member of the high (low) average group to be better than the randomly selected member of the lower (higher) average group. • Under efficient exam both criteria are equivalent.

12. Applications • The arguments are relevant to any test based on ordinal variable. • I owe this point to Emil • This is the reason why I was invited

13. Thank you • For your Patience