Draft chapter 2 on education output

1. Draft chapter 2 on education output Background document:  « chapter 2: education services » 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

2. General links between output and outcome • Hill services and result of the production process Peter Hill (1975) says the production of services is inseparable from their use or consumption: two economic units are simultaneously involved in service production whereas production of a good takes place within a single producer unit. • Output of services should be viewed asthe attributable contribution of the producer to the outcome. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

3. Definitions: education output • Eurostat handbook on prices and volume in NA “the quantity of teaching received by the students, adjusted to allow for the qualities of the services provided, for each type of education” “In the area of education, the output can be defined as the quantity of teaching (that is, the transfer of knowledge, successfully or not) for a pupil, whereas the related outcomes are the skill and knowledge that a pupil achieves.” • ISCED definition “organised and sustained communication designed to bring about learning”, where “Communication” involves the transfer of information (messages, ideas, knowledge, strategies, etc.) between two or more persons; […] “Learning” is taken as any change in behaviour, information, knowledge, understanding, attitudes, skills, or capabilities which can be retained and cannot be ascribed to physical growth or to the development of inherited behaviour patterns 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

4. Definitions: education output (2) • This handbook proposal • “expectedtransfer of knowledge and skills provided by an education unit, for each type of level of education / curriculum” • Transfer of knowledge and skills can be seen as the true quantity indicator, and change in academic scores due to school could provide a right measurement. • Expected, we could say « average », means that each pupil receiving the same teaching is supposed to consume the same output, although some pupils will assimilate more than others (what the Eurostat handbook called « successfully or not »). This difference in outcome is supposed due to the pupil/consumer. • The ISCED-97 levels of education and secondarily fields of education define the natural strata. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

5. ISCED-97 levels of education 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

6. But no R&D or adm., like in COFOG or UOE Formal education (our focus) ISIC rev. 4 and ISCED 97 informal Support activities 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

7. Basic formula • Ivolume = Iquantity (number of enrolled pupils) x Iquality ed (average transfer of knowledge) x Iquality non ed (total expenditures / core educational) • Without information on the transfer of knowledge, we find the number of pupils, 2nd best quantity indicator for Eurostat. • If we assume that the transfer is continuous and proportional to time of teaching, we get the “pupil x hours”, 1st best quantity indicator for Eurostat. • Non educational quality is theoretically better estimated by an additive formula if detail by secondary product is known. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

8. Consistency between quantityand quality indicators • If pupil-hours are the quantity indicator and change in scores the quality indicator, it leads to a double accounting: • Imagine a pupil with an attendance of 90% instead of 100%. • As he has received 10% less teaching, it is probable that he will perform only 90% of the change in score of his “normal” schoolmates. • If we cumulate the quantity effect (pupil hours) and the quality indicator (change in scores), we obtain 81% instead of 90%, the expected transfer of knowledge for this pupil. • transfer of knowledge= number of pupils getting 1 year older * average transfer of 1 year= number of pupils moving up 1 year * average transfer of 1 grade • If the quality indicator relies on scores at a given age, the quantity indicator should be the simple number of enrolled pupils. If it relies on scores at a given grade, the quantity indicator should be the number of pupils moving up. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

9. Overview of inputs, output, outcomes in volume 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

10. Education output = average transfer of knowledge and skills, some remarks • No accounting of the distribution of individual levels of knowledge and skills, although “reduction of inequalities” is usually one of the targets of the education policy: consistent with the idea of “individual services” and common rules in NA for FC, income, etc. • No accounting of healthy, wealthy, well-rounded people / citizens, which are sometimes difficult to measure and which are more “indirect” outcomes. • We measure usually education status (outcome) at the end of a curriculum, rarely at the beginning, we need therefore a model (part of the pupil / consumer, time-lag) in order to estimate the part of the output in the outcome. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

11. The model: education output and outcome 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

12. Size of the SD: does it matter for NA? • The SD of the distribution of education levels can be standardized to 10% or 20% of the average • PISA, PIRLS, TIMSS standardized to 20%, US NAEP to 10%. • The human capital theory (relying on future real earnings) suggests8-10% (2 studies) or 20% (1 study). • But it does not matter in a NA view • The convention for the relative size of 1 SD has an impact both on cross-section (better school vs average school) and longitudinal distances (between ages 6 and 15 for instance, which will provide a measurement for average schools). • The quality adjustment in equivalent number of years of teaching is a ratio between cross-section and longitudinal distances. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

13. Relative scores across ages The larger the SD at age 15, the larger the output between ages 6 and 15, and constant proportions between numerator and denominator of QA. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

14. => For PPP, let us scale the average outcomeat age 15 = 500 and 1 SD = 20% (PISA) • PISA measures a national average outcome at age 15. To reflect the output of the national school systems, it should be neutralized of the family contribution (ESCS correction) and subtracted of inherited skills. The formula should be: ESCS corrected PISA – X. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

15. Estimation of what is to be subtracted to ESCS corrected PISA scores: at least 180 • Ideally, we should observe inherited skills at age 15, or more exactly inherited skills + average family contribution (for ESCS only neutralizes the differences in family contribution) at age 15. Requires a sophisticated model. • But we can assume that education status at age 6 reflects only inherited skills + average family contribution. Can be extrapolated with the help of 2 national longitudinal tests: the US NAEP and the UK 4 fey stages, which suggest about 180 at age 6 in average (with a convention of 1 SD = 20%). • Inherited skills + average family contribution can increase between ages 6 and 15, so that we could subtract more. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

16. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

17. Time-lag management Lower secondary education Total 2000: 2 PISA 2000 + 2 PISA 2003 Total 2001: PISA 2000 + 8/3 PISA 2003 + 1/3 PISA 2006 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

18. Time-lag management and revisions 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

19. Other outcome-based quality indicators • Although we think that academic scores are the first best direct education outcomes, some curricula could be better estimated (or are more traditionally estimated) by obtainment of a degree or professional insertion (this latter an indirect outcome). • Professional insertion: employment rate (in the right qualification if possible) and real earnings can be used (multiplied) and are often suggested by human capital theory (discounted sum of future real earnings as return on “GFCF” in education), but they are to be considered in an incremental point of view: what is obtained after the curriculum minus what would have been obtained without, if we want to measure the “output”. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

20. Which output/outcome for tertiary education? • Degrees are certainly the most natural outcome indicators for tertiary education, but they are of different “values”. • Harmonization of degrees in Europe with the Bologna process (bachelor, master, doctorate), but what are the relative values of these 3 degrees? • Distinguish by (ISCED-97) field of education; • Double degree for one curriculum: double outcome? • Curriculum shared between different countries/units; • The output should not be null for students leaving without degree. • Time-lag adjustment. • Number of credits reflect perhaps more directly the “quantity of transfer of knowledge and skills”, i.e. our definition of the output, and avoid most problems quoted above. Still need to distinguish by field of education and perhaps by “prestige” of university. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

21. If not “outcome-based” then “outcome-oriented” • If the output is defined as the attributable contribution of the producer to the outcome, and if the outcome cannot be directly measured, then some “inputs-based” or “process-based” quality indicators could be used, but “outcome-oriented”, it means converted in an expectation of incremental contribution to the outcome. • Example of the Italian model of the class size (see item 3.3). • If the more classical way of treating different qualities as separate goods was chosen, it would lead to a “quality presentation” but not to a “quality adjustment”: the price index would be nothing else than a cost weighted combination of cost indices, i.e. the same price index as in the input method. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

22. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS

23. 2nd OECD workshop on “measuring Education and Health volume output” Alain GALLAIS, OECD/STD/NAFS