PISA for Development Technical issues to be addressed (for ToRs)

1. PISA for Development Technical issues to be addressed (for ToRs) • Contextual Questionnaires and data collection instruments • Review of PISA constructs and indices (and variables) to identify those that are likely to be operable and not operable in developing country contexts, beginning with participating country contexts • Scale expansion and equating with current PISA indices identified to be operable and relevant in these contexts • Review of relevant evidence and examples (including input from participating countries) to identify relevant constructs to be considered for inclusion , including but not limited to the following: • Educational history of student (cognitive, non-cognitive) • School resources (including possible scales) that have effects on learning outcomes • Health and well-being of student • Socio-economic status

2. PISA for Development Technical issues to be addressed (for ToRs) • Contextual Questionnaires and data collection instruments • Review of contextual data collection used in PASEC, SACMEQ, STEP, LAMP, and participating countries… to identify relevant and viable constructs (indices and variables) to be considered and possible paths for incorporation into the field work of PISA for Development. • Review of evidence, methods and examples of collecting contextual information from out-of-school children (and students) that may not be able to engage with and respond the paper-and-pencil questionnaires. • Review of availability and quality of data on education systems and data collection systems in participating countries

3. PISA for Development Technical issues to be addressed (for ToRs) • Test-targeting, descriptive power and comparability of cognitive items • A1. Scale expansion and equating with current PISA scales and proficiency levels at the lower ends of performance (i.e. 1a, 1b and below in reading) for all three domains (including data available from PISA countries with majority of student performance at lower levels) • A2. Review of scaling models used in PISA and how these could be operational with new participants (e.g. specific variables used for conditioning) starting with challenges found and the data sets available from PISA countries • Review and discussion of the PISA frameworks and constructs being assessed in reading, mathematics and science in light of component skills included in frameworks of PASEC, SACMEQ, Pre-PIRLS and others • Review of available item pool of PISA to classify those that may be used for the revised instruments based on item difficulty, framework coverage and bias/non bias for developing country contexts (item parameters).

4. PISA for Development Technical issues to be addressed (for ToRs) • Methods and approaches for including out-of-school children in data collection and assessment • Review of specific challenges and technical issues to be addressed in the data collection methods with OOSCI including, but not limited to, unavailability of sampling frame of population, identification and engagement with children, and instruments for contextual information. This should identify specific field examples to be explored. • Review of methods for implementing manually adaptive testing methods using gateways/channels so that an initial basic instrument identifies the appropriate streams of questions/data collection. This should include other methods such as verbal surveys. • Review of evidence and approaches to the assessment of knowledge, skills and competencies in reading and numeracy/mathematics (and science) with out-of-school children. This should include frameworks, constructs and variables used in the assessment, scaling and reporting.

5. PISA for Development Technical issues to be addressed (for ToRs) • Methods and approaches for including out-of-school children in data collection and assessment • D. Review of evidence and relevant studies to establish an appropriate definition of the target population - ‘out of school’ children with regards to PISA data collection - starting with the target population of 15-year-olds. As much as possible, this should then be discussed in relation with the population of participating countries (preliminary). • E. Technical review of possible application of the 5DE framework of the UNICEF/UIS study in PISA for Development (for students and OOSC), relating to issues such as disparities between age/grades (e.g. many 15-years olds enrolled in primary school). As much as possible, this should refer to information and contexts in participating countries (including sub-groups of particular interest to countries, e.g. SEN, ethnic minorities).