Download
1 / 18
180 likes | 191 Views
This article examines the importance of validity as a quality indicator in student assessment and highlights the need for establishing standards. It discusses the process of test development and provides an overview of the standards set in Jamaica for instrument construction, materials management, scoring and analysis, documentation and reporting, and fairness in testing.
E N D
Fitz-Albert R. Russell, PhD JP Establishing Standards for Students’ Assessment
INTRODUCTION • Psychometric assessments used for decisions about work, jobs, and careers. • In education, students assessment used for four types of decisions. • Administrative & policy formulation, instructional, certification, selection decisions. • Testing & measurement applied to other persons & for other reasons-not dealt with here
BACKGROUND • > 1,000 incidences of between Grs. 1-11. • Each is important to the child life chances • Some are high stakes & summative in nature • Examples of high stakes students’ assessment • Grade Six Achievement Test (GSAT) • CXC Secondary Education Certificate (CSEC)
VALIDITY AS QUALITY INDICATOR • Are decisions made about: possibilities, abilities, achievements, prospects, & predisposition based on assessments appropriate for purpose? • Three considerations: • Are testing conditions are fair to all test takers • Are scores reliable: consistent across time, space, thought, culture variations, instrument form and test takers’ ability • Is test valid: uses made of results are appropriate & consistent with intended use.
MESSICK’S VIEW • Validity: the pre-eminent consideration in students’ assessment • Incorporates concerns of fairness and reliability • Drives need for the establishment of standards. • Messick (1989a, 1989b) viewed validity as “evaluative judgment of a unitary concept that is based on evidence and consequences of appropriate test use and score interpretation” • Professionals must recognize issues of bias & drift in items, be able to obviate them
STANDARDS IN ASSESSMENT • Classical concepts of validity do not represent all issues of quality. • All elements in arrangements & administrative conditions impact validity judgment, & require quality considerations. • Quality is not necessarily JIT delivery of material • Rather, a good regime and protocol consistently applied (Meeter 1997) is required • Standards established & rigorously applied serve to improve the validation judgment.
TEST DEVELOPMENT PROCESS • Curriculum Analysis • Assessment planning • Item development, improvement, and validation • Compilation of assessment instruments • Preparation of first copies • Enroll test takers & prepare registers • Materials Management & Presentation • Response digitization • Mark, score, scale, equate, bias assessment • Score utilization & reporting • Process documentation.
NCME STANDARDS BY APA • The “standards” organized into three parts • Part 1: Test Construction, Evaluation, and Documentation • Part 2: Fairness in Testing • Part 3: Testing Applications.
JAMAICAN STANDARDS: SET 1 • Relates to instrument construction & validation • Explicit curriculum referencing the assessment • Learning outcomes with assessment specification • A set of exclusive sample items; • Preliminary testing of item in final form of a test ; • Items reliability (pbis) >=.75 • At least one parallel form of a test available • No item re-used within a three year period.
JAMAICAN STANDARDS: SET 2 • Relates to materials management and student processing • First copy of instrument lodged & used as authentic paper. • Security features built into the printing, processes and packaging; • Packages opened in assessment room, with certification by principal; • Documentation of the process, at each step;
JAMAICAN STANDARDS: SET 3 • Relates to scoring, scaling equating & analysis • Turnaround on all scores within 30 days • Appropriate scoring procedures & scaling procedures pre-designated for open-ended/self response items • Pre-designated procedures for dealing with missing responses • Comparative scores reported as equated scores • Item & test analysis results documentation; • Procedures carried out by professionals
JAMAICAN STANDARDS: SET 4 • Relates to appropriate documentation, reporting & use of data • Interpretation: Appropriate pre-designated procedure for producing standardized scores; • Reporting: documentation to inform interpretation & judgment of validity • Documentation: assessment documented & presented to the NCE • Utilization: Response available for individual test takers, facilitating “value added” analysis.
JAMAICAN STANDARDS: SET 5 • Relates to issues of fairness • Administration: conditions for test takers to write tests under fair and equal conditions • Presentation of assessments: Tests may be presented as paper based or tablet form. • Rate of work: empirically determined providing time allowing 90 % of test takers to complete • Right of appeal: An appeal panel established
CONCLUSION I • Standards for governance & management, accountability, quality assurance measures, & expected service delivery • Standards proposed a) assessment instruments construction & validation b) materials management & student processing c) scoring, scaling equating & analysis d) appropriate documentation, reporting & use of data e) issues of fairness. • Standards for high stakes assessments
CONCLUSION II • Standards represent important advance governance in Jamaica. • Process of consultation & agreement important as complex & technical issues are discussed, possibilities explored & expectations accepted • Standards codified & compiled into a document along with standards for other technical and related areas of service delivery. • Standards are about governance, management, and service delivery. • Accountability of the Permanent Secretary.
