Normalization of Board Marks for Admission to Engineering Courses

1. Normalization of Board Marks for Admission to Engineering Courses Findings and recommendations of the Joshi Committee

2. The Issue • Students leaving high school look for various career opportunities, including in Engineering. • Many courses, many examinations. • A lot of load on students. • High stakes. • Students turn to specialized coaching classes. • General education neglected. • Coaching distorts outcome of merit assessment examinations. • Institutes of Technology/ Engineering Colleges suffer from this distortion.

3. MHRD’s Response • Eliminate multiple entrance examinations by clubbing them together • JEE (Main) for all engineering institutions/colleges, • Another specialized examination for IITs. • Take into consideration Board performance for engineering entrance decisions • As an eligibility condition for admission to IITs, • As a component in composite score for merit lists of NITs and other institutions/colleges.

4. Committees Formed • Committees arising from deliberation in Council of IIT’s • DamodarAcharya Committee (8 March, 2010): Inadequacy of present admission system brought into focus. • T. Ramsami Committee (11 November, 2010): Feasibility of utilizing Board marks (in the form of percentiles) recognized. • Committee arising from deliberation in Council of NIT’s • S.K. Joshi Committee (13 August, 2012): Findings presented here.

5. Objectives of Joshi Committee • Specified Terms of Reference • Validating the normalization formula using actual results of various Board and refining it based on its validation. • Detailed objectives • To evaluate implementation methodologies and effectiveness of various possible schemes, • To validate the chosen scheme for its consistency and application for JEE (Main) 2013, • To identify and enlist relevant issues, which are not covered under the scope of current study, for proper implementation of the scheme.

6. Inputs Received by Joshi Committee • Inputs obtained by formed by the Chairman of CBSE (implementing agency) from • A ‘Core Committee’ formed by Chariman-CBSE, comprising experts from ISI, IIT’s and other prominent institutions, • Glenn Rowley of Australian Council for Educational Research (ACER), • Jim Tognolini and Jon Twing of Indian Centre for Assessment, Evaluation and Research (CAER). • Further analysis/validation by members of the Joshi Committee, including those participating from the ‘Core Committee’.

7. Decisions of Joshi Committee Objective 1: Evaluation of methodologies/schemes

8. Issue: How to Equate Board Marks • Framework provided by NIT Council : 60% weight on JEE-Main, 40% weight on Board scores. • Discarded option : Use of linear transformation • Adjustment only for mean and standard deviation , • 2012 Boards data showed board-to-board difference in score distribution – even after such transformation. • Accepted (with modification) option : Use of Board percentiles • Percentiles of different boards treated at par, • Recommended by Ramasami Committee. • Modification : Use of transformed Board percentiles • Modification of all percentiles to bring them to JEE (Main) scale • No change in relative ranking, • Makes Board scores ready for combination with JEE (Main) score.

9. Issue: How to Address Difference of Difficulty Levels of Subjects • ‘Rasch model’ explicitly take into account difficulty level of a subject . • Specialized computational methods based on such models are still at developmental stage. • Such model can ‘compensate’ for differences of difficulty levels of subjects • within a board, but • not across boards. • Use of this model also amounts to moving away from percentiles, recommended by Ramasami Committee. • Joshi committee does not recommend the use of such a model.

10. Issue: How to Address Differences in Marking Pattern of Different Subjects • Possible solution: Equate Board subject marks by using percentiles BEFORE aggregating. • Possible difficulty • Much heavier computational burden, • Greater chances of unforeseen hitches, • Greater need for communication with several boards to resolve confusion, • Time frame for computation is short, • Data gathering/validation mechanism is not yet in place. • Joshi Committee Decision: Equate Board marks AFTER aggregation. • Saving grace: Usual aggregates are widely used (and accepted) for determining ranks within Boards.

11. Issue: Which Subjects to Aggregate • Number of subjects should be as large as possible (emphasis on school education as a whole). • Most boards have at least five subjects. • Mathematics and Physics are REQUIRED for JEE-Main. • Joshi Committee decision is to use five subjects: • Physics, • Mathematics, • Any one of Chemistry, Biology, Biotechnology, Computer Science, • One language, • Any subject other than above four.

12. Issue: Basis Group for Normalization • Discarded Option: All students • All students do not have appropriate subject combinations. • Discarded Option: Passed students with appropriate subject combinations • Pass percentage varies from board to board, • Truncating at pass-mark would create board to board disparity. • Chosen Option: All students with appropriate subject combinations.

13. Issue: Nature of Calibration with JEE (Main) Marks • Normalize Board aggregate marks to make their distribution match • JEE (Main) aggregate marks of all appearing candidates (Option 1) • JEE (Main) aggregate marks of candidates from that board only (Option 2) • Choice between the two options were made on the basis of additional data analysis for validation.

14. Decisions of Joshi Committee Objective 2: Validation and fine tuning of chosen scheme

15. Assumptions Behind the Options • Assumption behind Option 1 • All boards have same merit distribution. • Assumption behind Option 2 • Different boards have different merit distributions. • This difference can be measured (and adjusted for) by the performance levels of students of different boards in JEE (Main)/AIEEE.

16. Risks of Adjustment through JEE (Main) Performance • Students of some boards perform poorly in JEE (Main) / AIEEE. • This disparity may be due to • Poor merit/ability, • Non-alignment of board examination pattern with JEE (Main) / AIEEE (rank correlations indicate this), • Lack of instruction in English and Hindi (only available languages for JEE-Main / AIEEE), • Less access to coaching, • Load of an extra subject in board (for some boards). • All these effects are confounded. • If performance disparity is attributed only to merit disparity, confounding factors are ignored. • Solution may be worse than the problem.

17. Findings from Analysis of 2012 Data • Option 2 requires different treatment of Board percentiles; Option 1 does not. • The differential treatment of Board percentiles under Option 2 can be quite extreme: • 80th percentile of Maharashtra Board equated with 50th percentile of CBSE; • Topper of Maharashtra Board has normalized Board score 331; Topper of Jharkhand Board has 274; • A CBSE student with AIEEE marks 130 and Board percentile 93.1 has a rank of about 18,000; a Maharashtra Board student with that profile has a rank of about 34,000. This amounts to penalizing the Maharashtra Board student for poor AIEEE performance of peers from that Board.

18. Key Findings • Vastly different treatment of percentiles of different Boards (Option 2) would not be fair in the presence of confounding factors. • Representation of various boards in different sections of the merit list • Option 2 changes the present (2012) pattern substantially, • Option 1 has less drastic impact. • Option 1 would produce more equitable performance across boards. • Option 1 would be the right choice.

19. Decisions of Joshi Committee Objective 3: Issues relevant for implementation

20. Issues and Actions • How to implement the selected method • Algorithm and Flowchart provided. • Operational issues • Timelines for Processing and Analysis provided. • Other issues (no action within purview of Joshi Committee) • Collection of data, • Formatting of data, • Validation / Authentication of data, • Adherence of time frames for data delivery.

21. Further Recommendation • A Core group may be formed by CBSE for implementation of the normalization scheme focussing on • Data Collection, • Nature of Data, • Validation of Data, • Timeline for Data Collection, • Data Processing.