Alternative System for Admission into Engineering Programmes

1. Alternative System for Admission into Engineering Programmes Report of the Committee Presented to the Council of Indian Institutes of Technology A proposal for consideration Alternative system

2. Acknowledgement • Prof Acharya and his Committee members • Dr BK Gairola; Sri V Joshi; Sri H Bhartia; Sri M Tuli • Directors of all IITs • 2063 people who participated in opinion poll • Director and Experts from ISI and their students • Colleagues from MHRD, DST and NIC • Chairpersons of school boards and • Chairman and members of the IIT Council • Some Media representatives and all those who helped in the exercise Alternative system

3. Decision of the Council in its 41st meeting • Prof Acharya committee had been commissioned to study the present system of conducting examinations for admission into engineering progammes of the country and suggest alternatives • The committee presented its interim report in the 41st meeting • There was unanimity that the present system required a change as proposed • The committee was enlarged to address the issue of the need to recognize diversity of learning Alternative system

4. Major findings of Acharya Committee • Screening based on normalized Board scores at Standard X and/or Standard XII and Multiple Choice examination replacing the two stage JEE from 2006. • Entry barrier to be raised to 60% in the +2 examinations. • Factors, other than the Standard XII marks and AIR based on PCM testing, such as raw intelligence, logical reasoning, aptitude, comprehension and general knowledge need to be considered. • Need to factor in school performance more significantly into the selection process. Alternative system

5. Major findings of Acharya Committee • Decision based on one time test needs to be re-examined. Opportunities to improve must be built in. • Students must be relieved of the pressure of multiple JEEs. • Influence of coaching for JEE needs to be minimized. • Urban-rural and gender bias has to be eliminated or at least minimized. • The objective type of examination lends itself to undue influence of coaching. The conventional pen and paper examination with well designed long and problem solving oriented questions should be revived by keeping numbers in any JEE within reasonable limits. Alternative system

6. Work of the present committee • Study of Acharya committee work • Made Non formal survey among hundreds of school students, parents, employers, faculty and media person • Met and decided on General approaches • Held with Chairmen and nominees of all school boards • Collected data on scores of school board examinations • Wrote down a philosophy document and placed in public domain • Carried out survey of public opinion poll • Enrolled ISI for carrying out exploring statistical methods for normalization of school board scores • Met with Directors and Senior Faculties of IITs • Prepared a draft report for comments and advice of the Council Alternative system

7. Consultation with states and School boards • First stage • Gained access to data on scores on school board examinations through formal mechanisms • Enrolled participation in development of NTS • Second Stage • Assistance in harmonization and electronic access to data • Building trust and development of a process with safeguards and integrity Alternative system

8. Public participation through opinion poll • On-line opinion survey among the people of India • For multi parametric grading system as against single test models of JEE • Screening out as against selection strategies • With Responder profile, opinion polls, suggestions for alternative national test systems; risk mitigation strategies • Survey time slot open for three weeks Alternative system

9. Responder profile to the poll • 2063 participated • 59% students; 8% teachers, 5.5% parents, 23.4% non-teaching professionals, less than 1% coaching • ~80% of respondents had taken entrance examinations in their lives • 1220 students had participated • 80% students were from engineering stream and 95% of them had taken entrance examination • ~160 teachers had participated • ~90% of them are engaged in tertiary education Alternative system

10. Analysis of the current system of admission into engineering • 960 of 2063 commented on the multiplicity of entrance examinations with different sociological implications • 715 agreed with the view expressed • 947 of 2063 commented on need for reform • 85% voted for reform in admission system Alternative system

11. Inputs for reforms fo admission systems • Reduction in the number of examinations to one • Testing knowledge intensity, alignment to the 12th class syllabus • Reduction of dependency on coaching and pressure on students • Emphasis on aptitude….. (More than 90%) • Transparency in processes • Removal of negative marking • Online processes • Multiple centers, better scheduling Alternative system

12. Responses to reform directions • Factor-in performance in school boards • 66% in favor and 34% not in favor • 45 of those disfavor fear that board examinations do not assessing capability and 30% fear non-uniformity • 620 responded to question on Indian equivalent of SAT type • 73% voted in favor of aptitude type test • 646 responded to types of tests • 70% prefer a mix of aptitude and advanced type tests • 629 responded questions on test features • Dominant support is for high-filter type and SAT type tests Alternative system

13. On summary • More than 85% supported the concept of a single entrance test for admission into engineering programmes and voted for reforms • 70 % voted in favor of one test with provisions for testing both aptitude and advanced knowledge • 66% of people favor factoring in school board scores • Of 34% People who disfavor fear primarily the problem of non-uniformity. This could be addressed. Alternative system

14. Some Important suggestions for the committee from opinion poll • National Test in place of multiple competitive examination is generally welcomed • Concerns expressed about • the process integrity and fairness of testing methodology • Normalization methodologies across school boards • Multiple chances for candidates for improving scores • A single screening examination with a mix of aptitude (like SAT type) and advanced (like JEE type) Alternative system

15. Study for normalization of scores of school board examinations • Committee gained access to some relevant past data with a view to examine • Stability of scores of the same school board over time • Potentials for normalization of scores across various boards • Enrolled ISI into carrying out statistical studies for normalization of board scores Alternative system

16. Work carried out Indian Statistical Institute For exploring normalization methodologies for school board scores Alternative system

17. Pilot testing • Selected four boards for pilot testing • Central Board, TN Board, WB Board, ISE • Evaluation years 3-4 years for each board • Variations in Density of population • CBSE (5-6lakhs), TN(5.6-7.3 lakhs), WB (3.0 -4.6 lakhs, ICSE 25000-56000 • Evaluation of stability of scores over time for the same board • Potentials for mapping the profiles of several boards onto one selected board through monotone transformations Alternative system

18. Models tested • Select a percentile score (P) for all boards and determine the scores (X1) for P across boards • Mapping Model 1: Y1= {Xn – X1}/{Xm-X1} Model 2: Y2 = Xn/X1 where Xn , X1 , Xm are scores obtained by any candidate, marks corresponding to percentile P, and maximum scores obtained by any candidate in each board. Y1 will range from 0 to 1.0; while Y2 will be ratios in the range from 1.0 to Xm/X1 Alternative system

19. Observed Relationships of Scores For the four boards over time Model 1: Y1= {Xn – X1}/{Xm-X1} Where aggregate score percentages are used Alternative system

23. Standardized (Normalized) score Normalized percentile rank Percentile rank of student – 75 X 100. = 100 – 75 Alternative system

24. Some general conclusions emanating from the ISI study • Percentile scores are relatively stable over the periods studied for each board when aggregate percentage scores are analyzed • It is possible to normalize percentile ranks across boards • Correlations of normalized percentile ranks against percentile ranks of various boards map on to common linear relationship Alternative system

25. Transformations onto one theoretical board ISI Experts are certain that this transformation will be the same for all boards for all years Alternative system

26. It appears that for normalization of school board scores A statistical method is feasible after all. ISI may be encouraged to develop the methodology further and extend it to all boards and reconstruct past scenario for present IIT and NIT students over the last four years for revalidation of the method Alternative system

27. A pilot test among a select group of students: A suggestion • A group of statistically significant number of some volunteers from current student population may be enrolled into a pilot test for evaluating the utility of different models and suggestions. This pilot test has to be designed with care and confidentiality based on informed consent of all involved. The merit or otherwise of the approach will be discussed within the committee before decisions are taken Alternative system

28. Summary of work done so far • Opinion poll reveals support for reforms and favor single examination • with provisions for both aptitude and advanced while making provisions for factoring-in scholastic performance in school board examinations • ISI study presents a methodology for scientific normalization of scores across school boards • Percentile scores are stable over each board and it is possible to carry out monotone transform board scores and accomplish normalization across boards • Weighting options for school and entrance tests’ performance are considered Alternative system

29. Two approaches considered • Approach 1 • weighing consistency of performance in school board examinations and employ them for testing ability to write solutions and • One objective screening test with two sections; one for testing the aptitude and the other advanced knowledge in domain areas. • Approach 2 • weighing consistency of performance in school board examinations and employ them for testing ability to write solutions and • one objective aptitude test based on multiple choices and computer based correction systems Alternative system

30. Considerations of six different options • Option 1: Deployment of Scores as criteria based on class XII performance only • Equal weighting of school board scores A1and A2 • Equal weighting of aptitude scores A4 and advanced scores A5 • Normalized score = {A1 + A 2+A4 +A5 }/4 • Option 2: Deployment of Scores as criteria based on class XII performance only • Equal Weighting of board score A3 which is {A1+ A2}/2 • Equal Weighting of Aptitude scores A4 and A5 • Normalized score ={A3 +A 4+A5}/3 Alternative system

31. Considerations of six different options • Option 3: Deployment of Scores as criteria based on consistency of performance at class X and Class XII levels as well as in National Level Aptitude and Advanced Tests • Equal weighting for aggregate as well as subject performance at class X and Class XII levels where ) 0.1X (normalized score at class X in aggregate + normalized score at class X in subjects of choice + normalized score at class XII + normalized score at class XII in subjects of choice) • One third weighting of aptitude score 0.3 A4 • One third weighting of advanced score A5 • Normalized score =0.1{ Normalized aggregate class X + normalized class X subject score + Normalized class XII aggregate + Normalized class XII subject score} + 0.3 A3and 0.3 A5 • Option 4: Deployment of School Board Performance as screening but not as determinant for National ranks • Specify a Cut-off normalized percentile rank score for school performance say as 80 or 85 percentile rank • 50% weighting of National Level Aptitude score A4 for candidates passing the cut off of percentile rank • 50% weighting of National Level Advanced Score A5 for candidates passing the • Normalized score = 0.5 A4 +0.5A5 Alternative system

32. Considerations of six different options • Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system according freedom for the individual institutions to select mixing proportions within a pre-specified guideline • Option 6: Equal weighting of School Board performance as subject score and National Level Aptitude Test as objective test system where Normalized score =0.5 A2+0.5A4 Alternative system

33. Recommended preferences of the committee • Option 2: Deployment of Scores as criteria based on class XII performance • Option 6: Equal weighting of School Board performance as subject score and National Level Aptitude Test as objective test system 0.5 A2+0.5A4 • Option 5: Deployment of School Board performance as subject score and National Level Aptitude Test as a combination and avoid the Advanced Testing system with freedom for the individual institutions to select mixing proportions within a pre-specified guideline • Option 4: Deployment of School Board Performance as screening but not as determinant for National ranks; Specify a Cut-off normalized percentile rank score for school performance say as 80 or 85 percentile rank; and rank by 0.5 A4 +0.5A5 Alternative system

34. A Suggestion • A committee of experts from engineering institutions could be assigned the task of interacting with ISI Group for internalization of methodology for normalization of board scores • IITs could be assigned the task of setting up a question paper for National Screening Test based on objective examination models and conduct the examinations for the year 2012-13 • For Aptitude examination like SAT, we may take the help of NTS or any other global agency Alternative system

35. Thank you For the patience and waiting We recommend normalization of school board scores for factoring-in based on ISI inputs Single National level test to cover aptitude and advanced or aptitude alone and Judicious mix of school and national test performance Alternative system