K. G. Viswanadhan Dept. of Management Studies IISc, Bangalore

1. How to get responses for multi-criteria decisions in engineering education An AHP based approach for selection of measuring instrument K. G. Viswanadhan Dept. of Management Studies IISc, Bangalore

2. Agenda • Introduction • Analytic Hierarchy Process • Background • Objective • Methodology • Results and Analysis • Conclusions How to get resposes: KG Viswanadhan

3. Introduction • Socio-Economic systems often face decision making with qualitative and intangible factors. • Values, beliefs and perceptions are the force behind many of the decision-making activities. • Decision making process relies on the information of alternatives. • Information may vary from scientifically derived hard data to subjective interpretations. How to get resposes: KG Viswanadhan

4. Introduction • Decision depends on multiple criteria. • Key problem is to elicit systematic judgments from unstructured information. • Analytic Hierarchy Process (AHP) is a powerful tool in this situation for decision making. How to get resposes: KG Viswanadhan

5. Analytic Hierarchy Process • AHP is a mathematical technique used for multi-criteria decision-making. • It is designed to incorporate tangible as well as non-tangible factors especially where the subjective judgments of different individuals constitute an important part of decision making (Saaty, 1980). • Intuitively solved decision problems can now be solved as procedure-orientated using AHP. • Applications of AHP can be seen in a wide range of areas. How to get resposes: KG Viswanadhan

6. Analytic Hierarchy Process • Merit salary recommendation system (Troutt M D & Tadisina S K, 1992). • Environmental impact assessment (Ramanathan R, 2001) • Performance measurement system (Suwignjo P, 2000). • TQM implementation (Chin K S et al, 2002). • Determination of key capabilities of a firm (Hafeez K et al, 2002). • Credit evaluation of the manufacturing firms (Yurdakula M & Tansel Y, 2003). • Selection of alternative transportation options (Yedla S & Shrestha RM, 2003). How to get resposes: KG Viswanadhan

7. Background of the study • Unprecedented growth of engineering colleges in India in the last two decades led to complex issues. • The main issue was that of maintaining the quality of education provided by these colleges. • Decided to conduct a study to find out the quality issues of undergraduate engineering education programmes. • Faculty opinions have to be collected. • Opinions will be subjective, and based on multiple criteria. • AHP is adopted to analyze the problem. How to get resposes: KG Viswanadhan

8. Background of the study • The hierarchy designed for the research problem includes two levels of decision-making. • Total number of pair wise comparisons for the two levels were 37. • A tiring exercise to the respondent. • Design of a proper questionnaire for data collection is very important. • Decision-making process again focuses on AHP. AHP for AHP Measuring Instrument How to get resposes: KG Viswanadhan

9. Objective of study To develop the best questionnaire for the pair wise comparison of ‘AHP for prioritizing the quality issues of undergraduate engineering programmes’ . How to get resposes: KG Viswanadhan

10. Methodology adopted • Five types of questionnaires were developed for collecting the pair wise comparison of ‘quality issues of engineering education’. • Three criteria were considered while designing the questionnaire . • easiness to fill up the questionnaire • clarity of questions • extraction of intended responses How to get resposes: KG Viswanadhan

11. Five-step process of AHP • Create a decision hierarchy by breaking down the problem into a hierarchy of decision elements. • Collect input by a pair wise comparison of decision elements. • Determine whether the input data satisfies a consistency test. If it does not, go back to Step 2 and redo the pair wise comparisons. • Calculate the relative weights of the decision elements. • Aggregate the relative weights to obtain scores and hence rankings for the decision alternatives. How to get resposes: KG Viswanadhan

12. Step 1: Formation of hierarchy How to get resposes: KG Viswanadhan

13. Step 2: Collection of inputs • Experts • Five faculty from five engineering colleges coming under five different states. (Gujarat, Andhra, Karnataka, Tamil Nadu and Kerala). • Nominal-ratio scale of 1 to 9 (Saaty, 1994) is adopted for pair wise comparison of the questionnaires. • All the five questionnaires were given randomly to the experts. • Asked to conduct a pair wise comparison of the five questionnaires based on the three criteria . • Responses were collected and recorded. How to get resposes: KG Viswanadhan

14. Step 3: Consistency Test • Results of pair wise comparisons were filled in positive reciprocal matrices. • Eigenvector and maximum eigenvalue were calculated. • Consistency ratios of all respondents were found out. All the consistency ratios were found to be less than 0.1 and all the responses were accepted for analysis. How to get resposes: KG Viswanadhan

15. Steps 4&5: Calculation of relative weights & ranking of alternatives • Though all the expert opinions are consistent, the opinion of one of the experts seemed to be entirely different from others. • This expert is informed of the variation and asked whether he wants to stick on to his earlier opinion or ready to modify the same to match with the others. • The expert selects second option and the pair wise comparisons are modified accordingly. • Geometric mean of individual opinions are calculated and entered in the final judgmental matrix for finding out the ranks of the alternatives. How to get resposes: KG Viswanadhan

16. Judgmental matrix and the ranks of the questionnaires How to get resposes: KG Viswanadhan

17. Results and Analysis • The priority vector indicates that Questionnaires 1 and 2 are equally good for data collection. • Questionnaire 4 is also acceptable to the experts. • A detailed and simple questionnaire is preferred to a compact and less time consuming method. • Fifth questionnaire, though very simple cannot gather the required information of pair wise comparisons. • Third questionnaire is very compact but a novice user who is unaware of matrix structures will find it very difficult to fill it up. How to get resposes: KG Viswanadhan

18. Results and Analysis Based on the rankings given by the experts and through the interactions with them, it is decided to develop a modified questionnaire, which is a combination of first, second and fourth. How to get resposes: KG Viswanadhan

19. Conclusions More insights about the essential features of a measuring instrument are obtained and the analysis finally leads to the development of a new structure to the measuring instrument. How to get resposes: KG Viswanadhan

20. Conclusions • Today most of the decisions are to be taken in increasingly complex environments. • AHP, which is a transparent technique, is very useful to handle this type of situations where qualitative data is involved in the decision-making. How to get resposes: KG Viswanadhan

21. Thank You How to get resposes: KG Viswanadhan