Ali Ekrem Ozkul , Zehra Kamisli Ozturk Anadolu University, Open Education Faculty, Turkey

1. MULTI CRITERIA EVALUATION OF DISTANCE EDUCATION IMPLEMENTATION MODELS USING ANALYTIC HIERARCHY PROCESS AliEkremOzkul, Zehra KamisliOzturk Anadolu University, Open Education Faculty, Turkey aeozkul@anadolu.edu.tr, zkamisli@anadolu.edu.tr NurayGirginer Eskisehir Osmangazi University, Business Administration Department, Turkey girginer@ogu.edu.tr

2. GOAL Evaluation of distance education delivery modes in implementation model by using a multi criteria decision making tool, Analytic Hierarchy Process (AHP) for the English Language Teacher Training (ELTT) Program at Anadolu University Open Education Faculty.

3. English Language Teacher Training Program (ELTT) • ELTT is 4-year English Language Teacher undergraduate education • Program aims to meet demand English language teacher in primary and secondary education in Turkey. • Program in ELTT is a blended learning model. Courses in ELTT program offered in the first two years are face to face while in the remaining two years are through Distance Education

4. Decision Problem Determination of the share of different delivery models in the implementation model.

5. contrIbutIONs • To realize group decision making like this multi criteria decision problem in distance education • To suggest a hierarchical approach with four levels including tangible and intangible criteria to evaluate different delivery modes • To indicate applicability AHP for this decision problem

6. METHEDOLOGY ANALYTIC HIERARCHY PROCESS (AHP) • AHP has a systematic approach to set priorities and trade-off among criteria • AHP can measure all tangible and intangible criteria in the decision process • AHP is a simple, intuitive approach that can be accepted by decision makers • AHP can easily be used to solve multi-criteria decision problems involving multi-actors or group decision making with multi-actors

7. STEPS OF AHP • Define the decision problem and determine its objective. • Define the decision criteria in the form of a hierarchy of criteria. • To make the pair-wise comparisons, structure the comparison matrices of criteria and decision alternatives. • Obtain the importance of the criteria and sub-criteria by the comparisons. • Determine the weight of each criterion. By hierarchical synthesis, determine the priorities of each alternative.

8. GroupdecIsIonmakIng There are three basic approaches that a group can use to assess weights: • consensus • vote or compromise • geometric mean of the individual judgments. Geometric mean is the most common approach used in AHP with group decision making

9. ELTT • It is important to determine the weight and priorities of each delivery modes (face-to face, online etc) to execute the program succesfully • In the selection of delivery modes for implementation phase should be taken into account besides the tangible criteria like cost, number of personnel and students etc and also intangible criteria like interaction, benefit etc. • These criteria must be considered together and so this complex decision problem obligate to use one of the multi criteria decision making techniques as AHP.

10. Formulating Hierarchy of Delivery Models Ranking • GOAL: Determination of the share of different delivery models in the implementation model in ELTT • MAIN CRITERIA AND THEIRS SUB-CRITERIA: • Cost (investment, developing course, operating, assessment costs) • Access(number of enrolled students, status of students) • Staff (number of personnel, status of the personnel) • Interaction (student-student, student-learning materials, student-teacher) • Media of Instruction(design, production, edition, control) • Benefit (experience, number of graduates, pioner in ELTT) • ALTERNATIVES • Traditional Delivery Model (T-ELTT) • Videoconferencing based Delivery Model (V-ELTT) • Web based Delivery Model (W-ELTT)

11. Hierarchy of Delivery Models Ranking

12. Obtaining Pair-wise Comparison Matrices • For the pair-wise comparisons, a questionnaire was prepared. • While taking the judgment for each expert (There are three experts), interviews were made separately each other, by using questionnaire technique. • The scale that takes integer values between 1 and 9 were used in the technique recommended by Saaty. The valuation scales in the pair-wise comparisons are those, where 1 is equal importance, 3 is moderate importance, 5 is strong importance, 7 is very strong or demonstrated importance, and 9 is extreme importance. Even numbered values will fall in between importance levels. Pair-wise comparisons were based on upper level control criteria

13. Obtaining Pair-wise Comparison Matrices • First of all, weights of main criteria must be determined first. For this reason, the experts made their pair-wise comparisons about main criteria and notified their judgments according to overall goal. • The paired comparison matrix attempts to capture the relative dominance of one element over another with respect to an attribute that they have in common. The questions to ask when comparing two criteria are of the following kind: Of the two criteria being compared, which is considered more important according to the goal and how much important? And the same question type is also used when comparing alternatives according to the criteria. • Three pair-wise comparison matrices belong to each expert have been obtained for pair-wise comparisons of main criteria. Each expert gives his judgments in 25 pair-wise comparison matrices.

14. comparIson of maIncrIterIA

15. Obtaining Pair-wise Comparison Matrices • The experts’ data based on pair-wise comparison matrices designed by Expert Choice were input. The software then calculated relative weights for the alternatives with respect to the main criteria and their sub-criteria. In the result of these, 75 pair-wise comparison matrices were obtained from judgments of experts for the whole model. Relative Weights of Main Criteria Due to Experts

16. Investment (0.013) Developing course (0.008) Cost (0. 035) Operating costs (0.006 ) Examination-assessment (0.008) Numbers of enrolled student (0.065) Access (0.183 ) Status of the students (0.118) Determine to delivery model’s the most appropriate share in implementation model of ELTT Numbers of staff (0.062) Staff (0. 101 ) Status of the staff (0.039) Student-student (0.051) Student-learning material (0.065 Interaction (0.182) Student-teacher (0. 065) Design (0.065) Media of Instruction (0.123) Production (0.027) Edition (0.016 ) Control (0.015) Experience (0.187) Benefit (0. 375) Number of graduates (0.128) Being a leader in English language teacher program (0.061) Final Relative Weights

18. DISCUSSIONS AND CONCLUSIONS • A blended distance education ELTT program can be based on %41.4 face-to face (traditional), %30.9 web and %27.8 videoconferencing technologies • It’s an expected result that face-to-face education takes the biggest ratio according to a foreign language teacher training program. Especially the assessment of the teacher applicants needs a synchronous delivery model. • Based on the program’s curriculum, the lessons can be given based on the ratios either yearly or per terms

19. DISCUSSIONS AND CONCLUSIONS • Based on the developments of information technology, determination of the weight of delivery modes that includes basic elements of distance education is an important decision. So • We propose a decision model for the institutions that will develop or implement a distanceeducation program. • The AHP is a powerful multi-criteria method that considers both tangible and intangible criteria. • We integrate different opinions of different expertsby applying group decision making • This methodology in our study can be used different decision problems in distance education management.