1 / 42

Improving assessment quality using electronic testing

Software Engineering Education and Reverse Engineering. Improving assessment quality using electronic testing. Goce Armenski, M.Sc. Institute of Informatics - FNS University “Ss . Ciryl and Methodious” – Skopje, Macedonia. 1. Content. CONTENT. - I NTRODUCTION.

galvin
Download Presentation

Improving assessment quality using electronic testing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Software Engineering Education and Reverse Engineering Improving assessment quality using electronic testing Goce Armenski, M.Sc Institute of Informatics - FNSUniversity“Ss. Ciryl and Methodious” – Skopje, Macedonia

  2. 1. Content CONTENT - INTRODUCTION - МOTIVATION AND GOALS - ЕТЕСТ – NEW SYSTEM FOR ELECTRONIC TESTING - BASIC CONCEPTS OF THESYSTEMS FOR ЕТESTING - АRCHITECTURE, CONCEPTS AND FUNCTIONALITY OF ЕТEST - APLICATION OF ЕТEST - RESULTS - CONCLUSION Goce Armenski

  3. 2. Introduction 1. INTRODUCTION Changes in the way people live influenced by:- Globalization- Increased meaning of knowledge- Information and communication revolution “Industrial society”  “Information society”“Information society”  “Knowledge society” 50% of working skills are getting old in 3-5 years Goce Armenski

  4. 3. Introduction Changes in educationTeacher  Student centered education

  5. 4. Introduction Changes in the education (lot more students)- Lecturing process- Delivery of materials- Knowledge assessment • What are the assessment functions? • Check the successfulness in achieving learning goals • Provide feedback • Improve learning process Keysachieved14% improvement of student results when assessing their knowledge once a week instead once a month. Pikunas and Mazzota also mention improvements in results of 10%, when tests are delivered every week instead every 6 weeks Goce Armenski

  6. 5. Introduction Institute of Informatics (FNS)- every year 150+ students- 4 questions (assignments) x 150 students = 600 questions- 5 minfor marking a question- 3000 min / 50 hours How long the marking will take??? Ontime feedback??? Personalized feedback??? Objectivity??? Goce Armenski

  7. 6. Motivation and goal 2. MOTIVATION AND GOAL Development of a system for assessment of large number of students (more that 150) every month, which can be used in distance learning, as well as any other form of knowledge or skill assessment. • Motivation for test construction Different tests for ecery student with same weight Lowering the possibility for memorizing • Motivation for simple data entry Possibility for bulk entering of data • Motivation for reports creation Statistical analyses of gathered information Goce Armenski

  8. 7. Motivation and goal Goal Focus of the research:To implement a system for computer based assessment and evaluate the results from its useGlobal goal:Identify the influence of this system to the teachers and studentsMain goal:Is computer based assessment more effective and objective than the traditional assessment Goce Armenski

  9. 8. еТest – concepts and functionality 2. еТEST – CONCEPTS AND FUNCTIONALITY Basic concepts of systems for eTesting QuestionMark, BlackBoard, WEB CT, Top Class, EduSystem - Question bank, - Algoritms for test creation, - Systems for data presentation, - Result reports Goce Armenski

  10. 9. еТest – concepts and functionality Question bank Database of unique questions, with needed characteristics for simple selection during the test construction.some of the parameters are created dynamicaly • Standards for question bank development.- Exchange of questions of university level Goce Armenski

  11. 10. еТest – concepts and functionality Fig. 2 Types of questions • Fixed answers questions(objective) - multichoice - short entry - questions with graphical selection (hotspot)- Free text answers - programming code as answer - essay answer Goce Armenski

  12. 11. еТest – concepts and functionality Algoritms for test creation Fig. 3 Algoritms for test creation Main difference between them in the adaptation to the characteristics of the person which knowledge is assessed Goce Armenski

  13. 12. еТest – concepts and functionality Systems for data presentation Content depending on the characteristics of the monitor and the network Marking and reporting In which way the system shows the results?- shows true answers?- the result is in points or percents?- negative marking??? Goce Armenski

  14. 13. еТest – concepts and functionality еТest Technology Web based sollutionsVS desktop based sollutions • Web based application- Active Server Pages (ASP)- JavaScript- SQL Server 2000- NT Server and Win 2000 compatible- IIS 4.0 or newer Web Browser (Netscape 4.x or Internet Explorer 4.x, and above) JScript Email (SMTP) SQL Server Access IIS 4.0 + ASP Win 2000 Server Goce Armenski

  15. 14. еТest – concepts and functionality еТest Architecture Fig. 4 Three tier architecture of the eTesting system Goce Armenski

  16. 15. еТest – concepts and functionality еТест Concepts • Types of users- Course organisation- Types of questions- Test creation algoritm- System for data presentation- Marking and reporting Goce Armenski

  17. 16. еТest – concepts and functionality Types of users Fig. 5 Types of users Goce Armenski

  18. 17. еТest – concepts and functionality Course organisation • learning objects- three structure Fig. 6 Course organisation Goce Armenski

  19. 18. еТest – concepts and functionality Types of questions • Multichoice questions (choose one of many, choose many of many, yes/no answers); • Short entry answer (text or numerical); • Essay answer. • Questions can have pictures or graphs in the text or offered answers Goce Armenski

  20. 19. еТest – concepts and functionality Multichoice questions Fig. 7 Choose one of many Fig. 8 Choose many of many Goce Armenski

  21. 20. еТest – concepts and functionality Short entry answers Сл. 9 Short entry Goce Armenski

  22. 21. еТest – concepts and functionality Questions with essay answer Fig. 10 Essay answer • these answers are not evaluated by the system- lowering the objectivity- Project Essay Grade (PEG)- Intelligent Essay Assessor (IEA),- Erater,- Bayesian Essay Test Scoring System (BETSY). Goce Armenski

  23. 22. еТest – concepts and functionality Test creation algoritm • dynamic linear tests (fixed number of questions) System for data presentation • adjusted to Web standards (800x600)- possibility for picture and graph display Marking and reporting • results are shown at the end of the test- negative marking Goce Armenski

  24. 23. еТest – concepts and functionality Statistical data analyses • Identification of content which is not well presented; • Personalized feedback to students; • Identification of week questions which need to be revised before used again; • Identifying the individual weeknesses of students. Goce Armenski

  25. 24. еТest – concepts and functionality Inportant statistical data about questions: • difficulty – how difficult a question is; • discrimination – how well the question diferentiate good from bad students; • guessing – how students answer the corect answer without knowing it; • Different achievements – what are the results of diferent user groups. Goce Armenski

  26. 25. Application of eTest 4. APPLICATION OF еТEST - Integration in the process of learning - Controled learning Fig. 12 Way of passing the learning objects • Succesfull strategy?? • all questions • Nquestions in a row • Nwright questions • 3 wright questions in a row Statistical analyses of the user activities Goce Armenski

  27. 26. Results 5. RESULTS Prctical Implementation • Institute of Informatics , FNS (2001) • А.D. Mobimak (2002) • UNDP (2003-2004) The use of technology in education is very dependent of the organization - logistics - socal changes Syncronisation with other systemsTechnical infrastructure Goce Armenski

  28. 27. Results Data gathering Institute of Informatics, FNS (january 2001) - 26 courses - 12391questions - 589 scheduled assessments - 9861 generated tests Does eTesting provides more effective and more objective assessment compared to the traditional forms, and does it help the learning process Goce Armenski

  29. 28. Results Cost savings Decreasing the use of paperDecreasing the use of print materialDecreasing the time for conducting the assessment Technical infrastructure Software development and maintenanceTraining of administratorsTechnical personal Goce Armenski

  30. 29. Results Teacher perspective Creating of question bank is time consuming - more than 1300 per courseTime saving after the creation Student perspective Questionary with 10 questions - Questions with Likerttype of answers with 5 values - 236 students were enroled in the survey Goce Armenski

  31. 30. Results Question 1 Goce Armenski

  32. 31. Results Question2 Goce Armenski

  33. 32. Results Question3 Goce Armenski

  34. 33. Results Question4 Goce Armenski

  35. 34. Results Question5 Goce Armenski

  36. 35. Results Question6 Goce Armenski

  37. 36. Results Question7 Goce Armenski

  38. 37. Results Question8 and 9 Goce Armenski

  39. 38. Results Question8 and 9 cont Goce Armenski

  40. 39. Results Question10 Goce Armenski

  41. Results 40. Summary of the results

  42. 41. Conclusion 6. CONCLUSION System implementation • Dependent of the institution in which it is implemented Improvements in the assessment process • Possibility to assess knowledge of more than 150 students • Immediate feedback • Lowering the subjectivity • Self testing possibility before the official assessment • Analyses of the gathered data • Possitive effects on the learning process • Incresed security??? Combining this method with other method of assessment can give great results Goce Armenski

More Related