1 / 17

MASSIVE APPLICATION OF ONLINE ASSESSMENTS

Learn how UABC utilized Questionmark Perception for collegiate evaluations, improving educational quality and examination processes. Experience the evolution from pilot experiments to successful execution, overcoming challenges and implementing effective strategies.

jbonnie
Download Presentation

MASSIVE APPLICATION OF ONLINE ASSESSMENTS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MASSIVE APPLICATION OF ONLINE ASSESSMENTS Espinosa Díaz Yessica, Figueroa Rochín Claudia Araceli, Perezchica Vega José Eduardo, Sepúlveda Rodríguez Jesuán Adalberto

  2. CollegiateExaminations The Universidad Autónoma de Baja California (UABC) has sought from the Institutional Development Plan (IDP) 2003-2006, ¨to know systematically the achievements in the learning processes for students¨ setting in line with the objectives of initiative 11.1 of the mentioned IDP to ¨have systemized information on the quality of learning achieved, to be taken into account to improve the educational process¨ [1]. The corresponding strategy was then to: “Develop a system of collegiate examinations for each one of the academic programs, to evaluate the achieved learning levels of students at the beginning, during and end of their studies, through the selection of key courses that serve as a parameter for evaluating the quality of the learning accomplished.¨ [2] To continue with the departmental examinations, during the course and upon graduation of the Undergraduate Educational Programs (EP). Systematize the results of the learning evaluations to strengthen the evaluation processes of the EP. [4]

  3. Questionmark Perception in UABC Open Education Center (OEC), in order to fulfill its purpose of ¨providing institutional services for managing online courses, providing consulting, training and additional services on information technologies for teaching in accordance to the requirements of the academic programs at UABC¨ [5], Acquired in December of 2005 the software license for Questionmark Perception (QMP) for the application of online surveys and evaluations. This, in line with the fundamental purpose of support to the academic institutional projects and provision of a stable and reliable tool that allows the generation of results in a digital form, immediately after its use.

  4. Questionmark Perception in UABC QMP is a system for evaluation management that allows educational institutions to monitor all the processes related to an evaluation supported by computers. Its four main principles and tools are recognized in the following manner: • Authorship. To create questions and organize them into exams, quizzes and answers, tests and surveys. • Programming. To assign evaluations for individual and group participants. • Distribution. To distribute the evaluations through a Web server, a safe server (Questionmark Secure), PDA, Smartphones, CD or printed copies. • Reports. QMP allows the user to run result reports and to analyze it by using 11 different report types.

  5. Engineering Department experience (since 2008) Considering when the ED approached OEC to make use of QMP en their collegiate exams, the project had experienced development in different times as: The firstpilot experiments (2008-2 and 2009-1): QMP use, exploring of their features and logistic application with laboratories supervision. Experiences of growth (2009-2 and 2010-1): use of more robust and complex exams (randomization, question banks, use of graphs and equations, interface customization, etc). During this period, technical problems were detected and also some loss of information associated with network quality or loss of evaluation connection of the QMP server. Consolidation experiences (2010-2 and 2011-1): After the QMP migrated to Version 5 in the summer of 2010, relevant updates were done, compatibility checked, a specific test was performed, and collective exams were applied in order to know the QMP stability in this version at the new server. The test was successful and the final implementation was achieved against data loss, disconnections, etc. thanks to the previous preparation and the application´s stability.

  6. Challenges During the application of evaluations, 2009 to 2010 using QMP (Question Manager Perception), some problems were found, such as: 1) records were lost once the student was finished with the exam, because of a lack of answer from the server, 2) some computers did not show graphics for some questions, 3) some computers presented error messages, 4) the assessment did not automatically started in the expected time.

  7. Actions Even though the problems encountered were minimum compared to the obtained benefits, the goal was to reduce or eliminate all failures and the next actions were taken: • A server was acquired with high performance features and the latest release of QMP was installed with a stable internal structure. • Local tests were performed on computers that did not display graphics, identifying the local antivirus system performed the blocks. • Error messages did not appear again after working with the lasts version installed. • Evaluations were currently scheduled to be active with a range of 5 minutes before and after the set time.

  8. Economic-Administrative SED experience (since 2010-1). Unlike other AUs (Academic Units) in UABC, those belonging to the Economic-Administrative SED (Administrative Sciences Department-Mexicali, Accounting and Administration Department-Tijuana, Economy and International Relations Department -Tijuana, Tourism and Marketing Department -Tijuana, Administrative and Social Sciences Department -Ensenada, Engineering and Business Department - San Quintín) joined the project. When they decide to start exploring the application of collegiate examinations, it is done as a joint work among all Economic-Administrative SED since their offered academic programs are homologated in the common core structure (basic stage).

  9. Economic-Administrative SED experience (since 2010-1) Both semesters in which Economic-Administrative SED have participated in collegiate examinations have meant an increment in the quantity of targeted users and, in general terms; it represents a successful project desirable for a continuous support. So far, the exams available are for Accounting and Mathematics, both for fresh year students in almost all the academic programs of Economic-Administrative SED.

  10. Challenges • Some of these were mainly associated to the internal logistic, others to the stability of the platform and complexity of the designed evaluation. • However, a challenge to keep in mind was that initially the SED is distributed through the State of Baja California, and therefore, are unequal in a connectivity condition.

  11. Chemical Sciences and Engineering Department -Tijuana (CHSED) experience (since 2010-2). • The CHSED started to explore QMP since the application of admission exams for graduate studies during 2010-1 and 2010-2 in the Math placement level, aspart of the propedeutic course for freshmen students. However, it is until 2010-2 that collegiate examinations are implemented with support of QMP; previously these evaluations were developed by FIM in a paper-based version.

  12. Challenges • Due to the work of ED and Economic-Administrative SED, the project successfully over passed the initial pilot problems and those related to the increment of participants. Therefore, it can be stated that the problems identified in the project of CHSED were isolated cases encountered on the internal logistic on a first time application. • The next chart shows • the project growth in relation • to targeted students

  13. Good Practices And General Recommendations • Logistics. • Designate a team or staff that will be responsible for undertaking the evaluations and defining the attributes concerned with the design, development and validation of collegiate examinations. • Define the security mechanisms that are required in an evaluation applied via Internet such as: keyboard blocking, use of mobile phones, generation of random questions, etc. • Identify, in accordance to the number of students that will undertake the evaluation, the physical spaces or computer labs that will be needed for this purpose, with the purpose of assessing in advance their conditions, characteristics and configurations. • To have a contingency plan, in the event that the application cannot be used as expected. Alternative solutions can be defined, for example the application of paper based exam, rescheduling the evaluation, changing the venue, connections, etc.

  14. Connectivity and equipment • 1.- Evaluate the internal network´s conditions and its link to the server which maintains the application in order to avoid problems of connectivity, response time and information loss. • 2.- Inform all involved entities (Computer Science Area, local and departmental, and Direction of Academic Unit)  during the time of evaluations, so that all agents will be ready to act upon a possible failure, providing support and solutions. • 3.- Previously check all computer equipments that will be used during the evaluations since although they are expected to maintain a same configuration, a different parameter (such as the actions of an antivirus software) may result in a different configuration of the equipment. • 4.- Local adequate installation (upon approval) of the QMP programs, such as Questionmark Secure, software that blocks keyboards and popup windows, among other functions.

  15. Connectivity and equipment • 5.- Not to use the computer equipment right after it has been configured and while testing the application, but rather after the evaluations. • 6.- Constant update of QMP application. • 7.- In order to diminish or avoid problems during an evaluation, a strong communication with the QMP support service or community is desirable. This will allow having information with anticipation to an event about the experience or problems that occurred in other sectors and the course of action for solving these.

  16. References • [1] Universidad Autónoma de Baja California. Institutional Development Plan 2003-2006. 2003. • http://www.uabc.mx/planeacion/pdi/2003-2006/PDI.PDF • [2] Ibid. • [3] Universidad Autónoma de Baja California. Institutional Development Plan 2007-2010. 2007. • http://www.uabc.mx/planeacion/pdi/2007-2010/PDI.PDF • [4] Ibid. • [5] Universidad Autónoma de Baja California. Acuerdo de creación del Centro de EducaciónAbierta. 2006. • http://sriagral.uabc.mx/Externos/AbogadoGeneral/index_htm_files/(19).pdf • [6] Questionmark. Questionmark Secure. 2010. http://www.questionmark.com/esp/perception/questionmarksecure.aspx

  17. Thanks for your attention

More Related