Electrical and Electronic Engineering Assessment Network (e 3 an)

1. Electrical and Electronic Engineering Assessment Network (e3an) a three year project supported by the ‘Fund for the Development of Teaching and Learning’ (FDTL)HEFCE (Higher Education Funding Council for England) DEL (the Dept for Education and Learning in Northern Ireland)

2. motivations • “In the best examples the use of directed and independent learning is well integrated with taught elements. …In the best examples of assessment, assignments provide an appropriate level of challenge and feedback to students on coursework is prompt and informative. In other cases, there are delays in the return of marked work and inconsistencies in the quality of feedback.” QAA overview report

3. motivations (?)

4. drawbacks (?)

5. build a people network peer reviewed test banks for electrical and electronic engineering questions produced and reviewed by paid consultants model of processes trojan horse for educational development supporting materials on methods and practices May 2000 …the plan

6. questions banks containing: objective tests (MCQ, MRQ numeric,++) text based traditional tests course assignments exam questions self assessment, diagnostic,feedback, reference/exemplars technical issues must be in range of output formats must be interoperable must have good descriptive content (meta data) equal importance people network and identifying good practice …the details

7. UK academics consultants, externals PSBs core participants

8. resource potential time savings improved feedback to students ready to run questions: easier to trial MCQ software (e.g. web based QM Perception) practice evaluation of assessment innovations on campus identification of good practice dissemination and discussion of assessment practices the benefits

9. production model - theme teams • theme leader • theme mentor • four or five academic consultants

10. database contents

11. consultant recruitment ‘00 partner pilot • 18 consultants • recruit by word of mouth • two briefings ‘01/’02 uk-wide; 35 HEIs • 50 consultants • two early launches • five briefings • recruit by flyers, email, word of mouth

12. theme team process identify question targets • discuss levels and approaches • agree division of tasks • dummy run author • write, mark up and review questions • trial and evaluate questions

13. available test bank via download or CD • currently 1500 questions • ongoing conversion and addition • ultimately around 2800 questions further development • contributions welcomed • plans for ongoing use and renewal download from e3an website at http://www.e3an.ac.uk

14. overview theme sub-theme related themes description keywords detail type mark time level cognitive level discrimination style examples View question View metadata metadata

15. output formats of questions tutor or student ready, various formats • Microsoft Word/RTF HTML • QML (Question Mark Perception v2) • XML Question & Test Interoperability (QTI) specification (compatible with a range of test engines)

16. related developments & resources acronym hell • TOIA http://www.toia.ac.uk/ Technologies for Online Interoperable Assessment • STOMP QTI viewer http://www.stomp.ac.uk/ • CETIS assessment SIG http://www.cetis.ac.uk/assessment/ • LTSN-ICS question bashes http://www.ics.ltsn.ac.uk/question-bank • TQEF/NCTeam assessment project network http://www.ncteam.ac.uk/projects/fdtl/fdtl4/assessment/ • LTSN generic resources on assessmenthttp://www.ltsn.ac.uk/genericcentre/

17. test bank evaluation • extensive usability testing of the database which stores and retrieves questions • implementation studies • currently extending project collaborators willing to help evaluate: • question selection methods • potential benefits to tutors and students • appropriateness of the question descriptions (metadata) • additions and maintenance

18. e3an - more details • Su White http://www.ecs.soton.ac.uk/~saw/ • saw@ecs.soton.ac.uk • +44 (0)23 8059 4471 • http://www.e3an.ac.uk/ • info@e3an.ac.uk/