Innovative Assessment Software: Cadmus Demo & Evaluation

1. CAPHIA 2019 • Dr Lesley Andrew • Postgraduate Courses Coordinator • Edith Cowan University • WA • I.andrew@ecu.edu.au

2. A demonstration and evaluation of Cadmus software • L Andrew, L Balmer

3. Presentation • Introduce the software and its underpinning philosophy • Demonstrate its features • Evaluate its potential

4. New student assessment software

5. What is Cadmus? • Cadmus is described as “a shared, online environment where teachers create and students complete written assessment” (Cadmus, 2019). • The software is promoted as a positive, standardised and flexible approach, which simplifies the learning experience and in doing so, enables students to focus on the content of their assignment rather than the navigation of the assessment environment.

6. What’s stopping us improving assessment? • Our current technologies and processes: • 1. Make it hard to understand the effectiveness of our assessment tasks. • 2. Lack flexibility in how we design and conduct our assessment. • Add complexity and increase staff and student workload. • A shared, online environment where teachers create and students complete written assessment. With Cadmus we can: • 1. Simplify assessment, making it easier to manage and clearer with students. • 2. Easily design and deliver flexible, authentic assessment tasks. • 3. Gain insight into student learning, and actively guide them towards the assessment outcomes.

7. Progress reports • Summary Report: HST5161.2019.1.ONCAMPUS_OFFCAMPUS • 45 students submitted a final, which is a submission rate of 90%. On average, students spent 5 hrs and 48 mins preparing their submission. • Activity over time • The average document word count was 2361. • Resources Accessed % cohort • 2013-Evaluating_information_sources.pdf 42% • Marking rubric.docx 44% • Library Reference Guide 20% • Academic Writing Guide 20% • Annotated_bibliography.pdf 46%

8. Tutor evaluation • Reports on progress very helpful • Reports on reading of attachments • Links with Turnitin and gradebook • Resources all in one place • Paste Chips (“prompts”) — Paste Chips are designed to help students identify and follow correct academic practices when using the ideas of others in their own work. When a student pastes content into Cadmus from an external source, a Paste Chip will be added to the end of the pasted text — this tag tells the student how many words have been pasted, and reminds them to consider paraphrasing or add a citation if necessary.

9. Student evaluation: ECU-wide 2018-2019

10. Student evaluation: Medical genetics undergraduate units • 78 students in their second year of the Medical Genetics degree. • “Student satisfaction was measured by indicating if they would use CADMUS for all other assignments in their chosen field of study. Seventy two students were involved in the study, 70% of students would use the program again, 10% would not and 20% were unsure.”

11. Master of Public Health introductory unit Comments from HST5161 “Fine” “Simple and easy to use” “It was awesome.”

12. Potential limitations- my perspective • The potential to over-support the assessment process, and thereby limit individual skill development, such as formatting • The work of students who do not work solely in Cadmus (i.e. cut and paste from word instead) cannot be analysed for contract cheating. • Not useable on mobile devices • Requires internet connection • No thesaurus option • Overwhelming for less computer literate and international students

13. A way forward to stop contract cheating? • “Cadmus is an online web authoring environment (similar to Google Docs) that aims to address the problem of contract cheating by providing an authenticated, online authoring environment for individual written assessment. This environment includes: keystroke biometrics; multi-factor geolocation authentication; and the capture of unique data that is analysed automatically to highlight statistical outliers. Taken together, the system presents an opportunity for identified cases to be further investigated to determine whether the work was authentically constructed within Cadmus, and whether the submitted work was authored by the appropriate student.” • University of Melbourne

14. Further drawbacks • The work of students who do not work solely in Cadmus (i.e. cut and paste from word instead) cannot be analysed for contract cheating. • As a response to Student Union concerns- the emphasis on this aspect of Cadmus is now played down

15. University of Melbourne Student Union Freedom to work in a way that suits you Being monitored while you work is extraordinarily intrusive. Cadmus could mark your assignment as deviating from the norm if you copy & paste a lot when editing or transcribe from another word processor or hand-written notes. This means you will be investigated for potential academic misconduct by the University. If this happens, the burden is on you to prove you did not cheat. Personal data Cadmus will be tracking data such as how many words you type and at what speed, how many deletions you make, how many sessions it takes for you to complete your work and when those occur, and where you are when you access Cadmus. Lack of equity Currently Cadmus requires you to go through online verification every time you log in, disadvantaging students who do not have regular internet access.

16. Conclusion • Usefulness depends on student cohort • Equity issues • The way it is introduced and used