Peer Learning Assessment System PeLe

1. Knut Bjørkli Rolstad(knut.b.rolstad@ntnu.no) Norwegian University of Science and Technology (NTNU) Peer Learning Assessment System PeLe M-Lang project, agreement number 2014-1-NO01-KA200-000425

2. Contents • Brief overview of PeLe • Using PeLe • Answering a test as «student» • Preparing a test • Monitoring a test • Reviewing a test • Discussion of methodology

3. What is PeLe? • Tools for running multiple-choice tests with automatic grading • Emphasizes immediate feedback and student collaboration («peer learning») • PeLe facilitiates learning activities immediately following a test

4. PeLe can be used for a wide spectrum of applications Peer learningtool Tool for automaticallycorrecting tests and immediate feedback

5. Doing a test withPeLe Test done in theclassroom Test is handedout to students onpaper Short break whileteacherpreparesreview Afterthe test: review + learningactivities PeLegives a realtime overviewofhow students perform

6.  You create your PeLe user account • You will use this account both as «student» (anwering tests) and as «teacher» (creating tests) • Fill in the form on this URL to create your account: https://goo.gl/P9Y2FG

7.  You answer a test as «students»

8. What do students need to answer a test? • A mobile devicewithinternetaccess • For example: • Smartphone (iOS, Android) • iPad eller tablet PC • laptop • Theyuse a web browser to answer tests

9. Accessing the PeLe student page • Enter the following URL in a web browser: app.one2act.no

10. Logging in Remove this checkmark if the test is NOT anonymous 1. Click Join session 2. Enter the session code provided by the instructor 4. You can now answer the test which has been given to you on paper

11. Answering a test 1. All the questions on the same page, and can be answered in any order 2. Click the checkmarks to answer 3. Click again to remove a checkmark. You can change your mind as often as you like

12. Submitting the test 3. Type the word «yes» into the box at the bottom to confirm 4. The page looks like this once your test has been submitted 1. A submit icon will appear once all questions have been answered. Click this… 2. … and you get this overview of your answers

13. Question 1 Question2 Question3 According to a recentnewsstory, theprices for packagedelivery have increased by 100 %. If packagedeliveryoriginallycost 5 €, what is thecostaftertheincrease ? 6 € 7 € 8 € 10 € 20 € • A lumberjackwith a chainsawcancut a log intofourpieces in 12 seconds. • How manysecondsareneeded to cutthe log intofivepieces? • 1 • 4,4 • 15 • 16 • 17 • 18 • A bottlewith a cap has a productioncostof1,50 €. The bottleitselfcosts 1 € more thanthecork. • What is thecostofthecap? • 0,25 € • 0,50 € • 0,75 € • 1 €

14. Where in the process are we now? Review test with learning activities Setting up the test in PeLe Creating the multiple-choice test Monitor test

15. Designing multiple-choice test items • Open («constructedresponse») exercisescan be turnedinto multiple-choiceitems • Useoldexampapers, tests etc. to construct plausible distractors • AnalyzePeLe test itemsafterwards • Difficultylevel • «Discriminationindex»: doesthequestionclearly separate high-performing students from lower-performingones? • Didthedistractorswork as intended?

16. Create a multiple-choice test • Create a multiple-choice test with 4 – four – questions • Label the alternatives A, B, C etc. • You can create the test in any program you like (e.g. Word) • We will transfer the test to PeLe afterwards

17. Where in the process are we now? Review test with learning activities Setting up the test in PeLe Setting up the test in PeLe Creating the multiple-choice test Monitor test

18. Entering the test in PeLe 1. Start PeLe from the desktop or Start menu 3. Click here to start setting up a test from scratch 2. Log in with your credentials

19. Adding a test (1 of 4) Enter a title for the test Welcoming message students see when they log in Select whether tests can be answered anonymously

20. Adding a test (2 of 4) Number of questions Default number of alternatives (more can be added later)

21. Adding a test (3 av 4) Lets students select as many alternatives as they like «Bleeding»: negative scores are deducted from the total score (makes the test harder) Points for correct answer (Negative) points for incorrect answer Points for no answer

22. Save test to disk Adding a test (4 of 4) Add test Delete question Add question Add/remove alternatives Set correct alternatives by selecting checkmarks Remove checkmark to set customized scoring rules for a question

23. Adding/uploading the test 1. Click Upload when the test is ready. This will produce a session code (below) 2. The test will be added to the list on the front page – click My assessments to access it

24. Viewing session codes Session codes are not shown by default. Click here and select Code to add a column with this label (selection is remembed on next logon)

25. Starting a test Session codes are now shown in this column The test is not yet active. Click on it to access this menu…

26. Starting a test (cont.) Click here to start the test so that students can begin answering it

27. Students can now log in with your session code to answer your test

28. Running a test in the classroom • Before the test, you have printed the test questions on paper, or prepared an intelligible presentation for showing on a projector • You have generated the session code beforehand, and this can be printed on the paper to let students log in right away

29. Set up your test in PeLe • Set up the multiple-choice test you previously made in PeLe • Write down the session code once it’s been uploaded – you will give this to the person sitting next to you later

30. Where in the process are we now? Review test with learning activities Setting up the test in PeLe Creating the multiple-choice test Monitor test

31. Monitoring the test Real-time display of correct/incorrect answers on each question Individual view of each students’ performance on each question

32. Ending the test Click here to end the test (students can no longer submit answers)

33. Students can now take a short break… • … while you • Review class performance on each question (percentages + histograms) • Identify problem areas • Prepare review activities

34. Reviewing a test 1. Select questions for review 2. Click here to hide most of the PeLe interface («presentation mode»)

35. Reviewing a test in presenation mode Click buttons 1, 2 etc. to see percentages for question 1, 2 etc.

36. Showing percentage graphs to students Moveable toolbar Click Percentage to access the result graph for the selected question

37. Making scores available to students Click here to make test results available to students, who log on to their mobile devies to see their scores

38. Organizing finished tests 2. Tests can be cloned, deleted, archived etc. 1. Select a test from the list

39. Complete your neighbor’s test • Swap tests and session codes with your neighbour • Complete each other’s test, and use the teacher interface to monitor the test as it progresses • Close («stop») the test afterwards

40. Where in the process are we now? Review test with learning activities Setting up the test in PeLe Creating the multiple-choice test Monitor test

41. Different types offeecback • Verification («C wascorrectbecause…» etc.) • Elaborative explanation: • Teacher provides an explanation in full • Group discussion • Class discussion • Follow-up assignment • Measure depth of knowledge by asking a question similar to one of the test questions «peer learning»

42. Different learning activities for different distributions of answers • Question 1: largepercentageofincorrectanswers • Teacherelaboration(«this is whatweshould do») • Question 2 + 3: somecorrectanswers (30-70 %) • Group/class discussion • Question 4: mostly correct answers • Teacher verification («Alternative C wascorrectbecause…») • Follow-up question

43. Questions or comments? • Thank you for your attention!