Evaluating Academic Performance:

1. Evaluating Academic Performance: Richard HarkessProfessorPlant and Soil Sciences richard.harkess@msstate.edu 142 A Dorman Hall

2. It’s Time to Assess: Now What?

3. Good teachers assess students’ achievements. Expected Learning Outcomes reflect what you want the students to know or be able to do at the end of the course. (Should be Actionable and Measurable)

4. Assessment • Deciding what to assess • Deciding how to assess • Assignment / Test construction • Performance assessment • Grading • Cheating, appeals, attendance, reporting

5. Deciding What to Assess • What are the importantlearning outcomes of the course? • What skills will students need in this course and in subsequent courses? • Are some skills more frequently used? • Balance: You may not need to cover everything in the book.

6. …what to assess • Inform students of the course Expected Learning Outcomes. • Research uniformly shows students prefer to know expectations and performance is often enhanced. • How? • Syllabus • Beginning of each lecture • Review and study sessions

7. Deciding How to Assess • Issues that may affect your assessment tool choices: • Task authenticity • Time required (preparation, administration, evaluation) • Class size • Frequency (more is better) • Formative vs. summative assessment

8. …how to assess • Open book vs. closed book tests Open book tests more likely to: • reduce anxiety • reduce need to memorize • measure “higher order” skills (if test is properly constructed) • Take-home tests: Studies show students do not learn or retain as well • On-line testsSecurity, suitability, quality • Collaborative tests

9. Some Testing Issues • Frequency of testing Frequent testing is beneficial, especially for lower ability students Students often prefer a frequent testing schedule • “Pop” tests are generally not recommended Create greater anxiety in the classroom. Often viewed as busy work or as punishment. • Order of exam question difficulty: mixed findings. Studies show no impact Most recommend easy to hard sequence

10. Test Construction • Use publisher’s “test banks” with caution--they are often poorly constructed. • Show your tests to someone else before using. Be prepared to scrap/revise items. • A good assessment tool is one that: • Clearly communicates tasks to students. • Measures Expected Learning Outcomes. • Yields a reliable indication of learning.

11. Steps in Assessment Tool Construction • Identify primary objectives for the test or assignment. Should be related to the expected learning outcomes. • Prepare test specifications; item specifications. • Construct initial item pool. • Have items reviewed; revise as necessary. • Preliminary item tryouts; revise as necessary. • Develop guidelines for administration, scoring, interpretation of scores. • Post-test: Statistical analysis of items, scores; eliminate poor items.

12. Test Format Considerations • Constructed response • Short answer, completion, essay, performance tasks. • Often easier to construct • Slower response • Slower scoring • Good for “higher-order” skills • Selected response • True-false, multiple choice, matching • Quicker response • Quicker, more objective scoring • More challenging to measure “higher-order” cognitive skills

13. General Rules for Test Writing • Have others review your work. • Be prepared to revise items. • Include clear directions for examinees. • Avoid questions from direct quotations. • Avoid trick questions. • Avoid double negatives. • Have item’s value reflect importance, time spent. • Prepare the key as you write the assignment/exam.

14. Avoid easy or “give-away” questions(note: keyed answer indicated by asterisk) 1. The primary color of ripe oranges is a. orange. (*) b. green. c. yellow. d. brown.

15. Avoid questions with debatable answers* Who was the most important governor of Mississippi? What is the best thing about America’s economy? Who was the better poet, Wordsworth or Keats? *(Unless your intention is to measure debating skill.)

16. Evaluating Students’ Work/Answers • Remove names from your sight. • Prepare model answers or scoring guide in advance. Is partial credit possible? • After completion, re-read some early papers to assure consistent standards. • Take breaks while grading. • Read the same question for all; move on.

17. Why It Helps to Prepare Questions and Keys in Advance… Be precise in how you ask questions and know the answer you are looking for before giving the test.

18. …you know what I meant!

19. …and your problem is?

20. …I knew that!

21. Grading and Marking • Grades should represent performance only. • Objective vs. subjective or relative assessment. • To “curve” or not to curve?

22. Feedback • Plan first graded assessment and feedback prior to drop date (36th semester day); absolutely by midterm time. • Prompt feedback on performance is welcome and helpful. • Give constructive feedback whenever possible, not just a score or grade. • Do not post grades in public! (FERPA violation) • Recommend using Canvas to post assessment grades.

23. Grade Appeals Happen • Make sure your syllabus is clear on how grades are formulated; don’t deviate from your published policies. • Save your records: One regular semester. • Keep copies of tests and papers, if feasible. • Do offer to meet if a question about grades arises. Document your discussion(s). Know MSU policy.

24. Don’t forget these in the syllabus! • Assignments and due dates • Late assignment policy • Test dates; material covered • Make-up test policy • Cheating, plagiarism, honor code policy • Attendance policy • Participation policy • How course grades are generated.

25. Attendance • Attendance reporting required by MSU for undergraduates, students receiving certain financial support. Keep a record! • Can attendance be part of the course grade? Yes • MSU policy limits attendance to no more than 5% of course points. • Try using as incentive, not punishment. • Be clear about what comprises attendance. • Be aware of the University policy on University excused absences. You are required to allow work to be made up.

26. Reporting Grades • Progress grades are strongly recommended for undergraduates (required for Freshmen). Post on Banner system. • Post attendance for undergraduates. • Final grades are ordinarily due within a few days of the final exam date. Don’t be late! • Do not openly post student grades. On-line posting in Banner and Canvas is virtually immediate.

27. Resources for teaching • Faculty, former TAs. (*) • Instructor or publisher web sites (*) • Text publishers/representatives (*) Review copies, instructor resources, web sites • Measurement and evaluation texts • See the last two slides for specific URLs (*) Take with large grain of salt.

28. Other resources on grading/marking (from whence some of these ideas came) Article by Frisbie, D.A. & Waltman, K. K. (1992, Fall). An NCME Instructional Module on: Developing a Personal Grading Plan. Educational Measurement: Issues and Practice. See first URL: • http://depts.washington.edu/grading/plan/frisbie.html • http://depts.washington.edu/grading/plan/index.html • http://www1.umn.edu/ohr/teachlearn/resources/guides/grading/index.html • http://www.egr.msu.edu/~wolff/teaching/case_history_1.htm • http://www.evergreen.edu/washcenter/resources/acl/index.html • http://itc.boisestate.edu/tltr/disc02.htm • http://www.ust.hk/celt/ta/taguide/skills/dishonesty.htm • http://www.brookes.ac.uk/services/ocsd/firstwords/fw22.html • http://www.utdallas.edu/dept/graddean/ta_handbook.htm#efficiency

29. Resources on test construction • http://www.delweg.com/dpwessay/tests.htm (brief suggestions) • http://712educators.about.com/cs/assessment/a/assessments.htm (brief suggestions) • http://www.cte.cornell.edu/campus/teach/faculty/Materials/TestConstructionManual.pdf (very detailed, somewhat like a text chapter) • http://www.utexas.edu/academic/cte/sourcebook/tests.pdf (not quite as elaborate as previous link) • http://pareonline.net/genpare.asp?wh=1&abt=Test+Construction (excellent compendium of related topics)