Two countries separated by a common language? – Assessment, clinic and UK higher education

1. Two countries separated by a common language? – Assessment, clinic and UK higher education Philip Plowden Kevin Kerrigan www.northumbrialawschool.co.uk

2. PP Intro

3. Some assessment issues in clinic • Fear that assessment tends to be subjective / arbitrary • Concern about the relevance of the supervision process in assessment • Impact of the supervisor / student relationship on the grade • Is it possible to have a fair assessment based on very different clinical experience (clients, area of law, volume of work)? • How can consistency be encouraged between clinical faculty with different professional backgrounds and pedagogic attitudes? • Effectiveness of internal and external moderation in relation to work they have not supervised

4. Looking at an example of assessment

5. Context: Assessment in Northumbria’s clinic – Student Law Office 70% = practical work done during the year, as evidenced by student personal file portfolio. 30% = three pieces of reflection on student work, handed in as part of the portfolio. Skills competency assessments in (i) Interviewing/advising and (ii) Legal writing = pass / fail.

6. Interview competency – students should be able to: • prepare for the interview • deal with appropriate client care and professional conduct issues • allow the client to explain his or her concerns • identify the client’s goals; help the client reach priorities • elicit relevant information • use appropriate listening and questioning techniques • determine what further information is required • Identify courses of action and the legal and non-legal consequences • assist the client to make a decision regarding the best course of action, including the costs, benefits and risks of that course of action • agree the future action to be taken by both parties • accurately record the interview, confirm instructions and action • establish a professional relationship with the client

7. The video – what we want you to do • Please use the competency assessment criteria to decide whether this interview is likely to be a pass or a fail • Identify any particular strengths or weaknesses in this performance that would influence your view of its quality • Identify any further information you think you need to make you more confident in the reliability of your assessment exercise

8. Competency CriteriaPass or fail? Prepare Client care Client concerns Client goals Relevant info Listening; questioning Further info Courses of action Future action Record / confirm Professional relationship

9. The value and limits of competency assessment Value • Quick • Simple yes or no • Easy to complete • Confirms student can perform basic skills • Helpful for quality control of the legal profession – ensure minimum standards Limits • Only provides a snapshot • Fails to address student development • Crude measurement • Fails to distinguish student performance • Fails to consider lawyering ability holistically

10. The development of grade descriptors • The key aims of the descriptors are • (i) consistency - help ensure more consistency in the marking process by having a clear set of standards distinguishing different levels of student performance and • (ii) transparency - assist students to understand how they can improve their performance and therefore enhance their grade in clinic

11. What are our grade descriptors? • They describe different levels of student performances by the end of the programme: fail, lower second, upper second, first class and strong first class • The UK Quality Assurance Agency Benchmark for Law: • (i) Autonomy and • (ii) Ability to Learn. • the “key feature of graduateness”.

12. Autonomy and ability to learn Autonomy includes •  ability to identify and apply law with accuracy • ability to plan and progress client cases • professional commitment to the client’s best interests; • efficiency in managing the case. Ability to learn includes • ability to recognise areas of weakness and to improve • ability to build on existing areas of strength • ability to reflect effectively and relate experiences to wider academic, professional and societal contexts.

13. Key descriptor 1st class A student who is able to identify legal issues with accuracy and then to implement effective strategies for achieving the client’s goals. Such students may make errors of law, or may demonstrate initial weaknesses in their legal skills, but will show the ability to learn from experience so that their work is of an increasingly high standard. The supervisor will be able to have an increasing confidence in the student’s ability to operate with a higher degree of autonomy than might normally be expected.

14. Extracts from detailed descriptor • “This student requires little active prompting and can be expected to approach the supervisor with suggestions that have been thought through and are often highly developed.” • “The student is able to identify relevant legal principles with accuracy, and where appropriate will show the ability to think across legal “areas” so as to view the case holistically.” • “There will be a high level of commitment to the client and empathy with their situation.” • “The student’s written reflections demonstrate the ability to assess both the cases that they have conducted and their own performance in terms of wider academic, professional and socio-economic contexts.”

15. Key descriptor - fail • This student requires close supervision even towards the later stages of the SLO. This may be because of significant shortcomings in the student’s ability to identify the relevant law with accuracy. Additionally or alternatively they may be unable to apply that law effectively and in a manner that is likely to progress the client’s case. Additionally or alternatively such students will demonstrate little commitment to the client’s best interests, or may show such poor time management skills that any attempt to progress their cases efficiently is undermined. Similarly the student’s failure to comply with the administrative requirements of the Law Office may mean that they cannot be trusted to conduct any work without continuous oversight from the supervisor.

16. Relevance of the grade descriptors to the video interview? • The student’s interview would be assessed for competency • It may form part of the student’s mandatory written skills reflection • It would be part of the evidence base for the application of the grade descriptors at the end of the year – e.g. Opportunity to show accurate legal analysis / advice, ability to think across legal ‘areas’ (landlord and tenant law into homeless rights), empathy, commitment to client, strategic thinking, professional conduct etc.

17. Observations • Grading clinic clearly presents challenges • If we are going to grade clinic we need some way of allaying the fears of faculty and students that we grade in an arbitrary or unreliable fashion • The grade descriptor approach is a potentially useful mechanism for helping faculty and students understand the overall “feel” of student performance at different grade boundaries • It should help us make fair decisions between our own students and encourage consistency between clinical faculty • Feedback thus far has been positive from faculty and from students

18. Issues with descriptors • The descriptors need to be sufficiently detailed to cover the range of learning outcomes we seek for clinical students but sufficiently simple to be readily understandable by students and helpful for faculty members when it comes to the marking process. This is a difficult balance to strike. • Having the descriptors does not guarantee that they will be used. There is a need for an ongoing dialogue about expectations.

19. The labyrinthine UK assessment context • assessment tutor and team devise draft assessment • submit to subject specialist for comment and approval • submit to scrutiny committee for comment and approval • submit to external examiner from another institution for comment and approval • students complete assessment • marking meeting with team – agree a marking scheme • marking by faculty • internal review / moderation process • external examiner moderation / approval • examination board approval

20. Clinicians aren’t the only law teachers with assessment issues • “An examination of the incidence of ‘error variation’ in the grading of law assessments” Hanlon et al 2004 www.ukcle.ac.uk • "To give the most extreme example - one student's paper was given a mark of just 13 by one examiner and 56 by another.” Times Higher Education Supplement, 9th January 2004.

21. Information • Contacting us: • Kevin Kerrigan: kevin.kerrigan@northumbria.ac.uk • Philip Plowden: philip.plowden@northumbria.ac.uk • Finding the Northumbria grade descriptors: via www.studentlawoffice.co.uk • Visiting us: visitors very welcome - we’re in Newcastle upon Tyne, in the North of England, near the Scottish border.