Download
1 / 32
320 likes | 509 Views
Introduction. ?Assessment is the most political of all the educational processes: it is the area where issues of power are most at stake. If there is no staff-student collaboration on assessment, then staff exert a stranglehold that inhibits the development of collaboration with respect to all other
E N D
1. Assessment and Collaborative Learning in Thesis Rings
A.G.L. Romme
Maastricht University
Netherlands
s.romme@mw.unimaas.nl
2. Introduction “Assessment is the most political of all the educational processes: it is the area where issues of power are most at stake. If there is no staff-student collaboration on assessment, then staff exert a stranglehold that inhibits the development of collaboration with respect to all other processes. Once varying mixtures of self, peer and collaborative assessment replace unilateral assessment by staff, a completely new educational climate can be created” (Heron, 1981, p. 63).
3. Introduction We have an enormous pool of experiences in supervising students writing a thesis, but there is hardly any research in this area
Goal of project: develop actionable knowledge about collaborative approaches to supervision
Goal of this workshop: explore how our findings can be used to help you (re)construct teaching and assessment systems
4. Background Problem Based Learning in small tutorial groups in all undergraduate courses at Maastricht University
In contrast to: individual supervision practice for masters thesis projects
Two generic problems in thesis supervision: few economies of scale (synergies between students) and highly vulnerable relationship between supervisor and student
5. Background: Social Engineering
(Re)design and development of social systems in order to achieve intended outcomes which persist over time
Our ideas, together with the physical and social structures we are in, shape our behaviors
If structures become obsolete and/or ideas are outdated, behavior will fail to produce the intended outcomes
Focus on leverage points, where small inputs can make major differences
6. Limited vs Effective Learning Systems Deeply ingrained repertoires of action that inhibit / stimulate learning in groups (see: Argyris & Schön, and others)
These repertoires can be described in terms of the core values, behavioral strategies and consequences of limited versus effective learning systems
7. Limited Learning System Core values: a) define goals and try to achieve them b) win, do not lose c) suppress negative feelings d) emphasize rationality
Behavioral strategy: unilateral control over others
Consequences: defensive interpersonal and group relationships, low freedom of choice, reduced production of valid information, etc.
8. Effective Learning System Core values: a) valid information b) free and informed choice c) internal commitment
Behavioral strategy: shared control
Consequences: information sharing which facilitates public testing; participants define own objectives and methods; absence of coercion or manipulation; responsibility for decisions, etc.
9. Key Questions How can we create an effective learning system for supervising thesis students?
How to exploit the potential synergy between students doing a thesis? And the potential contribution of students as (junior) supervisors?
What about the formal / informal authority and power of professors?
10. Preliminary Idea The idea of circularity of power: shared leadership and control over the process and outcomes of supervision
In April 1996 a group of seven students and one professor started a “thesis ring” for supervising thesis projects
This first ring was highly experimental; it developed its own procedures and rules (within the existing legal framework)
11. First Experiment First thesis ring produced (unexpected) high level of internal commitment, better quality of masters’ theses, and active learning about supervisor-student relationship
Most impressive was the time and energy students are willing to spent on giving feedback to other students
Gradually, the results of first thesis ring inspired other professors to start their own rings
12. Results and Experiences Maastricht University now has six rings: Organization, Market Research, Strategy, Info Systems, Info Management, and Systems Thinking
The professors involved are Dutch, German or American; students involved come from different countries (but mainly from NL and Germany)
In addition, experiences have started to feed back to undergraduate curriculum: four courses have been redesigned, using some of the principles developed in thesis rings
13. Results and Experiences In general we found: if professors give students role of apprentice (co-supervisor) and acknowledge the values and principles of how effective learning systems operate, they can create supervision and learning systems that are much more efficient and effective
14. “Master-apprentice-student” Model
15. “Master-apprentice-student” Model
16. “Master-apprentice-student” Model
17. Students Switch Roles Students continually switch between their role as co-supervisor and as thesis student (at least once every meeting)
As thesis writers, they learn from each other and are less likely to become insulated and feel “left alone”
As co-supervisor, students learn how to diagnose problems and coach other people, which also supports their own thesis process
As co-supervisor, they also “translate” and extend the feedback given by professors, and vice versa; thus, the feedback becomes more effective
18. Additional Advantages for Professors The chair and secretary of the ring are selected from the student members, so the academic supervisor can concentrate on giving feedback
For the academic supervisor, the thesis ring can produce economies of scale in reading drafts and talking to students: major econo-mies (30-50 % decrease in time spent) arise when the professor acknowledges and exploits the co-supervisor role
19. Shared Control Power issue must be explicitly dealt with: solution was to decide by consent (“no argued objection”) on a certain domain of decisions
Thus, the chairperson, secretary and formal arrangements are selected / decided by consent
Evaluation and grading of final thesis (internal quality control) is done by consent
Second assessor tends to participate in the meeting where thesis is graded, but can also give independent evaluation outside the meeting (external quality control)
20. Assessment procedure Assessment procedure was developed in first experimental year with following conditions and guidelines in mind:
Conditions:
Assessment of final version of thesis is done by the staff supervisor, co-supervisors and at least one external assessor who comes to the meeting for that particular purpose
All assessors – supervisors and co-supervisors – prepare the assessment adequately; the secretary makes sure all information is available
The assessment (incl. grading) is done in the presence of the assessee (author of thesis)
21. Assessment procedure Guidelines:
Interdependency between assessors is guaranteed by consent rule
The assessment process should start as open as possible; that is, interaction between initial grades & arguments is avoided as much as possible
Subsequently, argumentation and dialogue should serve to increase interaction in order to come to a final grade that is well-argued as well as broadly accepted by supervisors and co-supervisors
The grade given should be internally and externally legitimate
22. Assignment: Divide into teams of 3 or 4 participants
Decide who is going to chair, and who is going to report the findings to the group
Each team should design / describe a procedure which fits these conditions & guidelines as much as possible
If more than one procedure emerges in the discussion, decide by consent which procedure is preferred (in view of the guidelines given)
23. Steps in the procedure Before the meeting in which the assessment takes place, all participants are informed in writing about the assessment; they all receive a copy of the thesis at least one week before the meeting. (Each participant has access to the ring’s regulations, which includes an outline of this procedure and a check-list with assessment criteria)
The assessment process at the meeting starts with a short introduction & reflection on the thesis by the author. The author, so to say, hands over the thesis to the assessors, and remains silent in the subsequent assessment process
24. Steps in the procedure Subsequently, each assessor is asked to write his/her own name and proposed grade (quanti-tative assessment) on a piece of paper: “I, Paul propose a 7.” All proposals are handed in with the chair, who then asks each participant to state his arguments for the proposed grade (qualitative assessment). “Paul, you propose a 7 for Maria’s thesis, please motivate this proposal?”. The chair does not allow any discussion when each assessor explains his/her proposal. This step reduces the initial interdependency among individual assessments to a minimum, and creates openness about individual opinions & assessments
25. Steps in the procedure The chair then goes around the assessors again, now asking whether anyone would like to change his/her proposal in view of the arguments heard in the previous round. (Typically, most participants stick to their initial proposal, but some change their proposed grade, for example, because “having heard John’s arguments on the readability of this thesis, I change my proposal from 7 to 6”. This step is the first one where interaction between arguments and proposals is deliberately allowed; any discussion is led by the chair in order to make it as open and visible as possible
26. Steps in the procedure An open, relatively unstructured discussion may develop, in which arguments are tested, questioned, clarified, compared, ranked, etc. This step is often skipped, particularly when the chair feels the arguments and proposals are converging to a large extent. (In that case, the chair moves on to the next step.)
27. Steps in the procedure At some point, the chair proposes a certain grade, referring to the main arguments raised. The chair will go around the group to ask each assessor to give consent to this proposed grade. At this stage, one or more assessors may withhold their consent. Depending on any (additional) arguments, the chair can adapt the arguments for the same proposal, until each assessor gives consent to the proposal. If the objections of one or more assessors are paramount, the chair moves to next step.
28. Steps in the procedure If at least one assessor has paramount objections to the proposed grade, the chair
first asks each assessor who proposed this particular grade to change this grade;
And also gives other assessors opportunity to change proposals
This step gives chair information about which direction to choose:
@ if proposals are diverging, then discussion about interpretation and application of criteria is necessary
@ if proposals are converging to a certain grade, then propose this grade
This process continues until all assessors agree on a grade
29. Main problem undermining collaboration in thesis ring If the professor is not able and/or willing to acknowledge and exploit the apprentice role of co-supervisors, the thesis ring becomes both inefficient and ineffective
30. Major Misconceptions: “I am the only expert and authority”: students are not competent to supervise or assess a thesis
Power shared with students as co-supervisors undermines the authority of the professor
“Stick to your opinion”: changing your assessment is a sign of weakness
31. Constructive Conceptions Knowledge is a social construct and learning is a reacculturative process that helps students become members of knowledge communities different from those they already belong to
Coercive power undermines the supervising professor’s identity as being a trustworthy and credible expert
“It’s all about dialogue”: intersubjectivity is the key value of the academic community
32. Assessment and collaborative learning in thesis supervision
