gtPathways W-GTA Orientation: Grading and Responding to Undergraduate Writing

1. gtPathwaysW-GTA Orientation:Grading and Responding to Undergraduate Writing Sue Doe Assistant Professor of English Colorado State University Sue.Doe@colostate.edu

2. Overview of Orientation Part One • Introduction to the CSU gtPathways Writing Integration • A Sample Assignment—Overview, Initial Read, Holistic Criteria and Ranks PartTwo • Initial Grading of the Paper—Use Course-Provided Scoring Sheet • Develop and Apply Holistic Criteria • Analytic scoring—strategies and choices Part Three • Commenting/responding to student writing • Common problems students have with academic writing; grading and responding as part of the instructional team • Interventions through peer review, conferences, office hours • Resources: • International Student Writing • TILT Teaching Certificate and Other Resources • Writing Center

3. Part One • Introduction to the CSU gtPathways • Writing Integration • A Sample Assignment—Overview and Holistic Sorting

4. Local Writing Resources • http://writing.colostate.edu • Google search possible on virtually any writing topic. Over 100K pages of writing information, most authored at CSU. • Writing tools available through Writing Studio-keep track of your drafts, your biblios, your reading, etc. Same tools available for undergrads and others • http://writing.colostate.edu/gtPathways • specialized resources to support your efforts with your assistantship • The Writing Center and WAC • Visit Eddy 6 (The physical writing center) or submit papers electronically for feedback • Request a workshop on any writing subject

5. gtPathways What it is, where it came from

6. State Guaranteed Transfer: gtPathways • gtPathways Curriculum Adopted as part of the CCHE (now CDOE) Academic Affairs Policy I, Part L: Statewide Transfer Policy. • Built upon concepts found in the Student Bill of Rights (a.k.a, the King Bill), § 23-1-125 C.R.S: • “The Commission, in consultation with each Colorado public institution of higher education, is directed to outline a plan to implement a core course concept” • “The core of courses shall consist of at least thirty credit hours, but shall not exceed forty credit hours” • “Individual institutions of higher education shall conform their own core course requirements with the guidelines developed by the Commission…”

7. Major Changes to Colorado Colleges and Universities Adams State College: Faculty Senate agreed to adopt gtPathways curriculum for institutional general education curriculum Fort Lewis College: Restructuring entire general education curriculum to meet gtPathways requirements; modifying junior-level writing courses to meet gtPathways requirements (i.e., 200-level) University of Northern Colorado: Charting the Future; reducing general education course offerings to 60-70 courses; restructuring curriculum to meet gtPathwayscurriculum Colorado State University: Integrating writing into general education AHUM and SOCS courses (25% of grades in writing assignments); adding 3 credit hours in AHUM

8. One Policy Goal of gtPathways Students shall have assurance of: “A quality general education experience that develops competencies in reading, writing, mathematics, technology, and critical thinking through an integrated arts and science experience.”

10. What Matters in College Writing? • Write for a few minutes about • the qualities of writing that you believe all first-year college students should develop—essential abilities they’ll need • your beliefs about student writing ability right now and what that belief is based on • the kinds of support students need to improve • where, how, and when writing instruction should be given in college contexts • Put your name on this piece of paper as you’ll be turning it in. Bring to front table at the end of session today or at a break. • Discuss your beliefs with 2-3 neighboring people

11. Select Paper 1, 2, or 3 If your last name begins with A-H, take a copy of P1 If your last name begins with I-R, take a copy of P2 If your last name begins with S-Z, take a copy of P3

12. Intro to Anthropology Assignment Pick a question to write about: Option #1) Katherine went to Mali to address the issues of childhood malnutrition. A)What did she discover to be the primary causes? B) How did she come to these results? (i.e., What questions did she ask? What data did she collect?) Option #2) Malian life is very different from American life. What are some of the biggest differences Katherine encountered? Did Katherine embrace or resist these differences? Explain your answer.

13. Guidance About Source Use While you don’t need to formally cite my lectures, please use language such as “according to lecture” or “as discussed in class” to alert me to class-specific material. Please remember, any concepts taken from the textbook the ethnography, or RamCT must be cited with an in-text citation and bibliography entry. Examples of how to cite a chapter/article from the textbook: Labajo, J. (2003) Body and voice: the construction of gender in flamenco. In T. Magrini (Ed.) Music and gender: perspectives from the Mediterranean (pp 67.86. Chicago: University of Chicago Press Tollifson, J. (197).Imperfection is a beautiful thing: On disability and mediation. In K. Fries (Ed.), Staring back (pp. 105-112). New York: Plume.

14. Additional Instructions Write a 3-5 page response. Be sure to: • Provide a short introduction (3 sentences or so) • Include specific details from the ethnography • Incorporate vocabulary and concepts from lecture and the textbook. • Use 1-inch margins and 12-point font • Staple your pages together • Do NOT include a title page or title. Simply put your name and question number in the right-hand corner of the first page • Double space the paper • Follow APA citation and bibliography guidelines • Use proper in-text citation • Include a bibliography • Obtain information on APA format at http://writing.colostate.edu/guides/researchsources/documentation/apa/

15. Hierarchy of Rhetorical Concerns Audience, Purpose, Occasion Focus: Thesis, Reasons, Unity/Coherence Development: Reasons, Evidence, Explanation Style/Mechanics/Conventions: Readability, Care and Polish, Patterns of Error

16. Hierarchical Concerns Detailed Audience • Who is the writer’s audience? Is this an academic audience? What are the expectations? Purpose • Is this piece of writing intended to inform? Analyze? Explore? Summarize? Argue? Development • What kinds of evidence does the audience expect? Does the context demand clarification through examples, data, etc.?

17. Hierarchical Concerns continued Organization • Is the writing organized in a coherent way? • Do transitions guide the reader through the logic of the paper? Style and Conventions • What style is appropriate for the context in terms of audience and purpose? What register or level of formality Is appropriate? (For instance, can the writer use “I” in this context?) • Are there locations where the writing is hard to follow or comprehension is disrupted? If so, can I discern why? • Are there patterns of error showing any of the Top Five error patterns: 1) subject–verb agreement, 2) run-on and fragmented sentences, 3) unclear or incorrect pronoun agreement, 4) verb tense inconsistencies, 5) weak comma use

18. Holistic Process • Before you begin reading the sample papers, read the assignment information • In groups of three, do a “read-around” of the set of three papers you now have. Sort High, Medium, and Low. • As time allows, discuss the papers with your partners

19. High Medium and Low What are the standards you would apply to a minimally satisfactory (C-level) performance for this paper? Would the LOW paper meet these standards? Homework Review • In general, does the High Medium Low sorting method lend itself to a 6-point holistic scoring scale? • What parts, if any, from this scoring rubric would work with the essay assignment we’re examining?

20. During Break With Care, Read the “Rubric for Holistic Scoring of a Thesis-Restricted Paper.” Bring this rubric back to training tomorrow. Be prepared to discuss: • How does the High Medium Low sorting method lend itself to a 6-point holistic scoring scale? • What parts, if any, from this scoring guide would work with the essay assignment we are examining?

21. PARTTWO • Scoring and Grading Using Various Methods • Working for consistency—either faculty-led or GTA-led norming sessions • Grading consistency (for yourself across a stack of papers and also across a group of GTAs or faculty) is possible but isn’t easy or safely assumed to occur • Agreed-upon priorities are essential • Spot-checking by peers or others is desirable

22. Sort, Read, and Comment (or Stop, Drop, and Roll) Apply the sorting strategy for a set of papers. While this sounds like a time-consuming extra step, it actually saves you time in the long run. The Sorting Strategy • Sort into three stacks—high, medium, low • If possible, stack within categories (High + and High -) so that you have 6 stacks • Read with hierarchy of concerns in mind • Provide an end comment that is forward-looking and focused • Substantiate end comment with a few marginal comments

23. Grading—Becoming Part of the Instructional Team • Consistency and fairness • Criteria-based grading vs. norming • Time management through Hierarchies of Rhetorical Concern • Holistic and Analytic Evaluation • Grading and Responding—Two Tass

24. GRADING Remember: you are only assigning a grade; students earn those grades. You do not GIVE grades. They do not GET grades. Consider using a 24-hour moratorium and a conference appointment system for grade protests Ask Your Faculty Supervisor: Will there be calibration or norming sessions to identify standards and/or achieve reliability? Are you allowed to return a paper ungraded in the case of careless or unacceptable work with a 24-hour window of opportunity before default to F? Are revisions allowed? If so, what are the processes?

25. Grading For What Matters—Purposes of Assignments What is the TASK being required by the assignment—to inform, to explore, to convince, to describe, to compare, to summarize, to persuade? Find the VERB or VERBS and you’ll know the task. Is this • a thesis-provided paper for which students must defend of refute? • a problem-solution paper in which students are given a problem or question that demands a thesis and support? Is • a data-provided paper for which students are expected to analyze and explain? • a genre-provided paper, in which students are expected to follow an organizational structure or format in an accepted form, such as a memo, case study, lab report, or executive summary? • write-to-learn or write-to-engage writing for which students are expected to explore and/or develop their thinking rather than to produce a polished paper? • an in-class essay, reflecting comprehension of course material?

26. Intro to Anthropology Assignment Pick a question to write about: Option #1) Katherine went to Mali to address the issues of childhood malnutrition. A)What did she discover to be the primary causes? B) How did she come to these results? (i.e., What questions did she ask? What data did she collect?) Option #2) Malian life is very different from American life. What are some of the biggest differences Katherine encountered? Did Katherine embrace or resist these differences? Explain your answer.

27. Grading Rubric

28. Holistic Grading Ranking 1-6 Think of 3 as minimally acceptable—aims of assignment are being met but only marginally. “Gestures” toward sound approaches are there, but student will need a great deal of assistance. This is a student who might be considered “at-risk” but also offers a tremendous opportunity for development through GTA impact.

29. Use scoring tools to assist with grading Consider Three Approaches benchmark and anchor paper approach traditional analytic rubric with dimensions continuum approach

30. Approach #1: Anchor Papers Consider writing an evaluation paragraph or narrative that explains what’s necessary to earn a C paper on this assignment. In other words, what MUST a paper accomplish to be deemed “adequate” and to exceed this lowest, acceptable standard for this assignment? --The C paper has a clear thesis or focus, shows a satisfactory degree of development /support of points, and is reasonably easy to read/follow --The B paper does everything the C paper does but goes further to provide deeper development of points, a more satisfactory selection of evidence, a coherent structure/organization, and a more compelling set of insights --The A paper does everything the B paper does but goes further to provide a more unified, fully developed, and polished paper that is a pleasure to read because it offers good insights that are expressed well It can be useful to distribute or post this explanation

31. Approach #2: Traditional Rubric Component Parts • Assignment itself • Dimensions/priorities/criteria • Scale with levels of achievement. Levels can be continuums or reflect categories such as “proficient,” “competent,” “needs work.” Can associate levels with points* • Space for specific comments * Be careful to not create a checklist effect. Remember that meaningful quality indicators will be indicated so the simple presence or absence of a feature is insuficient.

32. Traditional Rubric with both Holistic and Analytic FeaturesDimension Excellent Competent Needs Work Overall Comment: Grade: ___

33. Course-Provided Grading Rubric

34. Steps for Creating Traditional Rubrics • List key elements/features to assess, based on course and assignment objectives • Refine and simplify key elements, then consider their relative importance or weight • Place most important dimension at the top • Do a common sense check to see if weighting of criteria is meaningful and rational. If possible, avoid points. Percentages are better but keep them broad. Too much refinement of points can lead to “grade-grubbing.” • Decide where you will comment--on the rubric or on the paper itself? Commenting itself is not optional. • Decide if you’ll give feedback/comment on all criteria or only on certain ones • Make clear where the overall grade appears

35. PointsApproach—Advantages/Disadvantages?

36. Approach #3: The Continuum Approach Once you have determined the most important aspects or criteria for grading, consider using a continuum to describe where the student is in their application of this criteria. This avoids the oft-times awkward approach of assigning points or percentages with criteria-based evaluation. Example (criteria 3) from the Washington State U “Critical Thinking Guide”: Identifies and considers salient perspectives and positions important to the issue’s analysis Scant Substantial ----------------------------------------------------------------

37. Likely Grading Criteria • Clarity of points and coherence of organization • clearly state the purpose of your essay • answer all questions related to the ethnography • accurately define causes of malnutrition or cultural differences • explain methods of inquiry/discovery and explain whether differences are embraced or resisted 2) Quality/depth of analysis. You need to make your position on each development point clear. Provide arguments that are supported by information (i.e., evidence from text, lecture & ethnography). Meet or exceed the page requirements. MOU seeks synthesis! 3) Quality of writing. Your ideas need to be clearly expressed. This includes proper spelling, grammar, expression of ideas, and citation of sources

38. Discussion • What strengths and weaknesses of scoring approaches do you see here? • Would you combine strategies from the three approaches?

39. PART3—Commenting & Responding • We rehearse “commenting” with HINT (How to Improve Next Time) in mind • We consider our role as coach, as teacher, as mentor, and role model, as writer. We are part of the INSTRUCTIONAL TEAM. • We try out various kinds of comments—end comments & opening comments; marginal substantiation of the end comment; marginal questions; marginal reader-response comments, marginal queries about substance; positive reinforcement of good insights or writerly choices; error pattern comments

40. Write to Engage • What was the most helpful feedback you’ve ever gotten on a piece of writing? • What made it helpful? • What was the least helpful feedback you’ve ever gotten on a piece of writing? • What made it less than helpful? • As you hear others’ experiences, what generalization(s) can you draw?

41. Managing Your Time Through a 3-Part End Comment • Sum up the strengths of the paper • Identify the main problems to be worked on • Provide a specific suggestion for how to improve the paper, based on the main problem(s) already identified And Remember: • You can’t respond to everything in a paper. • There are real people on the receiving end. • Comments are not principally for “justifying” a grade. Your are providing formative feedback students can use with the next paper, even if it’s not in this class. Remember: HINT • Consider using questions and other response approaches besides “correction” in your marginal comments.

42. Comment on the sample paper • Now try the 3-part end comment with your sample paper • Find locations in the paper that substantiate your observations in the end comments and add marginal comments here • Ask an important question in the margin at a location where the writer is on to something but not quite there • Find a location where you can observe something positive the writing is doing or trying to do

43. Reviewing Marginal Comment Options • Substantiate the end comment by providing an exact location where the paper problem occurs • Identify KEY points that are missing and focus on these learning needs • Ask a question that will get the student to think further • Provide advice on locations to add evidence or choose different evidence • Offer a “reader-response” comment that shows you are engaged and interested as a reader • Point out any location where the reader is making a good decision in his or her writing • Identify one of the Top 5 mechanical errors that the student demonstrates

44. Opening Comment vs. End Comment End comments are often not read because the student is focused on the grade. Try writing the “end” comment at the top of the paper to direct the student’s reading of your marginal comments and to briefly put off the inevitable turn to the back of the paper for the grade.

45. Basic Principles of Commenting-Also Covered in Grade Reviews • Your goal is formative, as in “Next time…” • Focus on the most important advice a writer needs at this time. • Precede your commenting with holistic sorting and keep your comments consistently focused • Vary your comment types in the margins rather than only observing deficiencies • Play the believing game and find a positive feature • Use a 3-part end comment at first but know how predictable it is • Show how weaknesses relate to one another • Make sure the grade and evaluation criteria are connected and accurately reflected in your comments . . . Do a common sense check

46. Avoid the Grammar Trap • Students’ writing usually contains fewer mistakes than instructors and graders perceive. Errors are nearly always “patterned” rather than discreet • Students often have more linguistic competence than the surface features of their writing suggests • Errors in student writing increase in direct correspondence to increasing difficulty of the assignment • Errors often disappear in students’ writing as they progress through multiple drafts • You can expect to see more serious sentence problems in first drafts and on essay exams

47. Avoiding the Grammar Trap • Not all errors are equal! • Don’t get into the habit of correcting/marking student errors. It’s NOT helpful and it’s a poor use of your time. Mark ONE paragraph or identify a pattern and say, “The problem continues” and explain what the problem or error pattern is. Perhaps it’s not grammar at all but “academic voice”? • Traditional procedures for marking student papers may make matters worse. Save your students! • Traditional procedures for marking student papers may make you sour and ineffective. Save yourself! • Remember that once you start responding, you are the WRITER and the student is the READER.

50. Peer Review of Comments • Identify the major strength your partner noted in this paper. What locations did the GTA point out to substantiate this claim of strength? How accurate do you believe this evaluation is? • Identify the guidance or advice your partner noted as a central concern in this paper. What locations did your partner identify to substantiate the claim of “needs improvement” How accurate do you believe this evaluation is? • Identify the concrete suggestion for improvement that your partner noted. Would an undergraduate understand this advice and be able to follow it? How accurate do you believe this advice is? • Characterize the tone/attitude of feedback your partner has provided. Could it be improved and if so, how? • Are your partner’s comments forward-looking and formative in nature or do the comments seem defensive, as if justifying the grade? Has the grader used the HINT advice (how to improve next time)? HOW MIGHT YOU USE PEER REVIEW (IN-CLASS OR ELECTRONIC) TO INTERVENE IN STUDENTS’ WRITING PROCESSES? DISCUSS IDEAS WITH YOUR FACULTY SUPERVISOR