Portfolios, Rubrics, Learning Outcomes…..oh my!

1. Portfolios, Rubrics, Learning Outcomes…..oh my! Session Five The Cordon Bleu Barbara Packer-Muti, EdD

2. A Performance Assessment • Requires a student to perform a task or generate his/ her own response.  For example, a performance assessment in writing would require a student to actually write something, rather than simply answering some multiple-choice questions on grammar or punctuation.

3. Performance Assessment =Task + Rubric • The task may be a product, performance or extended written response to a question that requires the student to apply critical thinking skills.  Some examples of performance assessment tasks include: • written compositions, speeches, works of art, science fair projects, research projects, musical performances, open-ended math problems, and analysis and interpretation of a story the student has read.

4. Why are rubrics important? • Because a performance assessment does not have an answer key in the sense that a multiple choice test does, scoring a performance assessment necessarily involves making some subjective judgments about the quality of a student's work. 

5. 2 Teachers = same score! • The rubric should organize and clarify the scoring criteria well enough so that two teachers who apply the rubric to a student's work will generally arrive at the same score.  The degree of agreement between the scores assigned by two independent scorers is a measure of the reliability of an assessment.

6. An excellent scoring rubric will: • Help instructors define excellence and plan how to help students achieve it. • Communicate to students what constitutes excellence and how to self-evaluate. • Communicate goals and results to others. • Help instructors to be accurate, unbiased and consistent in scoring. • Document the procedures used in making important judgments about students.   Herman, Aschbacher, and Winters (1992)

7. Elements of a good rubric: • One or more traits or dimensions that serve as the basis for judging the student response • Definitions and examples to clarify the meaning of each trait or dimension • A scale of values on which to rate each dimension • Standards of excellence for specified performance levels accompanied by models or examples of each level   Herman, Aschbacher, and Winters (1992)

8. Qualitative rubrics ratings scales SCALE POINT EXAMPLES: • Not yet, developing, achieving • Emerging, developing, achieving • Novice, apprentice, proficient, distinguished • No evidence, minimal evidence, partial evidence, complete evidence

9. Qualitative/Quantitative scale points • 1. Unable to begin effectively • 2. Begins, but fails to complete problem • 3. Serious flaws but nearly satisfactory • 4. Minor flaws but satisfactory • 5. Competent response • 6. Exemplary response

10. How many points should you have? • Each point on the scale needs to be well defined.  • Longer scales make it harder to get agreement among scorers (inter-rater reliability). • Extremely short scales make it difficult to identify small differences between students. • Do you simply want to divide students into two or three groups, based on whether they have attained or exceeded the standard for an outcome?  (Short scale) • If you are rating a product/performance on several different dimensions, will you want to add up the scores so that each is equally weighted?  (same length)

11. Analytic vs. holistic • A rubric with two or more separate scales is called an analytical rubric.  This contrasts with a scoring rubric that uses only a single scale that yields a global or holistic rating.  Scoring rubrics may actually be a hybrid of the two types, with instructors assigning an overall, or integration score to each composition. 

12. General vs. specific: Scoring rubrics may be specific to a particular assignment or they may be general enough to apply to many different assignments.  The more general rubrics prove to be most useful, since they eliminate the need for constant adaptation to particular assignments and because they provide an enduring vision of quality work that can guide both students & instructors.

13. Examples of writing rubrics http://intranet.cps.k12.il.us/Assessments/Ideas_and_Rubrics/Rubric_Bank/WritingRubrics.pdf

14. Oral presentation rubrics http://tcet.unt.edu/START/instruct/general/oral.htm http://rubistar.4teachers.org/index.php?screen=ShowRubric&module=Rubistar&rubric_id=1128816&

15. Group presentation rubrics http://www.tracy.k12.ca.us/thsadvbio/files/Rubric%20for%20Group%20Pres%20w%20point%20explain.doc http://academics.smcvt.edu/cbauer-ramazani/BU113/present_rubric2_team.htm

16. Portfolios! • Collections of student work representing a selection of performance. • Derived from the visual and performing arts tradition in which they serve to showcase artists' accomplishments and personally favored works. • May be a folder containing a student's best pieces and the student's evaluation of the strengths and weaknesses of the pieces. • May also contain one or more works-in-progress that illustrate the creation of a product.

17. Why Try Portfolios? • Students have been stuffing assignments in notebooks and folders for years, so what's so new and exciting about portfolios? Portfolios capitalize on students' natural tendency to save work and become an effective way to get them to take a second look and think about how they could improve future work. • This method is a clear departure from the old write, hand in, and forget mentality, where first drafts were considered final products.

18. The Portfolio Content • Built from class assignments and as such corresponds to the local curriculum. • Students may develop portfolios focused on a single curricular area • or they may develop portfolio programs that span two or more subjects • Still others span several course areas for particular groups of students.

19. Three basic portfolio models • Showcase model, consisting of work samples chosen by the student. • Descriptive model, consisting of representative work of the student, with no attempt at evaluation. • Evaluative model, consisting of representative products that have been evaluated by criteria. Grosvenor (1993, pp. 14-15)

20. What does the research say? • Research shows that students at all levels see assessment as something that is done to them on their classwork by someone else. Beyond "percent correct," assigned letter grades, and grammatical or arithmetic errors, many students have little knowledge of what is involved in evaluating their classwork. • Portfolios can provide structure for involving students in developing and understanding criteria for good efforts, in coming to see the criteria as their own, and in applying the criteria to their own and other students' work.

21. What are the drawbacks? • Considerable effort by and time demands on teachers/instructors. • A thorough understanding of their subject area and instructional skills, but also additional time for planning, conferring with other teachers, developing strategies and materials, meeting with individual students and small groups, and reviewing and commenting on student work. • Extra space in classrooms to store students' portfolios or expensive equipment such as video cameras.

22. Learning Outcomes, revisited Characteristics of good learning outcomes? • The specified action by the learners must be observable. • The specified action by the learners must be measurable. • The specified action must be done by the learners.

23. Can we assess it? The ultimate test when writing a learning outcome is whether or not the action taken by the participants can be assessed. If not, the outcome probably does net meet all three of the characteristics.

24. Examples of poor learning outcomes: • Participants will understand the nine reasons for conducting a needs assessment. • Participants will develop an appreciation of cultural diversity in the workplace.

25. Action verbs resulting in observable, measurable behavior: • compile, create, plan, revise, analyze, design, select, utilize, apply, demonstrate, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique

26. Unclear action verbs without measurable behaviors • know • become aware of • appreciate • learn • understand • become familiar with

27. Syllabi Learning Outcomes • Use formulas based on bakers’ percentages. • Understand the characteristics and functions of the major baking ingredients. • Describe twelve basic steps in the production of yeast goods. • Understand and control the factors affecting dough fermentation • Prepare natural starters and yeast starters, and mix sourdoughs using them to produce whole wheat and rye breads. • Prepare a variety of breads and rolls using lean doughs. • Prepare a variety of specialty bread items such as English muffins, pretzels, and bagels. • Make a variety of products using sweet doughs and rolled-in doughs for sweet breakfast and dinner crescent rolls. • Understand the leavening process for baking powder biscuits, muffins, loaf breads, corn breads and popovers.

28. Another syllabus…. • Use percentages to compare expenses to profits, and actual results to budgeted goals. • Discuss the need for good inventory control, and methods for determining inventory levels • Use standardized recipes to control food and beverage costs and quality. • List and describe aspects of the receiving and purchasing function for both food and beverages.  • Explain how to compute actual food and beverage expenses and estimate daily cost of food or beverage consumed. • List the component of labor-related expenses. • Manage payroll costs using productivity standards, sales volume, employee schedules, and analysis of results. • Explain the difference between fixed and variable expenses, and controllable and non-controllable expenses. • Describe the use of menu analysis. • Use operating results to establish future budgetary goals.

29. Another one…. ·        Explain the development of modern foodservice including the use of modern equipment and technology.·        Identify the major historic Chefs and the contributions they made to the culinary profession. ·        Identify the major stations in a classical kitchen.·        Name the most important components of foods and describe what happens to them when they are cooked.·        Describe each of the basic cooking methods.·        Understand and explain protein dishes using a variety of cooking methods.·        Explain the preparation and cooking of eggs.·        Explain the preparation and cooking of starches, potatoes, pasta, legumes and vegetables. ·        Demonstrate how to properly handle and sharpen knives.·        Demonstrate proper knife skills and Classical Cuts·        Understand, explain and demonstrate Stocks and Fonds.·        Understand and explain the Mother Sauces of Classical French Cuisine. ·        Understand the practical use of the major French Cooking Terms and their meaning.

30. Another set of learning outcomes… • Apply the techniques of deep fry, stir fry, steam, steam fry, sauté, stew, braise, pan sear, gratin, grill & poach to a variety of proteins, starches, vegetables and legume • Prepare and illustrate principles of sushi and sashimi • Describe geographical and historical influences on international foods and cuisines • Label, identify and differentiate name brand and staple products, equipment, and utensils indigenous to specific international region • Recognize terminology specific to a variety of international regions • Apply the technique of stuffing to specific doughs and proteins • Prepare soups specific to different international cuisine • Explain and demonstrate fundamentals of plating prepared food • Develop and plan theoretical menus • Analyze and extend the cost of select recipes • Illustrate a variety of knife cuts.