For Factor 6: Analysis of Assessment Procedures

1. For Factor 6: Analysis of Assessment Procedures TWS Aid

2. MajorSection Requirements: • Graph comparing pre and post assessments • Worksheet showing Learning Gain Scores (LGS) for each student • Learning Achievement Table depicting % of students achieving “mastery level” for each objective • Text/Graph/Chart analyzing how subgroups performed relative to each objective

3. Examples of Objectives Classified Low Level of Difficulty Cognitive: Knowledge & Comprehension (Bloom); Knowledge (Sternberg) Affective: Receiving & Responding to Psychomotor: Reflex movements & Fundamental movements • In a given music example, the student should be able to identify all of the basic elements of music (cognitive) • The student will report two likes and two dislikes as a response to the assigned reading (affective) • The student performs a tennis serve with correct form (psychomotor)

4. Examples of ObjectivesClassified Middle Level of Difficulty Cognitive: Application & Analysis (Bloom); Skills/Performance/Applications (Stenberg) Affective: Valuing & organizing of values Psychomotor: Perceptual abilities & Physical abilities • After watching a video on scientific theories the student will list at least one difference between the use of the term “theory” in science and the use of “theory” in non-science contexts (cognitive) • During a debate, the student will defend the right of scientists to conduct research in a three-minute statement (affective) • Within the time allowed the student will adjust a microscope so that the image is clear (psychomotor)

5. Examples of ObjectivesClassified High Level of Difficulty Cognitive: Synthesis & Evaluation (Bloom); Reasoning Ability (Stenberg) Affective: Internalizing values Psychomotor: Skilled movements & Nondiscursive communication • Given a map with six distinct geographical features, students will be able to evaluate the best location for building a new city (cognitive) • Student members of jazz band will perform solo improvisations in Count Basie style for One O’Clock Jump (affective) • The student accurately performs a tennis serve, including correct spin, speed and placement of the ball in the opposite side of the court (psychomotor)

6. Graph for Pre/Post Assessment Data for Each Student This example graph shows a comparison of the pre & post-assessment data for each student. Pre-Assessment Score Post-Assessment Score

7. .82 82% Concept of Gain Scores A gain score is the actual gain divided by the potential gain. ACTUAL GAIN POTENTIAL GAIN GAIN SCORE = What does it mean?

8. Gain Score Worksheet Pre Post Gain Student #ScoreScoreScore 1 40 90 .83 2 90 95 .50 3 40 80 4 30 85 5 85 88 .20 6 75 100 +1.00 7 60 90 .75 8 40 95 .92 9 40 40 .00 10 75 50 -1.00 11 45 70 .45 12 60 80 .50 13 40 95 .92 Using the formula, calculate the missing gain scores. GAIN SCORE Solve

9. Gain Score Worksheet Pre Post Gain Student #ScoreScoreScore 1 40 90 .83 2 90 95 .50 3 40 80 .67 4 30 85 .79 5 85 88 .20 6 75 100 +1.00 7 60 90 .75 8 40 95 .92 9 40 40 .00 10 75 50 -1.00 11 45 70 .45 12 60 80 .50 13 40 95 .92 GAIN SCORE Solutions: 80 – 40 100 – 40 40 60 = = .67 85 – 30 100 – 30 55 70 = = .79 Include the average of the gain scores. “ 50%” Average Learning Gain Group Average .50

10. Negative Gain Scores What happens when a student scores lower on the post-assessment than on the pre-assessment? Example: Pre-Assessment = 75 % Post-Assessment = 50 % -25 25 Formula: Post-Assess % - Pre-Assess % 100% - Pre-Assess % • - 75 • 100 - 75 -25 25 = = = - 1.00 This student could have gained up to 25 points, but instead lost 25 points, a value equaling 100 % of what could have been gained.

11. Gain Scores for Pre-Assessments of 100% What happens when a student scores 100% on the pre-assessment? • When a student scores 100% on the pre-assessment, that score must be changed to 99%. Otherwise, the potential gain will equal zero and the gain score will be undefined. • Changing the student’s pre-assessment score to 99% will make the potential gain equal 1. Then, using 1 as the denominator (instead of 0) will allow the score to be defined. Explanation: Remember: If the potential gain equals zero, then Actual GainPotential Gain Gain Score = Actual Gain0 undefined =

12. Other Points About Gain Scores • Gain scores are not a perfect measure • The student teacher/intern should include more than just assessment of knowledge in the assessment plan and gain score calculations • The student teacher/intern will not being evaluated or judged on the amount of gain produced

13. The Learning Achievement Table Purpose: To identify the percentage of students who achieved a minimum mastery level for EACH learning objective. • All assessment data (tests, performances, portfolios, formal and informal questioning, checklists, rubrics, etc.) could be used in this table to calculate percentages. • Required in order to capture some forms of performance assessment that are criterion (given once at the end) assessments. • The student teacher/intern sets the criteria for what is considered mastery. • The student teacher/intern is not going to be evaluated by degree of mastery.

14. Example of a Learning Achievement Table

15. Setting Mastery Levels Example for: Knowledge Objective & Assessment Type Objective: List 12 reasons for the Industrial Revolution Assessment: Fill in the blank – paper and pencil test Mastery Level: 9 out of 12 is considered mastery Example for: a Performance Objective & Assessment Type Objective: Student integrates technology throughout presentation Assessment: Student presentations at the end of the unit Mastery Level: Score of a 3 on a 1-3 rubric factor for technology (see rubric) Mastery should not be set too high or too low. The student teacher/intern needs to be able to examine each student’s performance for each objective.

16. Sub-Group Comparisons • Purpose: To provide evidence that the student teacher/intern can deal effectively with students with various needs and from various backgrounds. • The student teacher/intern picks the groups of individuals • Could use SES, ESL/Non-ESL, race/ethnicity, disabilities, achievement levels, and gender as potential comparisons • Comparisons can be depicted: • in text using a narrative while referring to gain scores and objective mastery % • with a chart or graph (example on next slide)

17. Example Graph for Sub-Group Comparison Non-ESL, 86% ESL, 84% Percentage Could also use Gain Scores here.