August 2012 Coral Gables

1. August 2012 Coral Gables • PARCC Bias and Sensitivity Review Confidential and Proprietary.

2. Points of Discussion • Purpose of Bias and Sensitivity Reviews • Identifying Bias and Stereotypes • Sensitivity Awareness and Detecting Passage Bias • Guiding Questions and Sample Items Confidential and Proprietary.

3. Purpose of Bias and Sensitivity Review • Review test materials for potential sources of bias and stereotypes. • Apply professional test development standards to ensure materials are fair and not insensitive or offensive. Confidential and Proprietary.

4. What Is Bias? • Language or content that prevents members of a group from demonstrating they possess the knowledge and skills being measured. • Language or content that advantages members of a group in demonstrating they possess the knowledge and skills being measured. Confidential and Proprietary.

5. Common Forms of Bias and Stereotypes • Regional and geographic bias • Gender and age stereotypes • Ethnic, cultural, and religious stereotypes • Socioeconomic and occupational stereotypes Confidential and Proprietary.

6. Guideline #1 Avoid Cognitive Sources of Construct-Irrelevant Variance Confidential and Proprietary.

7. Construct = knowledge, skill, or other attribute (KSA) you are trying to test Construct-relevant = related to KSA that you are trying to test Construct-irrelevant = KSA that is not related towhat you are trying to test Variance = differences in test scores Some Background 7 Confidential and Proprietary.

8. Variance, Validity & Fairness • Construct-irrelevant variance lowers validity. • Construct-irrelevant variance that has different effects across groups lowers fairness. • Construct-irrelevant differences across groups decrease validity & fairness. • Construct-relevant differences across groups are valid and, therefore, fair. Confidential and Proprietary.

9. Guideline #2 Avoid Affective Sources of Construct-Irrelevant Variance Confidential and Proprietary.

10. Sensitivity Awareness • Any reference or language in an item or passage that might cause a student to have an emotional reaction during the test administration can prevent a student from being able to accurately demonstrate ability. • Note: The emotional factor is not limited to negative emotions, but can also include a “giggle factor” in items. Confidential and Proprietary.

11. Guideline #3 Avoid Physical Sources of Construct-Irrelevant Variance Confidential and Proprietary.

12. Samples of Barriers • Visual stimuli in the middle of paragraphs • Decorative rather than informative illustrations • Fonts that are hard to read • Letters that look alike (e.g., O, Q) used as labels for dif-ferent things in the same item/stimulus Confidential and Proprietary.

13. How Might Item Bias be Detected? • Judgmental Procedure • Bias and sensitivity review prior to field-testing • Statistical Procedure • DIF (Differential Item Functioning) analysis following field-tests and operational administrations Confidential and Proprietary.

14. Guiding Questions for Passages • Does the passage disadvantage any population (gender, race, ethnicity, language, religion, socioeconomic status, disability or geographic region) for non-educationally relevant reasons? • Does the passage contain controversial or emotionally charged subject matter that is not supported by the Common Core State Standards? • Is the passage potentially offensive, demeaning, insensitive, or negative toward any population? • Does the passage depict any population in a stereotypical manner? Confidential and Proprietary.

15. Sample Passage #1 Why You Get Hungry Not hungry? Don’t worry…you will be. Just a little while from now, you'll begin to feel something just under your rib cage. It may seem like a little pressure or warmth. Some people think it feels like hollowness. … You’ve been getting hungry every few hours since the day you were born. Isn't it about time to understand what’s going on? Don’t blame an empty stomach. Early in the twentieth century, doctors noticed something interesting about patients who had their stomachs surgically removed. The patents still felt hunger the same old way. So doctors and scientists decided the feeling of hunger must come from somewhere else. It turned out to be a tiny area, about the size of a garbanzo bean, in the middle of your brain. It’s called the hypothalamus (hi-po-THAL-a-mus). It constantly monitors what’s going on in your bloodstream and sends reports to different parts of your brain. Blood’s too warm? Make ‘em sweat. Blood’s too cool? Make ‘em shiver. Not enough fuel in the blood? Make ‘em eat. How? By making ‘em hungry. It’s your brain creating that unmistakable feeling called hunger. Hunger is powerful, and it’s meant to be unpleasant. Once it starts, there’s really nothing you can do to relieve it except eat. And so you eat—usually until a second part of your hypothalamus tells your brain that the amount of fuel in your blood is just right … Fairness Guideline 1, 2, or 3? Confidential and Proprietary.

16. Sample Passage #2 • The Dog Who Wouldn’t Be • My third class that morning happened to be French. The teacher was a desiccated female whose spiritual home may have been Paris, but who had never actually been farther east than Winnipeg. She was affected, humorless, and a tyrant. None of us liked her. Yet I actually felt sorry for her when, in the midst of the declination of an irregular verb, Wol whumped moodily in through the second-story window and slid to an unsatisfactory halt upon the top of her hardwood desk. The exclamation with which she greeted him was given in very old Anglo-Saxon, without even a hint of a French accent. • I had an interview with the principal after this incident, but he was a reasonable man and the upshot was that I escaped corporal punishment on the understanding that my owl would stay at home in future. • I achieved this end, but only at the cost of giving the owls the free run of our house. Some ten minutes before I was due to leave for school I would invite Weeps and Wol into the kitchen, where they were allowed to finish off the bacon scraps left from our breakfast. Apparently Wol accepted this as a sufficient bribe, for he made no further attempt to follow me to school; and Weeps, who always accepted Wol’s lead gave me no more trouble either. … Fairness Guideline 1, 2, or 3? Confidential and Proprietary.

17. Sample Passage #3 • Tiny Houses: a tiny trend that runs against the mainstream • In 2008, Ben and Sarina Speed of Maine challenged themselves to build and live in the smallest house possible—one that would accommodate only their most fundamental needs along with the needs of their young children and two cats. Among the reasons for downsizing was their conviction that smaller houses are environmentally friendly because they consume less energy. Indeed, the couple reports that now the electric bill for their 640 square foot home, which is smaller than some garages, averages $20 per month. • The Speeds have joined an increasing number of Americans committed to shrinking their residential footprints and purging their lives of excesses they have come to view as burdensome, indulgent, harmful to the environment, or all of the above. •  How small is small? Jay Shafer, the co-founder of the Small House Society, says designs can range in size from 65 square feet to 874 square feet. Even the high end of the range marks a substantial reduction from the average sized home. • Of course, not everyone approves of this philosophy. Supporters and opponents of tiny homes—also called eco-homes, micro-homes, or mini-homes—tend to be resolute in their positions. • That tiny houses use less energy and are therefore environmentally friendly is indisputable; the smaller the area, the less energy required for heating and cooling… Let’s discuss Confidential and Proprietary.

18. Sample Passage #4 A Treasury of Western Folklore Upon the 10th of May, 1869, the rival roads approached each other, and two lengths of rails were left for the day’s work. At 8 A.M., spectators began to arrive; at quarter to 9 A.M., the whistle of the Central Pacific Railroad is heard, and the first train arrives, bringing a large number of passengers. Then two additional trains arrive on the Union Pacific Railroad, from the East. At a quarter of 11 A.M., the Chinese workmen commenced leveling the bed of the road with picks and shovels, preparatory to placing the ties. At a quarter past eleven the Governor's (Governor Stanford’s) train arrived. The engine was gaily decorated with little flags and ribbons—the red, white, and blue. The last tie is put in place—eight feet long, eight inches wide, and six inches thick. It was made of California laurel, finely polished, and ornamented with a silver escutcheon, bearing the following inscription: THE LAST TIE LAID ON THE PACIFIC RAILROAD, MAY 10, 1869. …In San Francisco, the wires were connected with the fire-alarm in the tower, where the heavy ring of the bell might spread the news immediately over the city, as quick as the event was completed. Waiting for some time in impatience, at last came this message from Promontory Point, at 2:27 P.M. 18 Confidential and Proprietary.

19. Sample Passage #4, Continued Almost ready. Hats off, prayer is being offered. A silence for the prayer ensued; at 2:40 P.M., the bell tapped again, and the officer at Promontory said: We have got done praying, the spike is about to be presented. … Let’s discuss

20. Questions? Confidential and Proprietary.

21. How To Proceed • Read through a specific set of test materials, keeping in mind the guiding questions. • Do not review these materials from the point of view of a content expert, instead review them only from the perspective of fairness. • If possible sources of bias, stereotypes, or sensitivity are detected, note them in your comments, and then they will be discussed by the group. • The recorder will take complete and precise notes for the group. Confidential and Proprietary.

22. Passage Sources Botkin, B. A. . A Treasury of Western Folklore. New York: Crown Publishers, 1951 Haduch, Bill. Food Rules! New York: Puffin, 2001 Mowat, Farley. The Dog Who Wouldn’t Be. Ontario: Little Brown & Co, 1957 22 Confidential and Proprietary.