Download
1 / 48
480 likes | 485 Views
David C. Berliner Regents ’ Professor Emeritus Mary Lou Fulton Teachers College Arizona State University. How High-Stakes Testing is Hurting American Education Institute of Education, Higher School of Economics Moscow, September, 2014.
E N D
David C. Berliner Regents’ Professor Emeritus Mary Lou Fulton Teachers College Arizona State University How High-Stakes Testing is Hurting American Education Institute of Education, Higher School of Economics Moscow, September, 2014
The theory of action underlying NCLB Is this: Lazy teachers and lazy students abound, and that they can be motivated to work harder in school. Teachers and students will learn more if we have external tests with which to judge them and pressure them. Through the tests we can identify those lazy teachers and kids, as well as the schools with low expectations, and punish all those bad teachers and students for their laziness and improper beliefs. Side Issue of great importance: A great bipartisan switch took place under President George Bush, with the aid of hundreds of other elected leaders and the business community: Congress decided to worry about outputs, and ignore inputs. Let me illustrate:
Framing School Reformwww.whitehouse.gov, 2001-2007 ‘Achievement Gap’ • 344 documents ‘Equal Educational Opportunity’ • 3 documents
How you frame the problem (George Lakoff) often determines how you try to solve it! We have been subtly moved from concerns for social justice in order to have the schools we want for our children, to concerns about how those children score on tests. We have been moved from worrying about equal educational opportunity, and health care, and livable wages, and food insecurity, and neighborhoods that are filled with drugs, crime and poor role models, and a host of other variables that affect the achievement of students who walk into our schools; for that we have substituted instead concern for their test scores and what skills they have when they walk out of our schools, as if the two are unrelated! And if we find our students deficient when they leave schools, its natural to blame the school people, those that worked with them last, instead of looking in the mirror and saying we are all responsible for the performance of all our children.
The question we had: How might we test the theory of action that underlies NCLB? Our Solution: Scale the states on pressure and check how pressure affects NAEP scores. Our method: Prepare thick and rich descriptions of the pressure felt by teachers, administrators, parents and students in each state, as determined from a sampling of newspaper articles in each state. Then compare each state with every other state (a 25 x 25 state matrix, requiring >300 judges). Have one and only one judgment made per judge: “Which of the two state portfolios you are looking at seems to be exerting more pressure for high performance on the achievement tests that are used to comply with NCLB?”
So, point one is made: NCLB is probably not now working. Point two is that NCLB cannot work, ever. NCLB cannot work for many reasons, (e. g. 100% proficient and all kids reaching proficiency at the same time). But the most important reason it cannot work is that it violates a fundamental social science law demonstrated through myriad incidents to be applicable to a wide range of human endeavors. This is Campbell’s law.
Campbell’s Law • “The More any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” • (Donald Campbell, 1975, p.35)
The uncertainty principle • George Madaus, 2002, pointed out that this law leads to the social science equivalent of the Heisenberg uncertainty principle: • The higher the stakes involved in testing, the less likely you are to get an accurate measurement of the construct you most want to measure. • So you simply cannot have both high-stakes and high validity because the higher the stakes the more corrupt the measure becomes. • Thus, uncertainty about the meaning of a test score rises with increases in the consequences associated with scores on that test.
We found schools that did some odd classification of special education and English Language learners and even reclassified mixed race children in odd ways. We found schools that move kids around to they didn’t have to test them. We found schools that claimed they were safe when they were not. We found that the tests often forced a restriction in the cognitive processes specified in the standards. We found schools that vastly narrowed the curriculum in social studies, art, music, and physical education.
What we have learned is that tests may not predict the future very well. This is especially true when the future is Volatile, Uncertain, Complex, and Ambiguous--when a VUCA future is likely. And this serious lack of predictive validity is magnified when the scores are compromised. When the every-day problems of long-term predictive validity for tests are combined with high-stakes testing, the construct validity of the tests are compromised as well. That is, we may no longer be measuring what we think we are measuring due to the corruption of the indicator and the corruption of those who work with that indicator. Campbell’s law is ubiquitous! This is a social science law of great power, defied by Texas and the federal government at our great risk. High-stakes testing is wicked policy: It not only corrupts it may hurt the economy of the nation, as well.
The longest running high-stakes testing program in history: CHINA’S CIVIL SERVICE EXAMS 606-1905----1,298 years of very high-stakes tests 4 findings: test takers cheated a lot test preparers cheated a lot test scorers cheated a lot test proctors cheated a lot Around the year 1400 test changes were made and China changes at that time as well. Could it be causal?
The Bicycle Ride (1946) • I awakened early, jumped out of bed and had a quick breakfast. My friend, Mary Quant, was coming to our house at nine o’clock as we were going for a long bicycle ride together. It was a lovely morning. White fleecy clouds floated in a clear blue sky and the sun was shining. As we cycled over Castlemore bridge we could hear the babble of the clear stream beneath us. Away to our right we could see the brilliant flowers in Mrs. Casey’s garden. Early summer roses grew all over the pergola which stood in the middle of the garden.
A Day in the Bog (1947) • I awakened early and jumped out of bed. I wanted to be ready at nine o’clock when my friend, Sadie, was coming to our house. Daddy said he would take us with him to the bog if the day was good. It was a lovely day. White fleecy clouds floated in a clear blue sky. As we were going over Castlemore bridge in the horse and cart, we could hear the babble of the clear stream beneath us. Away to our right we could see the brilliant flowers in Mrs. Casey’s garden. Early summer roses grew all over the pergola which stood in the middle of the garden.
A Bus Tour (1948) • I awakened early and sprang out of bed. I wanted to be ready in good time for our bus tour from the school. My friend, Nora Green, was going to call me at half-past eight as the tour was starting at nine. It was lovely morning. White fleece clouds floated in the clear blue sky and the sun was shining. As we drive over Castlemore bridge we could hear the babble of the clear stream beneath us. From the bus window we could see Mrs. Casey’s garden. Early summer roses grew all over the pergola which stood in the middle of the garden.
****The lesson from the study of China is to be wary of tests that reflect a mimetic bias--they influence what is valued and thus what is taught. ****The lesson of Ireland is to give teachers some autonomy so they don’t want to please the test makers or the governement. ****For individuals and for societies the more VUCA-like the future, the less likely that the knowledge of the past will serve as a reliable guide for success in that future. ****Creativity, problem identification, knowledge seeking, problem solving, media literacy, ability to sort facts from fiction in a knowledge rich world, Bayesian and probabilistic estimation, systems thinking, collaborative work skills, and so forth, are more likely to be the skills that youth need if a society is to compete well in the 21st century. ****The US curriculum and its assessments do not reflect such ideas. The US is becoming much more of a nation that values the mimetic and not the transformative roles of the teacher! THAT IS A MISTAKE!
Teachers’ Voice: Colorado…..We don’t take as many field trips. We don’t do community outreach like we used to, like visiting the nursing home or cleaning up the park because we had adopted a park and that was our job, to keep it clean. Well, we don’t have time for that any more.”
Another Colorado teacher:”We only teach to the test, even at 2nd grade, and have stopped teaching science and social studies. We don’t have assemblies, take few field trips, or have musical productions at grade levels. We even hesitate to ever show a video. Our 2nd graders have no recess except for 20 minutes at lunch.”
A teacher in Florida says: 2 “Our total curriculum is focused on reading, writing, and math. There is no extra time for students to study the arts, have physical education, science, or social studies. Our curriculum is very unbalanced.”
John Adams: “I must study politics and war that my sons may have liberty to study mathematics and philosophy. My sons ought to study mathematics and philosophy, geography, natural history, naval architecture, navigation, commerce and agriculture in order to give their children a right to study painting, poetry, music, architecture, statuary, tapestry, and porcelain.”
Screwing up our kids views of schooling: Testing Ms. Malarkey Book by Judy Finchler Illustrations by Kevin O’Malley
“Testing and teaching to us is the same thing.”Secretary Paige quoted in the Herald-Tribune (Southwest Florida), July 14, 2004.
