By Thomas Ultican 3/10/2015
While discussing a student’s next class schedule with an outstanding student counselor, I learned my colleague had become dependent on testing data to place her students. That is not a surprise; it is one of the fundamental errors permeating education globally. Standards based testing is a fraud; a mirage that falsely projects an aura of objective analysis. As soon as high stakes are tied to these tests, their validity is undermined.
The learning standards upon which high stakes testing is based come from a mistaken philosophy of pedagogy that posits: a standardized learning rate, standardized interests, linear learning progression, developmental alignment, etc. Humans are not standard. Some learn to speak at 16 months and some don’t acquire that skill until 72 months. Some are short, others are tall. Some are fast, other are not. A child from urban Chicago has different perceptions and interests than a child from Winnemucca, Nevada. A global curriculum will not meet the needs of an endlessly diverse population. One size truly does not fit all. Even if it did, it would still be a bad idea to have political entities in centers of power deciding what that curriculum should be.
About 500 BC, the ancient Pythagoreans were a mathematically based cult. One of their beliefs (mathematical standard) was that there was no such thing as an irrational number. An irrational number is a number that cannot be expressed as a ratio of two numbers such as ½. The most famous irrational number is the ratio of the circumference of a circle divided by its diameter which is expressed as π (≈3.14). The writer, Morris Kline, claims that the man credited with the discovery of irrational numbers was a Pythagorean named Hippasus who had the bad sense to announce his discovery while at sea. He was thrown overboard for his heretical thinking![1] This story may be apocryphal but it does delightfully illustrate the danger associated with ossifying thinking with standards.
The longest most sustained use of standards and standardized testing arose in China more than 2000 years ago during the Han dynasty. It was an attempt to select government official based on some sort of meritocracy rather than feudal family station. However, local Lords were able to subvert the testing criteria and maintain the power of appointment. Starting in about 200 AD, China entered a 400 year long warring states period of instability.
In 581 AD, the new Emperor Wen of the Sui dynasty achieved a coup over his employer the Northern Zhao and then militarily unified China. Emperor Wen believed he needed to protect himself from the power of feudal lords, so he reintroduced the ‘keju’ or testing system to select government bureaucrats. Meritocracy based on test results became the sole path to a government position. University of Oregon’s dean of global education and professor in the Department of Educational Measurement, Yong Zhao, recently wrote about the ‘keju’ system:
“The emperor’s biggest concern was keeping China unified under his family’s rule. Learning from his own example, he realized he needed a way to weaken the hereditary power of certain families and tribes. Thus, he needed to find people who could help govern the country without relying on the existing ruling class. He also needed a way to prevent capable talents from rising against the empire and reinforce among his subjects the need to obey the rightful rule of the Son of Heaven.
We can’t know how much Emperor Wen planned and strategized, but the establishment of keju accomplished every one of the goals.”[2]
The ‘keju’ exams focused on memorization of the Confucian cannon and interpretive expository writing about the cannon and current political affairs. By far the most prestigious position in pre-industrial China was a government appointment obtained by success in these exams. This testing became the focus of education for virtually everyone living in the most advanced civilization on the planet.
It has been estimated that in 600 AD, China had at least a 400-year scientific lead on the rest of the world. So why didn’t the industrial revolution occur in China? Former Chief Economist and Senior Vice President of the World Bank, Justin Yifu Lin posits:
“I believe the real reason of the absence of scientific revolution was not due to the adverse political environment that prohibited the creativity of Chinese intellectuals, but due to the special incentives provided by the civil-service examination system. Because of this examination system, curious geniuses were diverted from learning mathematics and conducting controllable experiments. Because of this system, the geniuses could not accumulate crucial human capital that was essential for the scientific revolution. As a result, the discoveries of natural phenomena could only be based on sporadic observations, and could not be upgraded into modern science which was built upon mathematics and controlled experiments.”[3]
In China’s case the test became the education standard and that has been an ironclad law of standards and standards based testing ever since. Even David Coleman, the man who sold the Common Core to Bill Gates and then directed their development, knows that the tests become the real standards that will be taught. In a speech, he said:
“It was Lauren who propounded the great rule that I think is a statement of reality, though not a pretty one, which is teachers will teach towards the test. There is no force strong enough on this earth to prevent that. There is no amount of hand-waving, there’s no amount of saying, and ‘They teach to the standards, not the test; we don’t do that here.’”[4]
For America the first large scale standards movement was led by the Committee of Ten which published its report in 1894. Prepared under the auspices of the National Educational Association, this report was first published by the Bureau of Education, at Washington DC. The Committee of Ten was led by Harvard University President, Charles W. Elliot. The ten members supervised the development of standards for 9 different subjects. Nine subcommittees of ten members each worked in various locations around the country on their particular field of expertise.
There was an attempt to insure that the subcommittees were constituted by members representing diverse geographical locations. The committees were dominated by college staff and administrators but there were one or two classroom teachers on most of the committees, but the largest group of educators in America appears to have been ignored. There were no women on any of the committees.[5]
Two of the published standards – and there are many more – illustrate the innate problem with codified standards:
“They recommend ‘that the course in arithmetic be at once abridged and enriched; abridged by omitting entirely those subjects which perplex and exhaust the pupil without affording any really valuable mental discipline, and enriched by a greater number of exercises in simple calculation, and in the solution of concrete problems.’”[6]
So in mathematics, the committee called for more drill and skill which is exactly the kind of teaching Benjamin Bloom’s ‘Taxonomy of Educational Objectives’ called into question. The following comes from the science standards:
“As regards Resolutions 3, 4, and 5, it should be said that the order recommended for the study of Chemistry and Physics is plainly not the logical one, but all the members with one exception voted for Resolution 3 because they felt that the pupils should have as much mathematical knowledge as possible to enable them to deal satisfactorily with Physics, while they could profitably take up elementary Chemistry at an earlier stage.”[7]
This decision is why to this day we teach biology then chemistry then physics. This order is exactly backwards. To read a biology book a student needs knowledge of chemistry and to understand the forces that drive chemical reactions the principles of energy from physics is needed. San Diego Unified School District tried to rectify this order but they abandoned their expensive effort. The district was unfairly punished when their 9th grade physics student test scores did no compare well with the scores of mainly 11th and 12th grade students from across California on mandated standardized testing.
Mark Silver of John Hopkins reports,
“The standards movement has its roots in curriculum content and skills. The National Council of Teachers of Mathematics (NCTM) was one of the first national professional organizations to publish national curriculum standards and goals.”[8]
These standards were guiding documents for the development of the California math standards which are widely considered superior to the common core standards. The common complaint I heard from fellow teachers implementing the California math standards was that they were “a mile wide and an inch deep.” These standards took us from “uncovering” mathematical principles to “covering” the material. Constructivist approaches like problem based learning were completely undermined and math instruction was pushed back to 1894’s drill and skill model. The high stakes involved made success on a narrow range of tested material paramount.
Since the inception of the California math standards, ossified standards have existed; for example rationalizing denominators. This procedure insures that there are no square-root factors in a denominator. Before calculators, rationalizing denominators significantly eased difficult arithmetic, but after calculators dividing by radicals became no more difficult than dividing by sums. Still the California teachers were compelled to drill their students on an antiquated algorithm.
In the latter half of the nineteenth century, Japan and China reacted differently to being behind the West scientifically and technically. China maintained its education culture and purchased technology. Japan completely changed its approach to education and engaged in a strenuous effort to catch the West in science, mathematics and manufacturing. The hallmark of Japan’s approach was authoritarianism. The goals of education became modernizing and producing loyal disciples for the Emperor. These purposes were to be insured by tested education standards. The education reformer, Tsunesaburo Makiguchi, criticized both the goals and the methods. He denounced standards based education as mechanized education and agonized over putting children through “testing hell”.
With authoritarian standards based education, Japan became both a world power and a menace. China’s equally authoritarian standards based education led to weakness and vulnerability. Both countries were on a path of ruin.
The renowned historian, Arnold Toynbee established a criteria for judging whether a civilization was in a stage of growth or decline. He wrote:
“We must ask whether, as we look back over the ground we have traversed, we can discern any master tendency at work, and we do in fact unmistakably decry a tendency towards standardization and uniformity: a tendency which is correlative and opposite of the tendency towards differentiation and diversity which we have found to be the mark of the growth stage of civilizations.”[9]
Both John Dewey and Tsunesaburo Makiguchi counseled against standardized education. Dewey stated “Imposing an alleged uniform general method upon everybody breeds mediocrity in all but the very exceptional.”[10] And Makiguchi theorized that “Education integrated into the life of society will yield benefits of well-planned living, without the undesirable effect of mechanical uniformity an inherent danger in standardized education.”[11]
What is the root of the persistent and two millennial old tendency for politicians with minimal knowledge of education creating education standards and mandating testing accountability? It originates in a deep rooted innate and evil desire in humans to control other humans. If we do not fight this tendency, we are doomed to live in an authoritarian society where political elites ensure subservience by controlling education standards enforced by standardized testing.
A better path forward is the development of consensus about the purposes of education through continual dialogue. Then allow professional educators to create curricular guidance by a process of peer reviewed research. Community schools should use that curricular guidance in the best way they see fit and should be encouraged to experiment. Statistical sampling identical to the National Assessment of Education Progress should be used to assess progress and provide information for the purposes of continued dialogue, research and further progress. It should be an ongoing process with no shortcuts; no silver bullets and no miracles.
As for my colleague’s concern about guidance in student class placement, teacher grades and recommendations along with input from the student and their parents is a far more reliable method of placement than relying on dubious standardized testing scores.
- Morris Kline (1990), Mathematical Thought from Ancient to Modern Times, page 32. Oxford University Press
- Yong Zhao (2014), Who’s Afraid of the Big Bad Dragon? Jossey-Bass of John Wiley & Sons, Inc. Page 38
- http://www.uq.edu.au/economics/cepa/docs/seminar/papers-nov2006/Lin-Paper.pdf
- http://atthechalkface.com/2013/12/23/this-is-the-common-core-you-support/
- https://openlibrary.org/books/OL7129384M/Report_of_the_Committee_of_ten_on_secondary_school_studies
- https://archive.org/stream/reportofcomtens00natirich#page/105/mode/1up
- https://archive.org/stream/reportofcomtens00natirich#page/119/mode/1up
- http://education.jhu.edu/PD/newhorizons/Transforming%20Education/Articles/Trends%20in%20School%20Reform/
- Arnold Toynbee (DC Somervell), “A study of History” abridgement Volume 1 – VI, Oxford University Press, 1946, page 555
- Dewey, John. Democracy and Education. New York: Macmillan, 1916. Page 203
- Ikeda, Daisaku. Soka Education. Santa Monica, California: Middleway Press. 2001 Page 18
tultican 
I have a similar view: http://teacherbatman.org/standards/
LikeLike
I read your blog. Please keep up the good fight. What you are doing is really important.
LikeLike
Thanks, same to you.
LikeLike
“As soon as high stakes are tied to these tests, their validity is undermined.”
How can validity be undermined when there is no validity whatsoever to the standards and testing malpractices. I believe you’ve probably seen this tultican, but if I may re-iterate. Noel Wilson has shown us all the errors and falsehoods and psychometric fudgings involved in those malpractices that render any results COMPLETELY INVALID. I’ve been searching, pleading, digging around and have yet to encounter any refutation or rebuttal by anyone of Wilson’s seminal work “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other words all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self-evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
For once I was actually acquainted with something, I had read Wilson’s work. I would agree standardized testing has no validity. However, I also believe a no stakes involved assessments like Ravitch’s NAEP testing can be validly used to track trends in learning. As soon as any stakes are tied to the test, people will make changes to do better on it. Then the test is no longer useful for any purpose.
Thank you for the wonderful response.
LikeLike