Tag Archives: Eric Hanushek

CREDO’s New Study Biased against Public Schools

14 May

By Thomas Ultican 5/14/2020

The Center for Research on Education Outcomes (CREDO) started releasing the results of its new Cities Study Project in mid-2019. It is not a coincidence that the cities chosen for the study have long been targeted for public school privatization. The ten cities selected are: Indianapolis; Baton Rouge; Camden; Kansas City; Memphis; New Orleans; Oakland; St. Louis; San Antonio; and Washington DC. This CREDO study is even more opaque and biased than its previous efforts.

Who is CREDO?

Hanushek and Raymond

Husband and Wife Team Who Founded CREDO

In the early 1980s, Margaret (Macke) Raymond was completing a lengthy graduate school agenda at the University of Rochester, a relatively small private university in Rochester, New York. She garnered an MS of public policy in 1980, a community medicine MS in 1982, an MA of political science in 1983 and finished with a PhD in political science in 1985. From 1985 to 2000 she ran Raymond Consulting and worked a few years in the telecommunications industry.

At that same time, Eric Hanushek was Professor of Economics and Political Science at Rochester University. The former Air Force cadet had earned a doctorate of Economics from the Massachusetts Institute of Technology in 1968. It was Hanushek’s 1981 paper “Throwing Money at Schools” that put him on the watch list of right leaning philanthropies and institutions. In his notice making missive he stated,

“The conventional wisdom about public schools is that they face serious problems in terms of performance and that improving schools requires additional money. However, the available evidence suggests that there is no relationship between expenditures and the achievement of students and that such traditional remedies as reducing class sizes or hiring better trained teachers are unlikely to improve matters.”

In a 1981 Ed Week commentary referencing this paper, Hanushek points to SAT testing as the gold standard for judging school performance. In complete accordance with the old aphorism, “to a man with a hammer all problems look like a nail,” Hanushek brags, “Advanced statistical techniques are employed to disentangle the influences on achievement of schools and teachers from those of other factors such as family backgrounds and student abilities.”

A 1999 announcement from Rochester University said, “The Center for Research on Education Outcomes has been established at the University of Rochester’s Wallis Institute of Political Economy…” In the same posting, it was revealed, “Two foundations have committed $1.25 million to fund a three-and one-half year initiative to address the current shortage of evaluation research in education policy matters.” CREDO never made the name of the two foundations public, but a knowledgeable academic says one of them was the Walton Family Foundation.

The announcement listed two employees of the new center, Eric Hanushek and Margaret Raymond. Hanushek was listed first but Raymond was cited as the founding Director.

CREDO moved from the University of Rochester to Stanford University’s Hoover Institute in July, 2000 which made networking in conservative circles much easier.  In CREDO’s 2nd year report, they stated that moving to the Hoover Institute brought them many new contacts including the New Schools Venture Fund, the District of Columbia Charter School Board, the Teacher Union Reform Network and others.

A description of the Hoover Institute from Source Watch says,

“The Hoover Institution is influential in the American conservative and libertarian movements, and the Institution has long been a place of scholarship for high profile conservatives with government experience. A number of fellows have connections to or positions in the Bush administration, and other Republican administrations. … Other fellows of the Institution include such high profile conservatives as Condoleezza Rice, George Shultz, Thomas Sowell, Shelby Steele, and Edwin Meese.”

Macke Raymond’s 2015 Hoover Institute Fellow’s profile says in part, “In partnership with the Walton Family Foundation and Pearson Learning Systems, Raymond is leading a national study of the effectiveness of public charter schools.” 

Valerie Strauss of the Washington Post has pointed out that Eric Hanushek

“… a Hoover economist was a pioneer in creating systems that evaluate teachers by student standardized tests, a method that many assessment experts say should not be used in the high-stakes ways that school reformers are using them. He is often cited in CREDO studies as a ‘principal investigator.”’

Discredited and Biased

The Forbes commentator, Peter Greene, wrote about Eric Hanushek for his popular blog Curmudgucation:

“Now when Hanushek says that teachers make a huge difference, he is obliquely referencing his own crazy-pants assertion that having a good first grade teacher will make you almost a million bucks richer over your lifetime (you can also find the same baloney being sliced by Chetty, Friedman, and Rockoff). Both researchers demonstrate their complete lack of understanding of the difference between correlation and causation.”

Greene also shares the following graphic that clearly highlights the difference between correlation and causation.

Divorce caused by Margarine consumption

Does Margarine Consumption Cause Divorce?

Business writer Andrea Gabor states that CREDO studies which compare charter schools with public schools start with two key assumptions “A) That standardized-test scores are an adequate measure of school quality and B) that creaming in charter schools does not exist.”

With regards to assumption ‘A’, using standardized testing for this purpose has been shown faulty from studies dating back to the eugenics movement (which originated high stakes standardized testing) to recent works debunking them for mistaking correlation versus causation and for not being able to compensate for the problem of error.

As for assumption B, there is no doubt that most charter schools push out and avoid students that are classified as special education, language learners or discipline problems. The data proving that is in state enrollment reports wherever charter schools exist.

In the new research labled “Cities Studies Project”, the Technical Appendix says the reports uses growth models but doesn’t share which of the many growth models it uses. It also says,

“In our study, scores for all these separate tests are transformed to a common scale. All test scores have been converted to standardized scores to fit a ‘bell curve’, in order to allow for year-to-year computations of growth.”

The Education Growth Model Handbook lists seven types of growth models in general use and their requirements. Most growth models require vertical scales but that does not seem possible with CREDO’s use of multiple tests many of which are not vertically scaled. Their mathematical conversions add another locus of error. Growth models have proven to be unstable and have never been satisfactorily validated.

The research methodology used in the “Cities Studies Project” appears to be the same as that used in CREDO’s 2015 Urban Charter School Study Report on 41 Regions.” A particularly troubling practice employed then and apparently still being used is the “virtual twin” method which creates a pro-charter school bias.

Professor Andrew Maul of UC Santa-Barbara reviewed the 2015 study for the National Education Policy Center. He noted the CREDO method does not compare charter school performance to actual public schools; rather it creates mathematical simulations. Maul described the “virtual twin” schema employed to develop a “virtual control record.” He reports,

“CREDO’s approach to this estimate is the construction of a ‘Virtual Control Record’ (VCR) for each student in a charter school, obtained by averaging together up to seven students in “feeder” public schools (i.e., those schools whose students transfer to charters) with the same gender, ethnicity, English proficiency status, eligibility for subsidized meals, special education status, grade level, and a similar score from a prior year’s standardized test (within a tenth of a standard deviation) as the specified charter student.”

Maul adds, “The study’s “virtual twin” technique is insufficiently documented, and it remains unclear and puzzling why the researchers use this approach rather than the more accepted approach of propensity score matching.”

The stipulation that “virtual twins” come from “feeder schools” biases the study in favor of charter schools. Andrea Gabor explained that in practice, CREDO used less than five students transferring to a charter school as the cutoff for using a particular public school’s data. She reports that the, “study excludes public schools that do NOT send students to charters, thus introducing a bias against the best urban public schools, especially small public schools that may send few, if any, students to charters.” Gabor gave the example of two well regarded New York title-1 schools, Global Technology Preparatory and West Side Collaborative which were excluded. They are noted for scoring well on testing, but did not meet the transfer criteria yet easily matched the required demographics.

The CREDO study is singularly focused on test results as determinate of school quality and ignores other advantages of public schools. It is a well known fact that many charter school systems like IDEA and Success Academy spend an inordinate amount of time teaching to and preparing for standardized tests. To these criticisms, Professor Mark Weber of Rutgers University adds a few more observations:

“Cities Studies Project” Technical Appendix states,

“To assist the reader in interpreting the meaning of effect sizes, we include an estimate of the average number of days of learning required to achieve a particular effect size. This estimate was calculated by Dr. Eric Hanushek and Dr. Margaret Raymond based on the latest (2017) 4th and 8th grade test scores from the National Assessment of Educational Progress (NAEP).”

Converstion to Days of Learning

The CREDO Days of Learning Conversion Table from “Cities Studies Project”

This metric seems to have been created with next to nothing validating it. Mark Webber quoted the psychometrician Michael T. Kane,

“The 2015 study (p. 5) cites a paper published in Education Next (Hanushek, Peterson & Woessmann, 2012) that asserts: “On most measures of student performance, student growth is typically about 1 full std. dev. on standardized tests between 4th and 8th grade, or about 25 percent of a std. dev. from one grade to the next.” (p. 3-4) No citation, however, is given to back up this claim: it is simply stated as a received truth.”

CREDO tells us that Hanushek and Raymond did something with NAEP data from 2017 but still do not offer any justification for the conversion. It appears at best to be sloppy science and the headlines engendered from it are nothing short of propaganda.

Using CREDO Claims to Sell Privatizing Public Schools

Neerav Kingsland the Managing Partner of the City Fund posted to his Blog last July when the first results from “Cities Studies Project” arrived explaining,

“Last year, Arnold Ventures commissioned CREDO (out of Stanford University) to study the effects of charter, innovation, and traditional schools in select cities across the country.”

“Most of the cities included in the study were cities where Arnold Ventures (and now The City Fund) have partnered with local leaders to expand high-quality schools.”

“Camden’s city level effects are large.”

“In just two years, scores are up ~.15 standard deviations in math and ~.05 standard deviations in reading (compared to similar schools across the state).”

The reality is those changes are very small. Noise in the data is a better explanation than awesome charter schools for these tiny differences.

In Indianapolis, the CEO of The Mind Trust, Brandon Brown, just wrote an Indy Star opinion piece stating,

“A 2019 study from Stanford University found that students who attend Innovation Network Schools achieve the equivalent of 53 additional days of learning in English and 89 additional days of learning in math each year when compared to their traditional public school peers. This equates to several years of additional learning during the span of a K-12 academic career, and the gains are largest for students of color and students from low-income backgrounds.”

In the billionaire financed effort to privatize public education, CREDO has become their source for data proving things like smaller class sizes and teacher professionalism are not important. The “Cities Studies Project” commissioned by an organization intent on privatizing public schools through promoting the portfolio management scheme – The City Fund – is biased toward the privatization agenda. Rather than shining the light of scholarly work on education policy, it obscures reality with obfuscation.

Illusion Motivates Education Reform

29 Dec

By Thomas Ultican 12/29/2014

My earliest memory of public education criticism was overhearing my mother and her teacher colleagues talking about a book memorably named, “Why Johnny Can’t Read.” It was a sensation that implicitly stated that public education in America was a dismal failure and that the nation was on a path to its own cultural and fiscal demise. Recently Jim Arnold & Peter Smagorinsky wrote this amazing paragraph about the history of declaring the failure of America’s public education system:

“Admiral Rickover published “American Education, a National Failure” in 1963, and in 1959 LIFE magazine published “Crisis in Education” that noted the Russians beat us into space with Sputnik because “the standards of education are shockingly low.” In 1955 Why Johnny Can’t Read became a best seller, and in 1942 the NY Times noted only 6% of college freshmen could name the 13 original colonies and 75% did not know who was President during the Civil War. The US Navy in 1940 tested new pilots on their mastery of 4th grade math and found that 60% of the HS graduates failed. In 1889 the top 3% of US high school students went to college, and 84% of all American colleges reported remedial courses in core subjects were required for incoming freshmen.”

The paragraph above recalls more than a century of national failure to properly educate our citizenry yet in that same century America became the world’s leader economically, scientifically, militarily and culturally. Does this mean that education quality does not matter or is it more likely that the perception of American education failing – is and was an illusion?

By the middle of the 20th century, cities and villages throughout the USA had developed an impressive educational infrastructure. With the intent of giving every child in America the opportunity for 12 years of free education, this country was the world’s only country not using high stakes testing to deny the academic path to more than a third of its students. The physical infrastructure of our public schools was of high quality and schools were staffed with well trained experienced educators.

Furthermore, a trained administrative structure was in place and the path to continuous improvement had been established. No one should deny that public education can be improved, but what is also undeniable is that America’s current system has a history of producing successful citizens. To call public education in America a failure is to make better the enemy of good. It promotes a destructive illusion.

I feel a surrealistic connection to what might be the most important driver ever for this destructive illusion. Terrel ‘Ted’ Bell, Ronald Reagan’s Secretary of Education, instigated a commission that gave us the infamous, “Nation at Risk.” Beyond just claiming that public education in America was failing and needed drastic reform; the claimants said that reform needed the leadership of people who were not professional educators.

First of all, the idea that you should ignore the advice of practicing professionals concerning any field of human endeavor is foolhardy, but foolhardiness permeates modern education reform. It is through Bell I feel a chimerical connection to imprudent reform.

Bell was from Lava Hot Springs, Idaho. I too grew up in rural Idaho. He attended Albion State Normal School in southern Idaho. That was my mother’s alma mater as well as that of more than half of my public school teachers. Like most of the Albion graduates, Bell taught while continuing his own education. He eventually earned a doctorate of education from the University of Utah. This man with roots close to my own became a key instrument in advancing the attack on America’s public schools, which he loved.

Bell’s “Nation at Risk” was a political document written by business men and famous scientists. It claimed without substantiation or peer review that the very fate of the nation was at risk because of our failing schools. The authors said “the educational foundations of our society are presently being eroded by a rising tide of mediocrity that threatens our very future as a Nation and a people” and claimed, “If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war.”

The paper was a very successful polemic and not an honest scholarly effort. Its prejudicial analysis of education was based on unsupported beliefs and ignored data that countered those beliefs. “Nation at Risk” significantly advanced the destructive illusion that public education was failing and the rescue must be led by corporate chieftains.

Terrel Bell was a idealist and an realist. He supported the department of education and Medicare. He had a profound religious faith in Mormonism. As a classroom chemistry teacher for many years, he devoted his life to public education. Bell was an unlikely choice to head Ronald Regan’s Department of Education. Bell, however, was picked because of his high standing as a leader of education policy in the Nixon and Ford administrations which purchased him favorable standing among the education community.

Bell was supposed to oversee the demise of the department of education and support the states’ rights views (definitely not national standards) of the new President, but he had his own agenda. Bell said, “There are three things to emphasize in teaching: The first is motivation, the second is motivation, and the third is (you guessed it) motivation.” It is strange that an experienced educator with this kind of understanding about the crucial nature of motivation for learning would be a champion of national education standards. But he was.

By the 1980’s, many education theorist and observers knew that wherever standards had been tried they tended to circumscribe curriculum and engender jejune pedagogy which stunted the intrinsic motivation to learn. In many ways, Bell is typical of a modern education reformer. He was idealistic, moral, selfless and believed fiercely in the destructive illusion that American public education was failing.

Teach for America and the ‘education entrepreneurs’ who developed the “no excuses” charter school movement also spring from that same kind of idealistic passion based on the heartfelt belief that America’s schools are failing and that they have a mission to save our country’s future.

People like Doug Lemov, Stacey Boyd and John King deeply believed that public schools were failing. With no substantial background in education (which they saw as a strength), they started ‘no-excuse’ charter schools. Deriding education theories taught by university professors, it was clear to them that the first item to fix in schools was discipline so they put children in uniforms, made many rules about everything the children did and enforced those rules harshly. In some ways, it was reminiscent of 19th century teaching.

‘Education entrepreneurs’, because they have no respect for education professionals, are doomed to reinventing the wheel. The truth is that even with the tremendous financial and political support their movement secured; the results have been mixed at best and as far as improving education practices they are abysmal failures.

The truly difficult outcome from these disciples of economist like Eric Hanushek instead of educators like John Dewey is that they are still true believers in the need to ‘disrupt’ public education because it is failing. Everything they have done and believed in about education is based on the failure illusion that they hang onto tenaciously.

The godfather of the ‘education entrepreneurs’ seems to be the MIT trained economist, Eric Hanushek. His major claim to fame is proposing value added measures (VAM) to evaluate schools and teachers. Economists often function today as modern era soothsayers. To me Hanushek is little more than a mountebank.

His VAM has been widely criticized as being statistically absurd, but that hasn’t stopped “elite” schools like MIT and Stanford from singing his hosannas. Hanushek reminds me of a description of economists in the book “An Incomplete Education“:

“Economists are fond of saying, with Thomas Carlyle, that economics is ‘the dismal science.’ As with much of what economists say, this statement is half true. It is dismal.”

Hanushek’s bad science has done a lot to advance the perfidious illusion that American schools are failing.

I too was a victim of the American schools are failing myth. I decided to become an educator in 1999. I left a successful career as a research scientist working in the recording industry to become an educator. With strong beliefs about the importance of public education, I saw myself as a talented guy who could make major contributions to righting the distressed ship of public education.

My mindset wasn’t that different than that of the ‘education entrepreneurs.’ I was not in the classroom for many months when I started to realize I might be the most inept teacher at my school. Teaching is more difficult than being a research scientist and experience in teaching is also more important. It became clear to me that I had been bamboozled. The teaching profession and the condition of public schools in San Diego County in 2002 had never been in better shape.

It is interesting the way this deception has manifested in the neighborhoods of America. In middle class and upper middle class neighborhoods, people believe the illusion but are thankful that their own neighborhood public schools are such wonderful exceptions.

Here in San Diego, parents at Torrey Pines Elementary School or Challenger Middle School or Poway High School love their schools and would fight for them. At the same time they believe that public schools in poor neighborhoods are failing. The result is that people in poor neighborhoods who don’t have the political capital to fight for their schools no longer can send their children to quality public schools.

They must send them to charter schools or drive them to another neighborhood. Poor communities are having their wonderful public institutions like the venerable Compton High School taken from them.

I will conclude with words that the great historian Arnold Toynbee wrote in his masterpiece study of five millennia of human development concerning publicly supported education. He observed that “The bread of universal education is no sooner cast upon the waters than a shoal of sharks arises from the depths and devours the children’s bread under the educator’s very eyes.” I hope this was a warning not a prophecy!