Tag Archives: CREDO

COVID Learning Loss Over-Hyped

15 Oct

By Thomas Ultican 10/15/2020

Warnings about learning losses due to the pandemic dominate education media; especially the media created and financed by billionaires. Based on a briefing by NWEA, McKinsey & Company claims “the average K–12 student in the United States could lose $61,000 to $82,000 in lifetime earnings (in constant 2020 dollars) … solely as a result of COVID-19–related learning losses.” The Hoover Institute’s CREDO warns “the findings are chilling.”

One of my favorite education bloggers, Nancy Flanagan, says it well,

“Test-data estimates, alarmist language and shady research do nothing to help us with the most critical problem we have right now: keeping kids connected to their schoolwork and their teachers. However that’s offered and as imperfect as it may be.”

The popular blogger Peter Greene goes to the essence of the issue noting:

“So why has CREDO decided to throw its weight behind this baloney? Well, the testing industry is in a bit of a stir right now. The BS Test was canceled last spring, and nobody is very excited about bringing it back this year, either. So the testing industry and their reformy friends are trying to sell the notion that students and schools and teachers are adrift right now, and the only way anyone will know how students are doing is to break out the industry’s products and start generating some revenue data.”

The Billionaire Created Echo Chamber

The first COVID-slide bang on the bell came from the Northwest Evaluation Association (NWEA) which sells MAP tests. Their computer delivered assessments of mathematics and English are given three times each school year; fall, winter and spring. The tests are not aligned to one class level so they are only partially aligned with state curricular standards.

Using data from approximately 350,000 students who took MAP tests in school years 2017-18 and 2018-19, analysts at NWEA created a paper that guessed at what the negative education effects from the school shut downs would be. The paper was published on May 27, 2020 by the Annenberg Institute at Brown University.

In the paper, NWEA stated,

“In this study, we produce a series of projections of COVID-19-related learning loss and its potential effect on test scores in the 2020-21 school year based on (a) estimates from prior literature and (b) analyses of typical summer learning patterns of five million students. Under these projections, students are likely to return in fall 2020 with approximately 63-68% of the learning gains in reading relative to a typical school year and with 37-50% of the learning gains in math.”

“Specifically, we compared typical growth trajectories across a standard-length school year to learning projections that assume students are out of school for the last three months of the 2019-20 school year.”

In other words, NWEA used data from their computer generated testing which is noisy and only reliably measures student family economic status. They massaged this noisy data with debunked growth model algorithms which amplify noise. They assumed that no education at all occurred after March 2020 and correlated the results with disputed summer learning loss research to make their guesses.

Within four days, the famous consulting firm Mckinsey & Company produced its own report based on the NWEA paper. To the NWEA material they added some of their own economic predictions based largely on the work of Hoover Institute’s Eric Hanushek. He rose to prominence producing research showing “that there is no relationship between expenditures and the achievement of students and that such traditional remedies as reducing class sizes or hiring better trained teachers are unlikely to improve matters.”

Mckinsey’s consultants focused much of their report on the damage that is sure to visit minority communities. If the virus is not contained and school is not full time in person, they claim students will lose an average of seven months of learning. And they further state, “But black students may fall behind by 10.3 months, Hispanic students by 9.2 months, and low-income students by more than a year.”

Hard to comprehend how a student falls behind more than a year in one year.

On June 5, 2020, the well known neoliberal publication The Wall Street Journal weighed in. Using the NWEA report, they claimed “remote learning” did not work.

By June 9, the billionaire funded education news outlet, The 74 Media, Inc., jumped in with Learning Losses Will Widen Already Dramatic Achievement Gaps Within Classrooms. Their claim says, “Solid data about the specific concepts each student does or doesn’t understand will be crucial.” They are saying testing is vital.

Another billionaire funded education focused publication, EdWeek, delivered Tips for Measuring and Responding to COVID-19 Learning Loss.”

Based on the NWEA data the education publishing company Houghton Mifflin Harcourt published “What Schools Can Do To Make Up For COVID-19 Learning Loss.Market Insider reported that the publishers of I-ready, Curriculum and Associates, says, “According to the findings, while ‘COVID slide’ can be significant, the effects differ markedly based on a range of variables, including age, race, and income level.”

All of these claims are based on one very faulty paper produced by NWEA in May and this is only a sample of what has been published.

The 74 Media, Inc. was founded by the former NBC and CNN news anchor Campbell Brown. It was funded by the Walton Family Foundation, the Gates Foundation and others. Brown was erroneously convinced that teachers unions were protecting sexual predators and her husband Dan Senor, was on the board of Michelle Rhee’s anti-teachers union organization StudentsFirst. To this day, the publication adheres to its anti-teachers union foundation and promotes public school privatization.

The latest article in The 74 about the “COVID slide” along with a report from the Hoover Institute’s Center for Research on Education Outcomes  (CREDO) illustrates the billionaire financed media empire echo effect.

The 74 article says,

“Data released last week by the Center for Research on Education Outcomes at Stanford University provided a sobering estimate of the learning loss caused by school closures: Across 19 states, it ranged from a third of a year to a year in reading, and from three-fourths of a school year to 232 days in math. The report suggested frequent assessment going forward and said new approaches to teaching will be needed to ‘plot a recovery course.”’

This data is based on the much criticized CREDO days of learning metric. The report is saying that students can lose 232 days of math learning in 180 days. It is a specious argument; however their real agenda seems to be advocating “frequent assessment going forward.”

The 74 continues, For the report, CREDO worked with NWEA, a nonprofit assessment organization, to build on earlier estimates of the impact of school closures and the limitations of virtual instruction on student learning.”

CREDO added a projection of individual student future exam results to the NWEA data. It is not all that different than the exam score scandal in England this spring. The British government used projected computer-generated scores to replace exams that were canceled because of the COVID-19 and 40% of their students saw grades tumble. The bottom line is these projections are not that good.

The 74 reports: ‘“The takeaways from this analysis are upsetting, but needed,’ said Jim Cowen, executive director of the nonprofit Collaborative for Student Success, which advocates for high academic standards and holding schools accountable for student progress.” Cowen is also quoted recommending “those annual tests remain the best tool to inform accountability systems, school report cards, and continuous improvement efforts over the long-term.”

The bottom of the Collaborative for Student Success web page reveals, “The Collaborative for Student Success is a project of the New Venture Fund.” If unfamiliar with New Venture Fund, the article Organized to Disrupt details the massive billionaire pro-school privatization funding they receive.

Paige Kowalski, executive vice president of the Data Quality Campaign is quoted by The 74. The article states, “Kowalski stressed the importance of conducting assessments in the spring of 2021 and ‘getting at the heart of the data’ demonstrating why students might not participate, such as school buildings still being closed or parents opting out.”

The Data Quality Campaign lists as partners almost every organization in which billionaires working to privatize public education invest.

CREDO’s Conclusions

CREDO: “First, the findings are chilling – if .31 std roughly equals a full year of learning, then recovery of the 2019-2020 losses could take years.”

If the report was meaningful and learning could be measured in days the findings might be chilling. However the report is a gross use of arithmetic and learning cannot be measured in days. When days of learning related to standard deviation change is used in a study, the study is meaningless.

CREDO: “Second, the wide variation within states (and often within schools) means that conventional models of classroom-based instruction – a one-to-many, fixed pace approach — will not meet the needs of students. New approaches must be allowed to ensure high quality instruction is available in different settings, recognizing that different skills may be needed for the different channels.”

Here it appears CREDO is putting in a plug for competency based education (CBE) delivered by computers. CBE has a history of failure going back to the early 1970’s when it was known as mastery education or as teachers called it “sheets and seats.”

CREDO: “Third, the need for rigorous student-level learning assessments has never been higher.”

This is the apparent purpose of the paper; selling testing. People are starting to realize standardized testing is a complete fraud; a waste of time, resources and money. The only useful purpose ever for this kind of testing was as a fraudulent means to claim public schools were failing and must be privatized.

CREDO: “Fourth, the measures of average loss and the range around it immediately call into question the existing practice of letting communities plot their own path forward.”

Here CREDO has joined with the billionaire promoted call to end democracy and local control of schools. It is UN-American and disgusting. Even the Hoover Institute should be revolted. After all, they used to be champions of the American ideal.

This is not the first time America has faced a crisis and schooling was disrupted. There was the Spanish Flu, World War II, Segregation battles in the south, catastrophic storms, etc. Public school has been the one institution that continually rose to the occasion and taught the children.

Today, without much support from the federal government, public schools are once again stepping up to the challenge. Millions of cyber capable devices have been distributed, internet hotspots have been created and teachers are adapting to teaching on line. It is not wonderful and students are especially missing the social aspect associated with in person school, but it has value and students are learning.

The COVID-slide is about undermining public schools and is not a real phenomenon.

CREDO’s New Study Biased against Public Schools

14 May

By Thomas Ultican 5/14/2020

The Center for Research on Education Outcomes (CREDO) started releasing the results of its new Cities Study Project in mid-2019. It is not a coincidence that the cities chosen for the study have long been targeted for public school privatization. The ten cities selected are: Indianapolis; Baton Rouge; Camden; Kansas City; Memphis; New Orleans; Oakland; St. Louis; San Antonio; and Washington DC. This CREDO study is even more opaque and biased than its previous efforts.

Who is CREDO?

Hanushek and Raymond

Husband and Wife Team Who Founded CREDO

In the early 1980s, Margaret (Macke) Raymond was completing a lengthy graduate school agenda at the University of Rochester, a relatively small private university in Rochester, New York. She garnered an MS of public policy in 1980, a community medicine MS in 1982, an MA of political science in 1983 and finished with a PhD in political science in 1985. From 1985 to 2000 she ran Raymond Consulting and worked a few years in the telecommunications industry.

At that same time, Eric Hanushek was Professor of Economics and Political Science at Rochester University. The former Air Force cadet had earned a doctorate of Economics from the Massachusetts Institute of Technology in 1968. It was Hanushek’s 1981 paper “Throwing Money at Schools” that put him on the watch list of right leaning philanthropies and institutions. In his notice making missive he stated,

“The conventional wisdom about public schools is that they face serious problems in terms of performance and that improving schools requires additional money. However, the available evidence suggests that there is no relationship between expenditures and the achievement of students and that such traditional remedies as reducing class sizes or hiring better trained teachers are unlikely to improve matters.”

In a 1981 Ed Week commentary referencing this paper, Hanushek points to SAT testing as the gold standard for judging school performance. In complete accordance with the old aphorism, “to a man with a hammer all problems look like a nail,” Hanushek brags, “Advanced statistical techniques are employed to disentangle the influences on achievement of schools and teachers from those of other factors such as family backgrounds and student abilities.”

A 1999 announcement from Rochester University said, “The Center for Research on Education Outcomes has been established at the University of Rochester’s Wallis Institute of Political Economy…” In the same posting, it was revealed, “Two foundations have committed $1.25 million to fund a three-and one-half year initiative to address the current shortage of evaluation research in education policy matters.” CREDO never made the name of the two foundations public, but a knowledgeable academic says one of them was the Walton Family Foundation.

The announcement listed two employees of the new center, Eric Hanushek and Margaret Raymond. Hanushek was listed first but Raymond was cited as the founding Director.

CREDO moved from the University of Rochester to Stanford University’s Hoover Institute in July, 2000 which made networking in conservative circles much easier.  In CREDO’s 2nd year report, they stated that moving to the Hoover Institute brought them many new contacts including the New Schools Venture Fund, the District of Columbia Charter School Board, the Teacher Union Reform Network and others.

A description of the Hoover Institute from Source Watch says,

“The Hoover Institution is influential in the American conservative and libertarian movements, and the Institution has long been a place of scholarship for high profile conservatives with government experience. A number of fellows have connections to or positions in the Bush administration, and other Republican administrations. … Other fellows of the Institution include such high profile conservatives as Condoleezza Rice, George Shultz, Thomas Sowell, Shelby Steele, and Edwin Meese.”

Macke Raymond’s 2015 Hoover Institute Fellow’s profile says in part, “In partnership with the Walton Family Foundation and Pearson Learning Systems, Raymond is leading a national study of the effectiveness of public charter schools.” 

Valerie Strauss of the Washington Post has pointed out that Eric Hanushek

“… a Hoover economist was a pioneer in creating systems that evaluate teachers by student standardized tests, a method that many assessment experts say should not be used in the high-stakes ways that school reformers are using them. He is often cited in CREDO studies as a ‘principal investigator.”’

Discredited and Biased

The Forbes commentator, Peter Greene, wrote about Eric Hanushek for his popular blog Curmudgucation:

“Now when Hanushek says that teachers make a huge difference, he is obliquely referencing his own crazy-pants assertion that having a good first grade teacher will make you almost a million bucks richer over your lifetime (you can also find the same baloney being sliced by Chetty, Friedman, and Rockoff). Both researchers demonstrate their complete lack of understanding of the difference between correlation and causation.”

Greene also shares the following graphic that clearly highlights the difference between correlation and causation.

Divorce caused by Margarine consumption

Does Margarine Consumption Cause Divorce?

Business writer Andrea Gabor states that CREDO studies which compare charter schools with public schools start with two key assumptions “A) That standardized-test scores are an adequate measure of school quality and B) that creaming in charter schools does not exist.”

With regards to assumption ‘A’, using standardized testing for this purpose has been shown faulty from studies dating back to the eugenics movement (which originated high stakes standardized testing) to recent works debunking them for mistaking correlation versus causation and for not being able to compensate for the problem of error.

As for assumption B, there is no doubt that most charter schools push out and avoid students that are classified as special education, language learners or discipline problems. The data proving that is in state enrollment reports wherever charter schools exist.

In the new research labled “Cities Studies Project”, the Technical Appendix says the reports uses growth models but doesn’t share which of the many growth models it uses. It also says,

“In our study, scores for all these separate tests are transformed to a common scale. All test scores have been converted to standardized scores to fit a ‘bell curve’, in order to allow for year-to-year computations of growth.”

The Education Growth Model Handbook lists seven types of growth models in general use and their requirements. Most growth models require vertical scales but that does not seem possible with CREDO’s use of multiple tests many of which are not vertically scaled. Their mathematical conversions add another locus of error. Growth models have proven to be unstable and have never been satisfactorily validated.

The research methodology used in the “Cities Studies Project” appears to be the same as that used in CREDO’s 2015 Urban Charter School Study Report on 41 Regions.” A particularly troubling practice employed then and apparently still being used is the “virtual twin” method which creates a pro-charter school bias.

Professor Andrew Maul of UC Santa-Barbara reviewed the 2015 study for the National Education Policy Center. He noted the CREDO method does not compare charter school performance to actual public schools; rather it creates mathematical simulations. Maul described the “virtual twin” schema employed to develop a “virtual control record.” He reports,

“CREDO’s approach to this estimate is the construction of a ‘Virtual Control Record’ (VCR) for each student in a charter school, obtained by averaging together up to seven students in “feeder” public schools (i.e., those schools whose students transfer to charters) with the same gender, ethnicity, English proficiency status, eligibility for subsidized meals, special education status, grade level, and a similar score from a prior year’s standardized test (within a tenth of a standard deviation) as the specified charter student.”

Maul adds, “The study’s “virtual twin” technique is insufficiently documented, and it remains unclear and puzzling why the researchers use this approach rather than the more accepted approach of propensity score matching.”

The stipulation that “virtual twins” come from “feeder schools” biases the study in favor of charter schools. Andrea Gabor explained that in practice, CREDO used less than five students transferring to a charter school as the cutoff for using a particular public school’s data. She reports that the, “study excludes public schools that do NOT send students to charters, thus introducing a bias against the best urban public schools, especially small public schools that may send few, if any, students to charters.” Gabor gave the example of two well regarded New York title-1 schools, Global Technology Preparatory and West Side Collaborative which were excluded. They are noted for scoring well on testing, but did not meet the transfer criteria yet easily matched the required demographics.

The CREDO study is singularly focused on test results as determinate of school quality and ignores other advantages of public schools. It is a well known fact that many charter school systems like IDEA and Success Academy spend an inordinate amount of time teaching to and preparing for standardized tests. To these criticisms, Professor Mark Weber of Rutgers University adds a few more observations:

“Cities Studies Project” Technical Appendix states,

“To assist the reader in interpreting the meaning of effect sizes, we include an estimate of the average number of days of learning required to achieve a particular effect size. This estimate was calculated by Dr. Eric Hanushek and Dr. Margaret Raymond based on the latest (2017) 4th and 8th grade test scores from the National Assessment of Educational Progress (NAEP).”

Converstion to Days of Learning

The CREDO Days of Learning Conversion Table from “Cities Studies Project”

This metric seems to have been created with next to nothing validating it. Mark Webber quoted the psychometrician Michael T. Kane,

“The 2015 study (p. 5) cites a paper published in Education Next (Hanushek, Peterson & Woessmann, 2012) that asserts: “On most measures of student performance, student growth is typically about 1 full std. dev. on standardized tests between 4th and 8th grade, or about 25 percent of a std. dev. from one grade to the next.” (p. 3-4) No citation, however, is given to back up this claim: it is simply stated as a received truth.”

CREDO tells us that Hanushek and Raymond did something with NAEP data from 2017 but still do not offer any justification for the conversion. It appears at best to be sloppy science and the headlines engendered from it are nothing short of propaganda.

Using CREDO Claims to Sell Privatizing Public Schools

Neerav Kingsland the Managing Partner of the City Fund posted to his Blog last July when the first results from “Cities Studies Project” arrived explaining,

“Last year, Arnold Ventures commissioned CREDO (out of Stanford University) to study the effects of charter, innovation, and traditional schools in select cities across the country.”

“Most of the cities included in the study were cities where Arnold Ventures (and now The City Fund) have partnered with local leaders to expand high-quality schools.”

“Camden’s city level effects are large.”

“In just two years, scores are up ~.15 standard deviations in math and ~.05 standard deviations in reading (compared to similar schools across the state).”

The reality is those changes are very small. Noise in the data is a better explanation than awesome charter schools for these tiny differences.

In Indianapolis, the CEO of The Mind Trust, Brandon Brown, just wrote an Indy Star opinion piece stating,

“A 2019 study from Stanford University found that students who attend Innovation Network Schools achieve the equivalent of 53 additional days of learning in English and 89 additional days of learning in math each year when compared to their traditional public school peers. This equates to several years of additional learning during the span of a K-12 academic career, and the gains are largest for students of color and students from low-income backgrounds.”

In the billionaire financed effort to privatize public education, CREDO has become their source for data proving things like smaller class sizes and teacher professionalism are not important. The “Cities Studies Project” commissioned by an organization intent on privatizing public schools through promoting the portfolio management scheme – The City Fund – is biased toward the privatization agenda. Rather than shining the light of scholarly work on education policy, it obscures reality with obfuscation.