Tag Archives: PISA

Time to Leave International PISA Testing

20 Feb

By Thomas Ultican 2/20/2024

February began with Progressive Policy Institute and The 74 teaming up for a Webinar on PISA math test results, declaring “Historically Underperforming PISA Scores are a Call to Action”. 

Dr. Yong Zhao, Foundation Distinguished Professor of Education at University of Kansas and among his many accolades, an appointed Professor of Educational Leadership at University of Melbourne, has a different view, stating:

“Since 2000, our scores on PISA have barely changed. While there’s much chatter about learning from other systems, it has not happened. There is no reason that the U.S. should continue its participation in PISA.”

Zhao sees standardized testing as undermining student creativity. PISA tests stress cognitive skills while noncognitive skills are more related to creativity and entrepreneurship. His book, Who’s Afraid of the Big Bad Dragon, describes how in 605 AD the Chinese government developed a testing system known as “keju” to select people for prestigious government positions. That system is now blamed for stifling creativity and scientific development. When westerners showed up in their ships, guided by compasses and using gunpowder, both invented in china, the Chinese could not defend themselves. Zhao writes, “In fact, the keju system has been held responsible for the decline of the Chinese empire.” (Big Bad Dragon page 35)

PISA 2022 Testing Results

A quick peek at PISA’s data presentation reveals average scores in everything have gone the wrong direction since bankers from the Organization of Economic Co-operation and Development (OECD) got involved. Furthermore, their baseless use of translating test scores into time is fraudulent.

Why would anyone pay attention to their views on education?

Progressive Policy Institute

Going into the 1984 Democratic convention, several politicians, not happy with the party’s direction, met in a San Francisco hotel room but did not take action. After Walter Mondale’s thumping by Ronald Reagan, the Democratic Leadership Council (DLC) was created based on a plan developed by Al From who believed that being independent from the Democratic Party would allow them to be more “entrepreneurial”. Current PPI Director, Will Marshall, was hired to be policy director (Left Behind Pages 43-45).

To drive their policy strategy, From and Marshall decided to establish a think tank. Seeing the way Heritage Foundation, American Enterprise Institute and Cato Institute had driven the conservative revolution in the 1970s and 1980s, they created a new idea-generating center called Progressive Policy Institute (PPI) (Left Behind Pages 112-113).

Early on, PPI came out with articles opposing minimum wages which many people saw as anti-union. Later, a PPI fellow, David Osborne, famously campaigned for an entrepreneurial government to meld public and private to maximize productivity and effectiveness. Lily Geismer said, “Osborne developed these ideas into a book cowritten with Ted Gaebler, called Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector; which was released in 1992” (Left Behind Page 117).

Geismer continued:

“Under Osborne’s guidance, the DLC became one of the first political organizations to explore charter schools as a means of improving public education. Charters, along with the other programs, became a critical part of the new approach that DLC promised it would provide as the nation was starting a new decade.” (Left Behind Page 118)

It is no surprise to see PPI joining with The 74 to trash public education. Their “third way” agenda has a lot in common with the GOP driven school choice and anti-labor agenda.

For this Webinar, the PPI expert panel included Dr. Peggy G. Carr, Commissioner of the National Center for Education Statistics (NCES) in the U.S. Department of Education, Andreas Schleicher, Director of the Directorate of Education and Skills at the OECD and Jonathan A. Supovitz, Professor in charge of Organizations, Leadership, and Systems Division at University of Pennsylvania, Graduate School of Education. The moderator was PPI’s Tressa Pankovits, Co-Director of Reinventing America’s Schools.

Peggy Carr served in NCES for more than 20 years before Biden appointed her Commissioner. She is undoubtedly a gifted mathematician but knows little about America’s classrooms. Carr recently made headlines when the charter school she founded, Children’s Village Academy in North Carolina, was charged with paying exorbitant interest rates on a 15-year old loan she gave the school.

Andreas Schleicher is a fine mathematician but, just like Peggy Carr, he knows nothing about good education. His personal PISA involvement blinds him to how little value standardized testing is for evaluating student learning.

I really don’t want to say anything mean about moderator Tressa Pankovits because she is a fellow Aztec, unfortunately her resume reveals no training or experience as an educator. Still she is PPI’s “Co-Director of Reinventing America’s Schools, which researches innovations needed to create a 21st century model for public education that is geared to the knowledge economy.” It is hard to understand how her background qualifies for the position. Before she got this job she worked 10 years for Paul Vallas who hurt public schools in Chicago, Philadelphia, New Orleans and Bridgeport. He also has no training or experience as an educator.

The only educator in this group is Jonathan A. Supovitz. His values look shaky, based on the company he keeps.

PISA and the OECD

OECD was formed in 1961 as a follow-on to the Marshall plan, run by bankers and economists. Market Business states:

“All OECD member states claim to be market economies that are committed to democracy. The organization says it provides a platform where they:

  • Share and compare policy experiences.
  • Identify good practices.
  • Coordinate members’ international and domestic policies.
  • Seek answers to common problems.”

In 2014, German writers, Sija Graupe and Jochen Krautz, wrote “From Yardstick to Hegemony”, using OECD documents:

“The OECD Conference documentation of 1961 declares unequivocally: ‘It goes without saying that the educational system must be an aggregate of the economy, it is just as necessary to prepare people for the economy as real assets and machines. The educational system is now equal to highways, steel works and chemical fertilizers’”.

“What this unrealistic worldview setting in turn impedes is any critique or will to change because rather than being understood by the public as a theoretical construct it is, according to the neoliberal economist August Hayek, accepted by most as an immediately evident truth. Whether they are true or false, economic theories and all assessments based on these (such as PISA) determine reality. … As long as people believe having more PISA points is better than less in order to be successful economically they will, of course, do everything they can to acquire more. Education is then forced to uncritically yield to the pressure of comparative assessment, even if it is based on pure assertion.”

The meaningfulness of PISA testing results are pure assertion based on bad science.

Noel Wilson’s famous 1997 peer-reviewed thesis, “Educational Standards and the Problem of Error” fundamentally states the error involved in educational testing is so great that validity is compromised. In other words, standardized tests are not refined enough to make more than generalizations and are bunk for measuring learning or teaching.

Yong Zhao shared research showing an inverse relationship between test scores and economic development:

“In fact, a correlational analysis done in 2007 showed a negative correlation between international test scores and economic development (Baker, 2007). That is, countries with higher scores in the first international study did worse than countries with lower scores.” 

Because America does not filter students from the academic system before high school, tested populations do not compare well internationally. However, since 2010, in the yearly International Math Olympiad, the USA team has come in first four times and never finished lower than fourth … out of over 100 entrants.

So why are we still bothering with meaningless PISA exams???

Tags: , , , , , , , ,

Panic! Pandemic Learning Loss!

17 Dec

By Thomas Ultican 12/17/2023

Wal-Mart family’s propaganda rag, The 74, says those not hysterical over learning loss wear rose-colored glasses and damaged students are doomed, losing billions in future earnings, if nothing is done now. Their major recommendations are frequent testing, high-dose tutoring and tough grading. Unsurprisingly these lead to more corporate profits.

A gathering at the Aspen Institute asserted the dire situation.  Jens Ludwig,  University of Chicago economics professor, said, “We do not have our hair on fire the way it needs to be.” The other members of this panel were Nat Malkus of the American Enterprise Institute, T. Nakia Towns of Accelerate and Melissa Kearney of the Aspen Institute.

There was a strong smell of corporate education bias in the air. The Aspen Institute was the creation of corporate leaders and largely funded by foundations, such as, Carnegie Corporation, Rockefeller Brothers Fund, Gates Foundation, Lumina Foundation, and Ford Foundation. American Enterprise Institute is a center right research group that grew out of the American Enterprise Association which formed in 1938 to fight Roosevelt’s New Deal. Accelerate’s CEO is Michelle Rhee’s former husband, Kevin Huffman, also a founding partner at The City Fund. Listed funders of Accelerate are Gates Foundation, Arnold Ventures, Walton Family Foundation, Overdeck Family Foundation and Ken C. Griffin.

Testing Declines were Universal

The 74 claims:

“Parents expressed little concern about lasting damage from the pandemic and typically thought their children were doing well in school — a view that researchers say is belied by dismal state and national test scores.”

“The 2022 National Assessment of Educational Progress showed historic declines in math and flat performance in reading.” (Emphasis added.)

Plotted by NAEP from 2022 Testing Data

The 2022 8-point drop in mathematics scores was unusually large. In the spring of 2020, schools throughout America shut down and most of them did not reopen in class until fall 2021. If there were not a drop in testing scores, the NAEP assessment would have been meaningless.

The 74 further notes that a recent release of international scores shows U.S. students dropped 13 points in math between 2018 and 2022. Their linked article noted that many other countries had worse drops.

Because America does not filter students from the academic system before high school, the tested population does not score as well internationally. However, since 2010, in the yearly International Math Olympiad, the USA team has come in first four times and never finished lower than fourth … out of over 100 entrants.

The Organization for Economic Co-operation and Development (OECD) created and administers the Program for International Student Assessment (PISA). The table shown is from the 2022 math exam given to 15-year olds and score changes since the last administration in 2018. As normal, the US scored in the lower half of OECD countries but did improve one step from 2018.

Advocating More Standardized Testing

Fordham Institute has documented a growing discrepancy  between grade point averages and standardized test scores. TNTP produced a report showing an increase in B grades since the pandemic. The basic argument of corporate reformers is that parents should not trust public school grades and more standardized testing is required.

Policy Analysis for California Education (PACE) and California Office of Reform Education (CORE) sound like official governmental organizations but they are not. Billionaires created these institutions for the express purpose of undermining and controlling public schools. In 2019, PACE was determined to sell California on growth models to evaluate schools. University of Southern California (USC) Professor of Education Policy, Morgan Polikoff, produced a policy brief for PACE stating:

“Based on the existing literature and an examination of California’s own goals for the Dashboard and the continuous improvement system, the state should adopt a student-level growth model as soon as possible. Forty-eight states have already done so; there is no reason for California to hang back with Kansas while other states use growth data to improve their schools.”

Polikoff seems to be a sincere academic but growth models do not do well when scrutinized.  Jesse Rothstein, professor of public policy and economics at University of California, Berkeley, ran a verification test and found, “these models indicate large ‘effects’ of 5th grade teachers on 4th grade test score gains.” A verification test run at the University of California Davis, showed that teachers affect student height…??? “Using a common measure of effect size in standard deviation units, we find a 1σ increase in ‘value-added’ on the height of New York City 4th graders is about 0.22σ, or 0.65 inches.”

An article by Linda Hammond Darling notes the instability of VAM result: “A study examining data from five school districts found, for example, that of teachers who scored in the bottom 20% of rankings in one year, only 20% to 30% had similar ratings the next year, while 25% to 45% of these teachers moved to the top part of the distribution, scoring well above average.”

Standardized testing and growth models are as likely to be misleading as illuminating. On the other side of the coin, high school grades are more predictive of college success than standardized testing. Public school grades, though fraught with issues, are much more reflective of student progress and potential.

Dan Goldhaber, director of the CALDER Center at the American Institutes for Research, and Polikoff are among the “experts,” urging educators to make test score data a much larger focus of conversations with parents. Polikoff sees separation between parents and the nation’s education scholars as part of a larger anti-testing movement that started brewing long before the pandemic. The pandemic pause on state assessments and accountability sparked a renewed push to limit the number of tests and try different models.

“There’s just very close to zero constituencies advocating for tests or that they matter,” Polikoff said. Republicans “want only unfettered choice” while the left is not defending the usefulness of tests “to ensure educational quality or equity.” He says the backlash against testing has come “at the worst possible time given the damage that’s actually been done.”

Polikoff and his USC team recently published a report, based on interviews with 42 parents over the past two+ years:

“One of the clearest findings from our interviews is that caretakers, when making judgments about students’ performance, overwhelmingly rely less on standardized test scores than they do grades, other school-reported measures of student progress, and their own observations of their children’s work and work ethic.” (Page 15)

“A final insight our data provides into the parent-expert disconnect is that caretakers often, and very explicitly, noted that children are resilient.” (Page 20)

Observation and Conclusion

Noel Wilson’s famous 1997 peer-reviewed article, “Educational Standards and the Problem of Error” fundamentally states the error involved in educational testing is so great that validity is compromised. In other words, standardized tests are not refined enough to make more than great generalizations. They are bunk for measuring learning or teaching.

Clearly people like Professors Polikoff and Goldhaber believe in these tools. It is likely they embrace testing because they are good at math and strongly desire tools that provide clear, unbiased conclusions. Unfortunately, they have grabbed onto an illusion.

Parents are correct when they say “children are resilient.” What students and schools need now is to be left alone to do their job. The COVID pandemic was traumatic for us all and it may take two or even three years for student recovery. They will, unless we continue mindlessly over-testing and forcing some sort of academic acceleration.

Profiteers see this as a business opportunity. Protect the children and let kids be kids.

They will be fine!

Tags: , , , , ,