By Thomas Ultican 5/5/2020

It came as a surprise when Washington Post columnist Jay Mathews reached out to me. After indicating that he was writing a book, Mathew’s said that he had just come upon my piece “A Texas Sized Destroy Public Education IDEA.” He flatteringly wrote, “… your analysis is impressive and I want to include some of it in my book.” He also sent an article and a book chapter asking for comments.

In the article about IDEA, I had written,

“In 2016, Jay Mathews of the Washington Post rated IDEA charter high school the most challenging in the nation. Mathews rates schools by what he calls “the Challenge Index,” which takes the total number of Advanced Placement, International Baccalaureate and Cambridge tests given at a school each year and divides by the number of seniors who graduated in May or June. Charter schools that shed students without replacing them now own all of the top spots in this index; not deeply meaningful.”

Jay’s message to me was very respectful with a genuine feeling of interest in my opinion; however, we do have very different views. I will endeavor to address those differences honestly and respectfully.

The chapter he sent me is focused on his “Challenge Index” and its rationale. The article was a piece he did for his Washington Post column in November about a high school teacher who teaches AP English Language arts. He explained how that teacher came to appreciate the value of expanding AP access.

Some Personal Background

I grew up on a ranch in rural Idaho with a cowman for a father, a sheepherder for a grandfather and a school teacher for a mom. She studied teaching at Albion Normal School in the southern Idaho Mountains. Coincidentally, it is the same school attended by the force behind “A Nation at Risk,” Ronald Reagan’s Secretary of Education, Terrel Bell.

My mother often claimed that the key difference that made public education in America superior and more democratic than any other system in the world was that in America there was no high stakes testing.

In the 1960’s, before education standards and AP curricula there was an aphorism,

“American high school students are the laziest in the world. They seldom study and spend their time playing, socializing and competing in sports. Then they graduate from high school and over the summer a miracle occurs. They arrive at college to become the world’s leading scholars.”

After an engineering career in Silicon Valley as a researcher in the disc drive industry, I decided to become a teacher. During my fifteen years in the classroom, I taught advanced mathematics, conceptual physics, two flavors of AP physics and AP environmental science. In the end, I have concluded that my mother was right about high stakes testing and that the aphorism about American students captures an important reality.

Most surprisingly, I now believe that AP style college level classes in high school are bad pedagogy.

AP, IB or Cambridge Courses are Bad Pedagogy

This will probably be viewed as heresy by many of my fellow educators. It certainly would be by Will Robertson, the AP English teacher Jay wrote about in November. In 2005, Roberson’s Corbett High School near Portland, Oregon required that every student take at least seven AP classes. Robertson wrote a lengthy memo to the administration about why it was such a mistake. After three years, he realized that the students he feared would be defeated by the AP rigor were meeting the challenge and wrote another lengthy memo admitting his error.

I contend that the challenge Robertson’s students are meeting is not appropriate for their mental health and development.

A fundamental flaw in modern education reform is the push toward inappropriate curricular pacing. Kindergarten has become the new first grade where students who should be learning through play are sitting at desks doing math and language arts work sheets.

In their recent book, Let the Children Play, Pasi Sahlberg and William Doyle write, “The global education race for ‘higher standards’ at lower financial costs have turned many schools to factories that try to produce standardized products efficiently on tight schedules.” Modern education reform is developmentally inappropriate. As Gloria Ladson-Billings, Distinguished Professor in Urban Education, University of Wisconsin-Madison states, “Anyone who fully understands child development knows that children’s ‘play’ is children’s ‘work.’”

This problem is infecting all levels of k-12 education. High school is not the appropriate level for college work. Young brains need to be protected from high stress during a period of rampant hormonal changes and emotional immaturity. Vicki Abeles, the director of the documentary film “Race to Nowhere”, describes in her book, Beyond Measure, the damage modern education reform is reeking on children and our culture.

When we start talking about challenging teenagers we need to make sure those challenges are appropriate for healthy development. Two quotes from Abeles’s book strongly imply that we have moved way past appropriate.

Adele opens chapter one, “Sicker, Not Smarter”, with a quote from Saint Louis University School of Medicine professor and pediatrician Stuart Slavin:

“My personal feeling is that we are conducting an enormous and unprecedented social experiment on an entire generation of American children, and the evidence of a negative impact on adolescent mental health is overwhelming. This is particularly disturbing given the fact that having mental health problems in the teen years predisposes to mental health problems in adulthood.” 

Abeles also shared a powerful anecdote from a student named Emily:

“Junior year is supposedly the most important in high school and my effort just wasn’t going to cut it, not if I wanted to go to a decent college, and without a degree from a top university I was not going to be successful…. I had failed. All those years of late nights studying for AP classes followed by 5 AM water polo or swim practices, what would they come to? Nothing, just like me. In a world where we must excel in not one but many areas, I had not done so in any. I would rather be dead than face the years to come, sure to be filled with constant reminders of my failure. In my mind, there was only one way out.”

Last year a report in the Journal of the American Medical Association documented the significantly rising suicide rates for ages 15-19 in America.

AP and other similar programs are a net negative for the holistic growth of students and undermines their positive social development in the crucial teenage years. Just like kindergarten is not developmentally appropriate as the new first grade, high school is not developmentally appropriate as the new college.

Unlike genuine college courses taught by gifted professionals operating in a sphere of respected autonomy, these high school level college courses employ the enfeebled teach to the test methodology. An AP syllabus must be approved by College Board and in practice that means using an approved syllabus provided by College Board as a template. Worse yet, the entire course is centered on preparing for the end of year AP exam which is supported by a large test preparation industry.

I have read your opinion that “teaching to the test” is not a problem but we just disagree. I think it leads to lifeless uninspired classrooms.

The “Challenge Index”

Since I view college level work in high school as a negative, I obviously disagree with a system that rates schools higher if more of their students participate in these programs. However, I will respond to some of the points you make in your “Challenge Index” book chapter.

You definitely deserve credit for trying to come up with a method for evaluating schools that eliminates the standardized testing advantages schools in wealthy communities have. When reading about what you learned studying Garfield High in East Los Angeles, I appreciated the egalitarian motivation for the approach you invented. You wrote,

“I decided one way to draw attention to the issue was to rank high schools in a unconventional way that illuminated the hidden strengths I had found at Garfield. Instead of measuring them by state, SAT or ACT test scores, I assessed them by their success in getting less than stellar students into the most challenging courses and tests.”

As I was entering the classroom, the disparity in AP course offerings between schools in wealthier neighborhoods and those in poor and minority communities was a hot topic of discussion. What we were not discussing is whether AP, IB and other courses of their ilk were appropriate. It was assumed they were. As a new teacher, I was an enthusiastic advocate for my AP classes at the 90%+ minority populated high school where I taught.

The following quotes from the “Challenge Index” chapter highlights a problem facing public education. You write that after introducing the “Challenge Index,”

“Newsweek asked me to do another national Challenge Index list in 2000 and again in 2003. In 2005 the magazine decided to make it an annual feature, calling it “America’s Best High Schools.” It assigned two researchers to help me collect information. In 2009 the list got more than 20 million page views.”

“Principals and superintendents at a few public schools, mostly in affluent parts of New England, told me they did not want to participate. They said their schools were not accurately represented by such a simple ratio, just one number. I explained to them why this was a useful tool for parents choosing schools and policy makers analyzing achievement. I said it might not appeal to everyone, but newspaper readers liked it, as they did other newsworthy ratios like Wall Street’s Dow Jones Average or baseball’s Earned Run Average.”

The arithmetic behind the “Challenge Index” is a simple ratio of the number of college level tests taken at a school divided by the number of graduating seniors. It is an easily understood metric but like standardized testing it does not meaningfully evaluate schools. Education is an extremely complex system and a simple explanation may be popular but damagingly misleading.

The over-representation of charter schools in the “Challenge Index” is an arithmetic issue. Charter schools typically do not replace students who leave. By their senior year, the graduating class is often only a third the size it was in freshman year. Public schools do not see that big decline in class size so the smaller denominator for charter schools misleadingly engenders large “Challenge Index” ratios.

The No Child Left Behind rules, the Race to the Top rules and your “Challenge Index” all try to evaluate schools with a simple metric, but education is so complex that these simple metrics are counter-productive.

For many decades, states have had agencies send teams to schools for more than a week to evaluate every aspect of the school and write detailed reports. Here in California that work is done by the Western Association of Schools and Colleges (WASC). I have been involved in four WASC reviews. They were conducted by teams of administrators from around the state. WASC not only did a detailed evaluation of every aspect of the school but also worked collaboratively with the school to improve its roadmap for continuous improvement.

My bottom line is that simplified indexes run in popular news magazines may sell advertising but they are misleading and do damage. Many wonderful schools were erroneously deemed failures by No Child Left Behind testing. If education leaders had looked at the accrediting agency reports instead of just the simple standardized testing results, they would have never destroyed those schools operating mostly in poor minority neighborhoods. Likewise, your “Challenge Index” with its easy to understand ratio runs the risk of promoting unhealthy education practices.

Tags: AP, AP Tests, Cambridge, Challenge Index, IB, Jay Mathews