Quote of the Day — Stunning Illogic and False Hope from the New York Times

The Bush administration allowed states to phony-up statistics on everything from graduation rates to student achievement to teacher training and state education standards. As a result, the country has yet to reach not only the goals that were clearly laid out in the law but also farsighted education reforms dating to the mid-1990s. (emphasis added)

New York Times, “A New Day for School Reform,” editorial, February 21, 2009.

There may be some truth to the cause and effect on teacher training, but the implied idea that the failures of No Child Left Behind are due to  setting standards (curricular and Adequate Yearly Progress) too low is illogical and reinforces multiple flaws in the NCLB.

Some asides at this point.  I want to be clear that communicating high expectations to students in all contexts while giving them the support they need to meet those expectations is good policy.  Changing state standards and cut scores at best comprises a very, very small part of this concept and at worst leads to shaming and other counterproductive punishments.  Better — not necessarily higher — curricular standards do have a place in reform.

First, standards in practice mean standardized tests and standardized tests are very limited as assessments and even more limited as a means of improving education.  To be fair, there is some language in the stimulus package (the subject of the quoted editorial) that may induce a move away from standardized tests (see below).

Second, and most importantly, the whole notion that lax standards are the biggest problem in education defies logic and the historical record.

In terms of logic, just ask yourself if the way to improve archery scores is to use smaller targets.  If they can’t hit the larger target, how will they hit a smaller target?

As to the history, here is the data for Wisconsin under the current system:

Year	# Schools Failed AYP	# Districts Failed AYP
2007-8	153	4
2006-7	92	2
2005-6	87	1
2004-5	49	1

This graphic tells us about the history and projected future (more here).

The current standards have resulted in clear trend of increasing failure to meet those standards, a trend that is projected to increase with current standards.

Some quotes from “How Feasible is Adequate Yearly Progress (AYP)? Simulations of School AYP “Uniform Averaging” and “Safe Harbor” under the No Child Left Behind Act” by Jaekyung Lee may help clarify.

It does not appear to be feasible for many schools across the nation to meet the current AYP target within its given 12-year timeline. It is not realistic to expect schools to make unreasonably large achievement gains compared with what they did in the past. Many schools are doomed to fail unless drastic actions are taken to modify the course of the NCLB AYP policy or slow its pace. (emphasis added)

When a majority of schools fail, there will not be enough model sites for benchmarking nor enough resources for capacity building and interventions. This situation can raise a challenging question to the policymakers: is it school or policy that is really failing? There is a potential threat to the validity of the NCLB school accountability policy ultimately if such prevailing school failure occurs as an artifact of policy mandates with unrealistically high expectations that were not based on scientific research and empirical evidence. (emphasis added)

An identified problem with NCLB is that standards are unrealistically high, the New York Times’ solution, raise the standards.  Stunning illogic.

This is the kind of “harder is better” mentality reflected in the Pangloss Index and expected from people like the Walton and Bradley Foundation funded  Thomas B. Fordham Institute, not “the paper of record.”

Later in the editorial, the assessment reform potential of the stimulus bill is touted:

States will also be required to improve academic standards as well as the notoriously weak tests now used to measure achievement — replacing, for instance, the pervasive fill-in-the-bubble tests with advanced assessments that better measure writing and thinking.

This seems to be a gross overstatement.  Here are the relevant parts of the stimulus bill:

(4) STANDARDS AND ASSESSMENTS.-The State-
(A) will enhance the quality of the academic assessments
it administers pursuant to section 1111(b)(3) of the
ESEA (20 U.S.C. 6311(b)(3)) through activities such as
those described in section 6112(a) of such Act (20 U.S.C.
7301a(a));
(B) will comply with the requirements of paragraphs
(3)(C)(ix) and (6) of section 1111(b) of the ESEA (20 U.S.C.
6311(b)) and section 612(a)(16) of the IDEA (20 U.S.C.
1412(a)(16)) related to the inclusion of children with disabilities
and limited English proficient students in State
assessments, the development of valid and reliable assessments
for those students, and the provision of accommodations
that enable their participation in State assessments;
and
(C) will take steps to improve State academic content
standards and student academic achievement standards
consistent with section 6401(e)(1)(9)(A)(ii) of the America
COMPETES Act.

A and C  send us to the two prior acts, with vague “such as” language in A.  Here is the section cited in A:

(1) To enable States (or consortia of States) to collaborate with institutions of higher education, other research institutions, or other organizations to improve the quality, validity, and reliability of State academic assessments beyond the requirements for such assessments described in section 1111(b)(3).

(2) To measure student academic achievement using multiple measures of student academic achievement from multiple sources.

(3) To chart student progress over time.

(4) To evaluate student academic achievement through the development of comprehensive academic assessment instruments, such as performance and technology-based academic assessments.

and the section cited in C:

(ii) identifying and making changes that need to
be made to a State’s secondary school graduation
requirements, academic content standards, academic
achievement standards, and assessments preceding
graduation from secondary school in order to align
the requirements, standards, and assessments with
the knowledge and skills necessary for success in academic
credit-bearing coursework in postsecondary education,
in the 21st century workforce, and in the Armed
Forces without the need for remediation;

I certainly don’t see a requirement to end “fill-in-the-bubble tests” here.  I see some good but weak language opening the door to multiple assessments, some possibility of better assessments in general and buzz words about the “21st century workforce.”  I also have not seen anything in Wisconsin’s plans for the stimulus money that indicates that the WKCE will be gone anytime soon (since the contract requires two-year notice be given, I don’t see that long awaited day being pushed up).

This editorial is unfortunately typical of the confusion on education policy in our media and consequently in our society.  Education policy can be confusing.  This makes the role of the press even more critical and the failures of logic and accuracy like those in the Times editorial more damaging.

Thomas J. Mertz