Thomas J. Mertz
Category Archives: Equity
In December I announced my candidacy for the Madison Metropolitan School District Board of Education, Seat #5 (announcement below). There will be a primary election February 19, 2013 and the general is April 2, 2013. I hope I have earned the support of the readers of this blog. You can find out more about my campaign, endorse, volunteer and donate at MertzforMadison.com. I am not sure if I will be doing any blogging during the campaign, but if I do things directly related to the Madison schools will be posted at MertzforMadison.com, and anything posted here at AMPS will be more about state and national matters.
Prepared, Progressive, Passionate
I am excited to announce my candidacy for the Madison Metropolitan School District Board of Education, Seat #5.
Our public schools are the backbone of our community, the wellspring of our democracy, and the best means we have of providing a better future to all our children. As a parent, scholar, advocate, activist and organizer, I have worked with parents, professors, students, school boards, administrators, legislators, educators, and their unions to better understand and strengthen public schools. I don’t think there has ever been a time when the challenges to our schools have been greater. I want to help Madison meet these challenges by serving on the Board of Education.
I have stood against the pressures of privatization, worked against the expansion and misuse of standardized testing, and have fought for adequate and equitable funding based on the idea that all of our students deserve broad and rich opportunities.
These struggles will continue and expand. As Madison prepares to welcome a new Superintendent, I see opportunities to do more than react. Madison is a community and district where we have the means and the will to show that diverse public education can live up to its promises. To do this we must honestly assess those failings illustrated by the achievement gaps, but also listen to voices of our classrooms and community to understand what is working and build from our strengths.
None of this will be quick and none of this will be easy. I ask for your help and support. Visit www.mertzformadison.com to endorse, donate, or volunteer; and “like” the TJ Mertz, Madison School Board, Seat #5 Facebook page to keep updated.
Thomas J. Mertz
How one person’s abilities compare in quantity with those of another is none of the teacher’s business. It is irrelevant to his work. What is required is that every individual shall have opportunities to employ his own powers in activities that have meaning.
Democracy and Education, 1916
The current “accountability” madness is almost all based on misusing metrics of questionable value to make comparisons among students, among teachers, among schools, among districts, among nations (see here and here for two recent manifestations). If we are going to be “holding people accountable,” I’d prefer the metric be whether they are providing all students with the “opportunities to employ his [or her] own powers in activities that have meaning.”
Thomas J. Mertz
The WKCE testing and related assessments are scheduled for next week in the Madison Metropolitan School District schools (full schedule of MMSD assessments, here), but your child doesn’t have to be part of it. You can opt out. Families with students in grades 4,8, & 10 have a state statutory right to opt out of the WKCE; I have been told that it is district practice to allow families to opt out of any and all other, discretionary, tests. We opted out this year. In order to opt out, you must contact your school’s Principal (and do it ASAP, (contact info here).
The WKCE does your child no good. Just about everyone agrees that even in comparison to other standardized tests, it is not a good assessment. Because results are received so late in the year, it isn’t of much use to target student weaknesses or guide instruction. There are no benefits for students.
There are also no benefits for schools and the district, and some potential for harm. The WKCE is central to the new Wisconsin “Accountability” system (discussed here) and will be part of the new “Educator Effectiveness” system, being implemented. Both of these are built on the — likely false (see: “Snookered by Bill Gates and the U. S. Department of Education“) — promise of “SMARTER Balanced Assessments,” but because the Report Cards include a “growth measure” and the educator evaluations include a Value Added component, the WKCE will be part of the calculations for at least two more years (this will be accomplished by pretending that the WKCE is essentially the same as the new test, which in fact it likely is, in that it will no doubt measure scocio-economic status better than it measures anything else).
A large-scale, summative assessment such as the WKCE is not designed to provide diagnostic information about individual students. Those assessments are best done at the local level, where immediate results can be obtained. Schools should not rely on only WKCE data to gauge progress of individual students or to determine effectiveness of programs or curriculum (emphasis added).
But in the mania to compare and rate and evaluate that is the new Status Quo, this is almost exactly how the WKCE is being used. Not the WKCE alone, but in the Report Cards the WKCE dominates and in the Educator Evaluation the WKCE test scores may be decisive (test scores only account for a small part of the evaluations, but if the other portions show little variance, the test score portion will be determinative). No good can come from this and the mis-impressions created – about districts, schools, educators and students – are harmful, if only because they create confusion and make it more difficult to have productive policy deliberations.
I would be remiss if I didn’t acknowledge that opting out can have consequences for schools and districts. The new Wisconsin system takes away points based on low participation, so there will be an impact there. If your child is likely to score in the higher ranges, their absence will lower the scores used to produce the “accountability” measures. If the school consequently falls into one of the two lower tiers, extended day programs and school improvement plans are required. If it is in the lowest tier, then the plans must include out-sourcing to an approved “turnaround” vendor. As I noted before, this is privatization of public services and turnaround specialists do not have records of success that inspire confidence. A school or district that fails to “turnaround” is subject to further intervention by the State Superintendent. A school or that does not cooperate with these directives “will close.”
Although school administrators have criticized the system, I doubt districts will choose the noncooperation option. Too bad, that would be a fight that would shine a bright light on the this conception of “accountability.”
Opting out is a smaller version of noncooperation that is available to every family. You don’t have to be part of the madness.
It can also become something larger. Without all of childrens’ test scores, the machine grinds to a halt. There is a national Opt Out movement. Here are some places to find out more (including opt out rights and procedures in other states and districts):
In closing, I want to point to an alternative to the over-use and abuse of standardized testing. Thee are many; this one — New York Performance Standards Consortium’s performance-based assessments — was featured in a Washington Post post, “An alternative to standardized testing for student assessment,” by Monty Niel today. Check it out. We can do better.
Thomas J. Mertz
Educational “accountability” is in the news and on the agenda again this week. It seems it is always in the news and on the agenda these days. I have many problems with most conceptions of educational “accountability,” especially those that are based largely on standardized tests (a visit to the National Center for Fair and Open Testing is in order if you don’t agree, or even if you do) and are proudly dubbed “data driven,” (the link takes you to old AMPS posts, Esther Quintero has an important post up on the topic this week at the Shanker Blog: “The Data-Driven Education Movement,” read it). I’m not going to take on the big concepts here and now, but instead say a few things about the new Wisconsin Report Cards and offer some thoughts about imposing some accountability on those concocting and implementing educational “Accountability” systems, about cutting the barbers’ hair.
The new Wisconsin Report Cards are the product of the “School and District Accountability Design Team” led by Governor Scott Walker, State Superintendent Tony Evers, Senator Luther Olsen, and Rep. Steve Kestell and featuring a decided over-representation of privatizers and deformers (those friends of education at Wisconsin Manufactures and Commerce had a seat), and an under-representation of educators (one teacher, no union reps). The final version is a centerpiece of Wisconsin’s successful effort to garner a waiver of NCLB strictures from Arne Duncan.
A school rating system like this should do three things. First it should with some accuracy and transparency rate school quality. Second, it should honestly and effectively communicate what the rating means and doesn’t mean to policy-makers, educators, parents, and citizens. Last — and assuming that the ratings are accurate — it should direct appropriate resources to those schools that need improvement. The Wisconsin system does none of these well. In fact, because of the complexities of assessing school quality, I don’t think it is possible to do all of these well and know that it is very difficult to do any of them well. The whole enterprise is in many ways a fool’s errand.
A recent must-read post by Gene V. Glass for the Washington Post captures some, but not all, of the problems (I’ve touched on the use of NAEP cut scores previously, will be saying more about some other things below and will be writing more on the waiver, the abuse of NAEP cut scores, “accountability,” and “educator effectiveness” issues in the future; as I was writing this another fine critique came my way, this one from Steve Strieker, called “Another Distractor: School Report Cards,” it is a must read also).
In the introduction to Glass’s piece Valerie Strauss calls the Report Cards “another cockamamie way to grade schools for “accountability” purposes.” Glass refers to the Report Cards as “a dog’s breakfast of numbers,” and writes:
The report card for Wisconsin K-12 schools currently making the rounds is a particularly opaque attempt to grade the quality of education that Wisconsin’s children are receiving at the hands of their teachers and administrators. It is as though the Department of Public Instruction has decided to weigh cattle by placing them on a scale to get their weight in pounds then combining that with the wealth of the farmer who raised them, the number of acres of the farm, and the make of car the farmer drives.
The Report Cards combine multiple and often complicated measures in complicated ways. It takes 62 pages to explain how it is all done. If in order to understand the choices made you want to dig deeper into the nature of standardized test construction (hint, they are designed to sort students, not measure skills, knowledge or ability), or the controversies over graduation rate calculations, or the limitations of the Student Growth Percentiles ( the link takes you to Bruce Baker posts on SGP and related things) used in the “growth” calculations, or any of the other concepts and tools employed , you are probably looking at at least the equivalent of a graduate school seminar’s worth of work. The system fails the transparency test.
All this information is interesting, but what it means for any particular school or district is far from clear, even after the graduate seminar and that’s how it should be, that’s reality…all the test score data, and graduation rate data, and attendance data in the world isn’t going give you a full and true picture of schools and districts. That’s the first way it fails the accuracy test, a little more below.
With “accountability” the order of the day, the “accountability” mavens know that people want something easily swallowed (if not digested), so the Wisconsin team has given each school a score, based on those calculations that take 62 pages to introduce. That score is what everyone looks at, everyone remembers and everyone seems to think has some profound meaning. What you really have is a Rube Goldberg machine of black boxes inside black boxes that spits out a number. That number hides all the questionable choices in the measures and manipulations, as well as all unmeasured and unmeasurable things that contribute to or detract from school quality. Some in Wisconsin were proud that we didn’t assign letter grades like Florida has, but the number is just as bad, or even worse because superficially something like 66.7% seems to have more scientific accuracy., an a B-. It doesn’t. Superintendent Tony Evers and others have said many of the appropriate things about over-interpreting the scores given schools, but they put the score there and because of the inclusion of the score, the system fails the communication test.
This failure reminds me of the misuse of NAEP cut scores that is central to the accountability system, used for sorting individual students and in the growth scores sores that only recognize movement between NAEP based levels, not within them. This is what the National Academy of Sciences publication, “Grading the Nation’s Report Card: Evaluating NAEP and Transforming the Assessment of Educational Progress,” says about these cut score in chapter 5, “Setting Reasonable and Useful Performance Standards (I’ve quoted this before here, “The news from Lake Gonetowoe“):
Although standards-based reporting offers much of potential value, there are also possible negative consequences as well. The public may be misled if they infer a different meaning from the achievement-level descriptions than is intended. (For example, for performance at the advanced level, the public and policy makers could infer a meaning based on other uses of the label “advanced,” such as advanced placement, that implies a different standard. That is, reporting that 10 percent of grade 12 students are performing at an “advanced” level on NAEP does not bear any relation to the percentage of students performing successfully in advanced placement courses, although we have noted instances in which this inference has been drawn.) In addition, the public may misread the degree of consensus that actually exists about the performance standards and thus have undue confidence in the meaning of the results. Similarly, audiences for NAEP reports may not understand the judgmental basis underlying the standards. All of these false impressions could lead the public and policy makers to erroneous conclusions about the status and progress of education in this country. (Emphasis added)
The NAE-based cuts scores (WKCE scores “mapped” to NAEP are also being used with the results of individual students. Here’s what the people at NAEP say about that:
Does this mapping method allow us to link student scores received on state test to the NAEP scale? If not, why not?
No, student scores cannot be linked to the NAEP scale because the NAEP does not generate reliable scores at the individual student level, only average scores for groups of students (e.g. males, females).
I would hope that at least the DPI staff working on the “Accountability” system knew this. If they didn’t, that’s a problem; if they did and went ahead anyway, that’s a bigger problem.
In terms of accuracy, the Report Cards do one thing well, they sort schools by their relative poverty. Here is what Gene V. Glass wrote on this:
What emerges from this dog’s breakfast of numbers? A measure of the wealth of the community in which the school is located. The correlation between the OAI and the “% Economically Disadvantaged” in the school is nearly -.70. That means that the poorer the children in the school, the lower is the school’s number on the Overall Accountability Index; and the relationship is close. In fact, a correlation of .70 is even tighter than the relationship of adults’ height to their weight, and both measure a person’s size. So what the DPI has created is a handy measure of a community’s wealth (SES, Socio-Economic Status) without ever having to ask anyone their income.
Steve Strieker observes that this isn’t news to many of us:
Even an amateur’s analysis of the state’s school report card data is telling.
- A supermajority of Wisconsin’s public schools with over 70% economically disadvantaged students were graded “Failed to Meet Expectations.”
- Almost all below-standard schools had at least 45% economically disadvantaged students.
- In contrast, almost all graded schools with less than 10% economically disadvantaged students were considered by DPI’s measurement to surpassed expectations.Social Context Reformers must not be shouted down by the “no excuses” reformers who will surely shame Wisconsin schools graded below expectation by showcasing the few schools with high poverty rates and high-test scores.
Given this pattern and what we know from 1,000 sources, the remedy should be to provide additional, appropriate help to high poverty schools. We didn’t need the Report Cards to tell us that.
Unfortunately the new system fails this test too. Most of “help” under the new system is directed to Title I schools. In theory, Title I schools are high poverty schools, but not all high poverty schools are Title I. In Madison and some other districts, for reasons I’ve never understood, only pre-K-5 schools are Title I, which means that no matter how high poverty (or low scoring) middle and high schools are left out.
In this case, that is probably for the best, because the “help” being offered appears to be more of a diversion of resources than an addition. No extra resources will be provided and some of the scarce resources available must be reallocated to questionable purposes.
The “Schools Below Expectations, and Significantly Below Expectations” will be required “to submit a plan detailing the extended learning opportunities for eligible students.” And:
[S]chools must participate in an online district-directed diagnostic review of the current core reading and math curriculum including interventions for struggling students. The school must develop an improvement plan based on the diagnostic review, and implement RtI, working closely with the Wisconsin RtI Center. Specific interventions in the plan must address identified problem areas. The plan must be approved by DPI…o DPI will conduct electronic reviews of each school’s progress and monitor throughout the year.
So extended learning, an online review, with an online plan, and online monitoring.
For “Schools Persistently Failing to Meet Expectations” extended learning is also mandated, the diagnostic review will be onsite, and also must result in an approved plan. But there is a kicker, and the name of that kicker is privatization: “Schools must contract with a state-approved turnaround expert/vendor to implement reform plans aligned to the diagnostic review.” In other words, schools have to take money from the classrooms and give it to the likes of Paul Vallas and hope for the best (here is a selection of posts on “turnarounds” from Diane Ravitch, read them to understand my skepticism). And when the turnaround fails, as they almost always do, here is what happens:
o For public schools that do not participate in the diagnostic review, improvement planning and interventions with turnaround experts, they will close.
o For schools that do participate but fail to show demonstrable improvement after three years, the State Superintendent will intervene. Pending legislation, in the case of schools participating in the Parental Choice Program, the state will remove the school from the program. In the case of charter schools, the authorizer must revoke the charter.
Arne Duncan has always liked school closings. I think it is safe to say that the system fails the “direct appropriate resources to those schools that need improvement” test also.
Superintendent Tony Evers also has some school finance proposals that he has been touting. Unfortunately, his Fair Funding for Our Future plan does not include directing any extra resources to high poverty schools or even those identified as in need by this accountability system. The Fair Funding plan does some good things, but addressing poverty is not one of them. For years the only state program directing resources to classrooms based on poverty is the Student Achievement Guarantee in Education or SAGE), which only targets the early grades and in budget cutting moves over the last few years, done under the rhetoric of “flexibility” has been eroded by larger allowed classes and new allowances concerning the grades covered. There does not seem to be any desire to change that, either from DPI or the legislature.
Fair Funding claims that it “Accounts for family income and poverty.” In sense it does, but via tax relief for property owners, not by giving schools serving students in poverty the resources they need to meet their challenges. Under Fair Funding student poverty levels will be factored into calculations of state aid, but revenue limits will not have a poverty bump and there is no new categorical aid for students in poverty. So property taxpayers in districts with higher poverty will have lower taxes and the schools will not have an extra penny (btw –the Penny for Kids proposal from the Wisconsin Alliance for Excellent Schools/ Opportunity to Learn Wisconsin includes a poverty based categorical aid). So the Widow Hendricks of “divide and conquer” fame who owns property in multiple high poverty districts gets a tax break and the students of Beloit and Janesville get nothing.
Back to the titular questions, who cuts the barber’s hair?; who holds the people behind this mess created in the name of “accountability” accountable? We all need to.
Start at the top. For Arne Duncan, join the thousands who have signed the “Dump Duncan” petition. There is also an election on November 6th and Duncan’s boss Barack Obama is up for re-election. Diane Ravitch has made a case that “as bad as the Obama education policies are, they are tolerable in comparison to what Mitt Romney plans.” Others concerned with education, especially those not in swing states, should take a good look at Jill Stein.
In Wisconsin, for Senate and the House, more-or-less the same situation exists. Some version of NCLB/ESEA will certainly be before Congress, and for that all of the Democrats on the ballot are better than the Republicans, but none have distinguished themselves on Education issues the way Russ Fiengold did. Still, I’ll be voting for Tammy Baldwin and Mark Pocan and urge you to do the same. I’ve already warned Mark that I’ll be contacting him regularly on Education and other issues and calling on him be more of a progressive champion on this blog, just as I have when he was my State Rep.
That’s another version of accountability. It starts at the ballot box, but it doesn’t end there. Our elected officials need to here from us, all the time. They need to know — as we sing at the Solidarity Sing Along — “We’re not going away.”
At the state level, we don’t get another crack at Scott Walker this year, but there are State Senate and Assembly races. Again, the rule is Democrats better than Republicans, but there are also some Democrats who are not only better than Republicans, but are real supporters of education. The two I’d like to point to are Melissa Sargent (who is a good friend) and Mandela Barnes (who I have admired from afar). By all accounts the Senate is the key this time around. The key races where your support ($$$ and time) may make difference appear to be Susan Sommer, Jessica King,and Dave Hansen. Keeping the Senate is the best way to keep Walker in check. With all this, it is important to remember that the “Accountability” system has been presented as a work in progress and there is some legislative power to dictate changes in some areas (the Report Card portion did not require legislative action, but other parts of the waiver did), and that any changes to school funding — good or bad — have to go through the legislature. With the State Legislature, this time around accountability means at minimum limiting the power of the Walker allies who aided in the creation of the “Accountability” system.
I’ve saved Superintendent Tony Evers for last. He is up for re-election in April 2013 and as with all elected officials, the best place to assert accountability is at the ballot box. It is also likely that come April, Evers will be the better choice (I supported Todd Price in the Primary last time and Evers in the General Election against Rose Fernandez). Also as with all elected officials, imposing accountability includes making sure Evers hears from the voters throughout his term, both positive and negative, and I hold some hope he may listen and adjust his course.
There is much I like and admire about Evers, but as the above indicates there are many things he has pushed that I think are bad, wrong or at very least should be better. I understand that most of this was done in the context of a state in the control of the Fitzwalker gang and a federal policies set by Arne Duncan. Given the circumstances, it is impossible to tell which things he truly believes are good for our state and our students and which are pragmatic choices made in order to keep a seat at the table and maybe deflect even worse policies (one example where I believe he did this was the mandated grade retention that Walker initially wanted in the Read to Lead legislation). This situation keeps bringing to mind something Anthony Cody wrote recently about teacher leaders:
How can we make sure that we are not being used as tokens? For this, we have to look at why we are being asked to join the conversation. What are the power dynamics at play? Do we have a vote when decisions are to be made? Will we find allies around the table to help us have some influence? Do we have any real cards to play? This gets us closer to defining what real leadership is all about. Real leadership is not just the ability to speak with clarity and authority based on our experience in the classroom. It also involves a relationship to other teachers, and to some level of political power in these situations.
The bottom line is that we do not have the money to buy influence. We have to get it the old-fashioned way. We have to organize for positive change at our school sites. We have to join with others at our union meetings, and as our colleagues in Chicago showed, we may need to go on strike. We have to build strong relationships with our colleagues, with parents, with allies in other unions and social movements, and with reporters, and use this strength as the basis for our ability to speak for ourselves. We have to organize and build our strength from the ground up, because the strength that comes from the top down is like the strings on a marionette.
Thomas J. Mertz
The biggest item on the Madison Metropolitan School District Board of Education committee agendas this week is the “Expeditionary Learning Model at Toki Middle School and Timeline” before the Planning and Development Committee (Monday, October 15, 2012, 6:30 p.m. Doyle Administration Building, Room 103, there are Public Appearances on the agenda). The proposal is to convert Toki Middle School from a district school to a district instrumentality charter school.
This is the start of a long process of weighing the pros and cons of the proposal, with various decision points along the way. If the proposal gets that far, the final decision by the Board would be in early 2014 and if the vote is for approval the conversion would happen in September of that year.
Having first seen the proposal only a couple of days ago, I have many questions, concerns and observations, but am nowhere near having decided whether to oppose this. For the record, that’s exactly where I was at at this point with both Badger Rock (which I did not oppose) and Madison Prep (which I did). Also for the record, I can’t see myself actively supporting any charter school proposal (it could happen, but it is doubtful). Mostly I see charters as a distraction from improving the district schools that will for the conceivable future continue to serve the vast majority of students (see this recent post from Deb Meir for some related thoughts).
So on to the “questions, concerns and observations.”
A desire for Federal money appears to be driving the decision to seek a charter.
I’ve written about this before (worth rereading, really), but have never seen such a transparent example. Some excerpts from the documents, first from the introduction by the district administration (the pdf doesn’t cut-and-paste as text, you can click on the images to enlarge):
Note the “do more with less,” and the “need to look beyond the District allocation process” and remember that in the recent past MMSD has not used the full levy authority available and that the administrative recommendations to not use the full levy authority have been accompanied by assurances that the quality of education was not suffering due to the lack of resources. If that was true, then the statements here aren’t.
From the proposal itself:
More references to “budget constraints” past and anticipated in the future. A distinction should probably be made here that there are two related parts to the proposal. One is the proposal to continue and expand Expeditionary Learning at Toki (more on this below) and the other is to do this as a Charter School. It is clear that the desire to do the Expeditionary Learning is strong among some, and that chartering is primarily a financially driven means to that end. Here is what I wrote previously about these aspects of chartering:
The “we can’t do it without a Charter” attitude seems lazy. First I’d like to know in some detail why it supposedly can’t be done without a Charter. If that proves to be the case, than in most instances wouldn’t the best policy be to figure out why and change things so that the benefits of innovation could be achieved through district programs? It is sad that so many have given up on the reforms that would benefit all students in order to pursue those that will only touch very few (even the staunchest Charter advocates understand that for the foreseeable future the vast majority of American children will attend district schools).
I’ll offer one answer to the titular question: Money! Unfortunately Federal policy-makers, foundations and many others are all acting on the unexamined assumption that innovation or even diversity of educational programing requires Charters. I have a friend who is a Superintendent of a small district. He is justly proud of an environmental Charter school he helped create. We’ve never talked about it much, but a couple of months ago he started describing how the only reason to have it be a Charter was the money. This is pragmatic, but it only shifts the question to “Why is money available for Charters and not district-based creative programs?”
In the bigger picture, with Race to the Top (and the NCLB waivers), we have seen how chasing Federal money has led to less than stellar education policy-making. Much less than stellar.
The budget numbers assume maximum Federal Grants.
I don’t know if this is realistic. In order to qualify for the maximum, the Charter must have at least 50% free/reduced lunch enrollment. That has been the case with Toki only one of the last 11 years (2010-11). Beyond that, I’m not clear if the maximum grants have changed recently or if DPI awards grants below the maximums, but I do know that no grants given in 2012 were this large.
The dollar amounts cited in the proposal are over three times as large as those cited in a previous request.
On January 31, 2011 the Board was presented with a funding appeal (apparently directed at private donors) for Expeditionary Learning exploration and implementation at Toki, with a 4 year budget that totaled $310,000. In the pending 2012-13 budget MMSD will provide $60,000 for planning. It would be good to know why the numbers have changed. The proposers are probably correct that MMSD can not fund $975,000 for their purposes, but another $250,000 ($310,000 minus $60,000) over 2-3 years is possible.
At this point there is no indication of what the ongoing costs will be.
That’s fine, that information is required at later points in the process, but being aware that there may be extra costs that continue after the Federal money goes away seems wise.
They appear to be proposing a school that is both a charter and and an attendance area school.
On page 13, the proposal says the Charter School students will be “reflective of the current neighborhood,” that “all students within the Toki attendance area will attend,” but also includes references to a lottery to meet numbers. I would assume that some arrangements would be made for attendance area students who do not want to attend the Charter School, who would prefer to have the policies of their school set and implemented by an elected Board of Education instead of a self-selected Governing Board.
The research cited on Expeditionary Learning is less than convincing.
The proposal (in a very sloppy manner) cites three studies and one meta analysis in support of Expeditionary Learning as a means of addressing persistent gaps in achievement. I’m reproducing this entire section below in order to emphasize the centrality of achievement gaps to the proposal and what the research claims are:
The research citations here seem to come straight from the Expeditionary Learning organization that would be in line to receive most of the $975,000 being requested, so it seems worth looking a bit closer (it is also worth noting that the three studies all appear to have been commissioned by that organization, although I am not 100% sure of that).
Another important caveats is that these are all based on standardized tests and those offer very limited insight into school quality.
Two of the studies — “Expeditionary Learning: Analysis of impact on achievement gaps,” and “Impact of the Expeditionary Learning model on student academic performance in Rochester, NY” — were done by the Donahue Institute at UMass. I can’t find the first online, but both were submitted to the What Works Clearinghouse and found not to meet their standards for review. More on the Rochester study below. The third study — “The relationship between Expeditionary Learning participation and academic growth” — was done by Mountain Measurement (analysts for hire) and includes few — if any — schools comparable to Toki 9more on this study below, also). The meta analysis is from 2002, focused on whole school reform models (not achievement gaps or Expeditionary Learning) and categorized Expeditionary Learning among the programs that were “Highly Promising…but did not have research bases that were as broad and generalizable as those of the models that met the highest standard.”
The Rochester study includes two Expeditionary Learning schools. The Genesee Community Charter School, serving grades K-6, with a Free/Reduced Lunch rate of 17%, and 0% student mobility, 18% African American, 9% Hispanic, 2% ELL (can’t find Special Ed numbers). For all these reasons, I don’t see it as comparable to Toki (grades 6-8, not a charter, 48.9% Free/Reduced, most recent mobility factor of 15.8, 28.1% African American, 13.8% Hispanic, 11.5% ELL, 16.4% Special Ed). the other is the World of Inquiry School. This is not a Charter, but it may be a magnet or choice school (there is an “Admissions” page on the website, but the click through ends up at a page that isn’t working). World of Inquiry is a K-8 school, and appears to face similar challenges as Toki, some more pronounced (63% Free/Reduced, 76% African American, 9% Hispanic, 3% ELL). I’ll be returning to World of Inquiry.
The first part of the report on the Rochester schools cited in support of Expeditionary Learning seems legitimate, as far as it goes (I’d like more details on some of the steps and choices). It is a “quasi-experimental design, involving matching students in Expeditionary Schools with students in other schools by various characteristics (poverty, race, ELL, gender, grade, special education status). There is some manipulation around the score distributions, involving a regression analysis of the various characteristics, but the regression coefficients aren’t given. The found statistically significant, positive effect sizes for reading across the board, and in math only for elementary school. Here’s the table:
Statistically significant doesn’t always translate into significance for policy.
The next step in the attempts to address this by converting the effect sizes into “implied shifts” in proficiency percentages. The shifts in some of the categories (the same ones as the above chart) are large — about a 30% gain in proficient students for middle school reading in 2007-9, for example and bigger gains with elementary (along with losses for middle school math — so large that they make me doubt the whole analysis. remember that it did not meet the Department of Education’s What Works Clearinghouse standards. I’ll look closer later and consult with people who know more about quantitative work than I do (as I said at the top, this is going to be a long process), for now I want to keep this in mind and shift to looking directly how middle school students at World of Inquiry have been doing.
The short answer is, not very well, worse than Toki. No doubt at some point in the future I’ll produce nice charts and graphs on this, but here’s the quick and dirty numbers, only one year (2010-11), one grade (8th), and only Reading, percentages proficient or better (Toki data from WINNS, World of Inquiry data from here):
|Reading Toki||Reading WoI|
Note that state proficiency measures vary greatly, according to the NAEP mapping project, Wisconsin’s are lower than New York’s.
In every category, World of Inquiry is doing worse than Toki, far worse. This raises many questions about all the promises made in the proposal.
Even quicker and dirtier on the “The relationship between Expeditionary Learning participation and academic growth” report. It is a similar quasi-experimental design, but includes many more schools and students. Some are rural, some are suburban, some are charters, some are private, some are k-5, some include 6-8…only two seem to be comparable at all to Toki. MAP testing is at the center of this study, for what that’s worth. The results are all over the place (as they probably should be). I haven’t had time to dig into it too much, here’s the summary table:
Again, much here that doesn’t support the confidence of the citations or tone of the proposal.
King Middle School in Portland,ME seems to be one of the more comparable schools in this study, and one that many supporting the proposal seem to be pointing to as an example of success. It isn’t exactly comparable, having similar poverty numbers, but much different racial and ethnic percentages (more on that at another time) With that in mind, here is another chart comparing Toki and King (same years, grades…as above, King data from here):
|Reading Toki||Reading King|
Note that state proficiency measures vary greatly, according to the NAEP mapping project, Wisconsin’s are lower than Maine’s.
Students at King seem to be doing roughly the same as students at Toki, with African American students doing substantially worse.
These initial forays into the research and data find little or nothing to support the implementation of Expeditionary Learning (with a charter or otherwise) as the solution to the challenges and struggles Toki has been facing. There is much more to be done before a firm conclusion either way is arrived at. I hope that unlike with Madison Prep, the MMSD administration does their duty to thoroughly analyze the educational aspects of this proposal, so that the Board and the community have something more than my explorations to go by.
There are many other concerns and questions I have at this point, these include the present and future roles and thinking of those Toki staff members who don’t support the initiative, the way this all interacts with the recently approved Urban League program, how people at Toki came to champion Expeditionary Learning (and how much independent research have they done or looked at), why the “Equity” section of the administrative portion is blank …many, many questions and concerns. Plenty of time to air them; this is a long process.
I want to close by saying that I admire those behind this proposal for working to improve their school, for not accepting the frustrations of seeing students struggle and fail, for taking the initiative to find a way to try to make things better. I may not end up agreeing that the way they have chosen is a good way, a way worth trying, but that doesn’t change my admiration.
Thomas J. Mertz
Excerpts from a speech given to the 1916 Convention of the National Education Association, “The Public Schools and the Working Man,” (full speech linked). Gompers was followed by John Dewey on the program!
From the introduction:
On Vocational Education (more here):
Powerful and important ideas.
For those in Madison, please join the celebration of Labor Day at LaborFest, September 3, 12:00 Noon to 5:30, at the Labor Temple, 1602 S. Park St (poster/flier linked here). Good music, good food, good people, good idea.
Previous AMPS Labor Day posts:
This is the third in a new series on AMPS: Blasts from the Past. The series is devoted to historical materials that comment on or illuminate contemporary issues in education.
Thomas J. Mertz