Expeditionary Learning Charter for Toki?

The Replacements – “I Don’t Know” (click to listen or download).

The biggest item on the Madison Metropolitan School District Board of Education committee agendas this week is  the “Expeditionary Learning Model at Toki Middle School and Timeline” before the Planning and Development Committee (Monday, October 15, 2012, 6:30 p.m. Doyle Administration Building, Room 103, there are Public Appearances on the agenda).  The proposal is to convert Toki Middle School from a district  school to a district instrumentality charter school.

This is the start of a long process of weighing the pros and cons of the proposal, with various decision points along the way.  If the proposal gets that far, the final decision by the Board would be in early 2014 and if the vote is for approval the conversion would happen in September of that year.

Having first seen the proposal only a couple of days ago, I have many questions, concerns and observations, but am nowhere near having decided whether to oppose this.  For the record, that’s exactly where I was at at this point with both Badger Rock (which I did not oppose) and Madison Prep (which I did).  Also for the record, I can’t see myself actively supporting any charter school proposal (it could happen, but it is doubtful).  Mostly I see charters as a distraction from improving the district schools that will for the conceivable future continue to serve the vast majority of students (see this recent post from Deb Meir for some related thoughts).

So on to the “questions, concerns and observations.”

A desire for Federal money appears to be driving the decision to seek a charter.

I’ve written about this before (worth rereading, really), but have never seen such a transparent example.  Some excerpts from the documents, first from the introduction by the district administration (the pdf doesn’t cut-and-paste as text, you can click on the images to enlarge):

Note the “do more with less,” and the “need to look beyond the District allocation process” and remember that in the recent past MMSD has not used the full levy authority available and that the administrative recommendations to not use the full levy authority have been accompanied by assurances that the quality of education was not suffering due to the lack of resources.  If that was true, then the statements here aren’t.

From the proposal itself:

More references to “budget constraints” past and anticipated in the future.  A distinction should probably be made here that there are two related parts to the proposal.  One is the proposal to continue and expand Expeditionary Learning at Toki (more on this below) and the other is to do this as a Charter School.  It is clear that the desire to do the Expeditionary Learning is strong among some, and that chartering is primarily a financially driven means to that end.  Here is what I wrote previously about these aspects of chartering:

The “we can’t do it without a Charter” attitude seems lazy.  First I’d like to know in some detail why it supposedly can’t be done without a Charter.  If that proves to be the case,  than in most instances wouldn’t the best policy be to figure out why and change things so that the benefits of innovation could be achieved through district programs?  It is sad that so many have given up on the reforms that would benefit all students in order to pursue those that will only touch very few (even the staunchest Charter advocates understand that for the foreseeable future the vast majority of American children will attend district schools).

I’ll offer one answer to the titular question: Money!  Unfortunately Federal policy-makers, foundations and many others are all acting on the unexamined assumption that innovation or even diversity of educational programing requires Charters.   I have a friend who is a Superintendent of a small district.  He is justly proud of an environmental Charter school he helped create.  We’ve never talked about it much, but a  couple of months ago he started describing how the only reason to have it be a Charter was the money.   This is pragmatic, but it only shifts the question to “Why is money available for Charters and not district-based creative programs?”

In the bigger picture, with Race to the Top (and the NCLB waivers), we have seen how chasing Federal money has led to less than stellar education policy-making.  Much less than stellar.

The budget numbers assume maximum Federal Grants.

I don’t know if this is realistic.  In order to qualify for the maximum, the Charter must have at least 50% free/reduced lunch enrollment.  That has been the case with Toki only one of the last 11 years (2010-11).  Beyond that, I’m not clear if the maximum grants have changed recently or if DPI awards grants below the maximums, but I do know that no grants given in 2012 were this large.

The dollar amounts cited in the proposal are over three times as large as those cited in a previous request.

On January 31, 2011 the Board was presented with a funding appeal (apparently directed at private donors) for Expeditionary Learning exploration and implementation at Toki, with a 4 year budget that totaled $310,000.  In the pending 2012-13 budget MMSD will provide $60,000 for planning.  It would be good to know why the numbers have changed.  The proposers are probably correct that MMSD can not fund $975,000 for their purposes, but another $250,000 ($310,000 minus $60,000) over 2-3 years is possible.

At this point there is no indication of what the ongoing costs will be.

That’s fine, that information is required at later points in the process, but being aware that there may be extra costs that continue after the Federal money goes away seems wise.

They appear to be proposing a school that is both a charter and and an attendance area school.

On page 13, the proposal says the Charter School students will be “reflective of the current neighborhood,” that “all students within the Toki attendance area will attend,” but also includes references to a lottery to meet numbers.   I would assume that some arrangements would be made for attendance area students who do not want to attend the Charter School, who would prefer to have the policies of their school set and implemented by an elected Board of Education instead of a self-selected Governing Board.

The research cited on Expeditionary Learning is less than convincing.

The proposal (in a very sloppy manner) cites three  studies and one meta analysis in support of Expeditionary Learning as a means of addressing persistent gaps in achievement.  I’m reproducing this entire section below in order to emphasize the centrality of achievement gaps  to the proposal and what the research claims are:

The research citations here seem to come straight from the Expeditionary Learning organization that would be in line to receive most of the $975,000 being requested, so it seems worth looking a bit closer (it is also worth noting that the three studies all appear to have been commissioned by that organization, although I am not 100% sure of that).

Another important caveats is that these are all based on standardized tests and those offer very limited insight into school quality.

Two of the studies  — “Expeditionary Learning: Analysis of impact on achievement gaps,” and “Impact of the Expeditionary Learning model on student academic performance in Rochester, NY” — were done by the Donahue Institute at UMass.  I can’t find the first online, but both were submitted to the What Works Clearinghouse and found not to meet their standards for review.  More on the Rochester study below.    The third study — “The relationship between Expeditionary Learning participation and academic growth” — was done by Mountain Measurement (analysts for hire) and includes few — if any — schools comparable to Toki 9more on this study below, also).  The meta analysis is from 2002, focused on whole school reform models (not achievement gaps  or Expeditionary Learning) and categorized Expeditionary Learning among the programs that were “Highly Promising…but did not have research bases that were as broad and generalizable as those of the models that met the highest standard.”

The Rochester study includes two Expeditionary Learning schools.  The Genesee Community Charter School, serving grades K-6, with a Free/Reduced Lunch rate of 17%, and 0% student mobility,  18% African American, 9% Hispanic, 2% ELL (can’t find Special Ed numbers).  For all these reasons, I don’t see it as comparable to Toki (grades 6-8, not a charter, 48.9% Free/Reduced, most recent mobility factor of 15.8, 28.1% African American, 13.8% Hispanic, 11.5% ELL, 16.4% Special Ed).  the other is the World of Inquiry School.  This is not a Charter, but it may be a magnet or choice school (there is an “Admissions” page on the website, but the click through ends up at a page that isn’t working).  World of Inquiry is a K-8 school, and appears to face similar challenges as Toki, some more pronounced (63% Free/Reduced, 76% African American, 9% Hispanic, 3% ELL).  I’ll be returning to World of Inquiry.

The first part of the report on the Rochester schools cited in support of Expeditionary Learning seems legitimate, as far as it goes (I’d like more details on some of the steps and choices). It is a “quasi-experimental design, involving matching students in Expeditionary Schools with students in other schools by various characteristics (poverty, race, ELL, gender, grade, special education status).  There is some manipulation around the score distributions, involving a regression analysis of the various characteristics, but the regression coefficients aren’t given.   The found statistically significant, positive effect sizes for reading across the board, and in math only for elementary school.  Here’s the table:

Statistically significant doesn’t always translate into significance for policy.

The next step in the attempts to address this by converting the effect sizes into “implied shifts” in proficiency percentages.   The shifts in some of the categories (the same ones as the above chart) are large — about a 30% gain in proficient students for middle school reading in 2007-9, for example and bigger gains with elementary (along with losses for middle school math — so large that they make me doubt the whole analysis. remember that it did not meet the Department of Education’s What Works Clearinghouse standards.   I’ll look closer later and consult with people who know more about quantitative work than I do (as I said at the top, this is going to be a long process), for now I want to keep this in mind and shift to looking directly how middle school students at World of Inquiry have been doing.

The short answer is, not very well, worse than Toki.  No doubt at some point in the future I’ll produce nice charts and graphs on this, but here’s the quick and dirty numbers, only one year (2010-11), one grade (8th), and only Reading, percentages proficient or better (Toki data from WINNS, World of Inquiry data from here):

	Reading Toki	Reading WoI
All Students	77.50%	46.00%
African American	64.70%	39.00%
Hispanic	72.20%	63.00%
ELL	42.90%
Special Ed	40.00%	0.00%
Free/Reduced	64.20%	43.00%

Note that state proficiency measures vary greatly, according to the NAEP mapping project, Wisconsin’s are lower than New York’s.

In every category, World of Inquiry is doing worse than Toki, far worse.   This raises many questions about all the promises made in the proposal.

Even quicker and dirtier on the “The relationship between Expeditionary Learning participation and academic growth” report.  It is a similar quasi-experimental design, but includes many more schools and students.  Some are rural, some are suburban, some are charters, some are private, some are k-5, some include 6-8…only two seem to be comparable at all to Toki.  MAP testing is at the center of this study, for what that’s worth.  The results are all over the place (as they probably should be).  I haven’t had time to dig into it too much, here’s the summary table:

Again, much here that doesn’t support the confidence of the citations or tone of the proposal.

King Middle School  in Portland,ME seems to be one of the more comparable schools in this study, and one that many supporting the proposal seem to be pointing to as an example of success.  It isn’t exactly comparable, having similar poverty numbers, but much different racial and ethnic percentages (more on that at another time)  With that in mind, here is another chart comparing Toki and King (same years, grades…as above, King data from here):

	Reading Toki	Reading King
All Students	77.50%	78.00%
African American	64.70%	46.00%
Hispanic	72.20%
ELL	42.90%	47.00%
Special Ed	40.00%	38.00%
Free/Reduced	64.20%	64.00%

Note that state proficiency measures vary greatly, according to the NAEP mapping project, Wisconsin’s are lower than Maine’s.

Students at King seem to be doing roughly the same as students at Toki, with African American students doing substantially worse.

These initial forays into the research and data find little or nothing to support the implementation of Expeditionary Learning (with a charter or otherwise) as the solution to the challenges and struggles Toki has been facing.  There is much more to be done before a firm conclusion either way is arrived at.  I hope that unlike with Madison Prep, the MMSD administration does their duty to thoroughly analyze the educational aspects of this proposal, so that the Board and the community have something more than my explorations to go by.

Closing

There are many other concerns and questions I have at this point, these include the present and future roles and thinking of those Toki staff members who don’t support the initiative, the way this all interacts with the recently approved Urban League program, how people at Toki came to champion Expeditionary Learning (and how much independent research have they done or looked at), why the “Equity” section of the administrative portion is blank …many, many questions and concerns.  Plenty of time to air them; this is a long process.

I want to close by saying that I admire those behind this proposal for working to improve their school, for not accepting the frustrations of seeing students struggle and fail, for taking the initiative to find a way to try to make things better.   I may not end up agreeing that the way they have chosen is a good way, a way worth trying, but that doesn’t change my admiration.

Thomas J. Mertz