The Leadership Academy: the real deal?

Last week the Aspiring Principals Program of the New York City Leadership Academy, made headlines. On August 24, a NYU press release announced:

“Public elementary and middle schools in New York City led by ‘Aspiring Principals Program’-trained principals have achieved comparable or higher rates of student improvement than schools led by other new principals ... These results were obtained even though APP-trained principals were more likely to be placed in chronically low-performing schools.”

The New York Times chimed in: “Graduates of a program designed to inculcate school principals with unconventional thinking have gone on to help drive up English test scores even though the graduates were often placed at schools with histories of academic failure.” The article went on to explain that the APP graduates helped increase English Language Arts scores at elementary and middle schools “at a faster pace than new principals with more traditional résumés”; while in math the APP principals made progress, but “at a pace no better than their peers.”

The report

What did this report actually say? Written by Sean P. Corcoran, Amy Ellen Schwartz, and Meryle Weinstein of NYU’s Institute for Education and Social Policy, it compared the performance of schools under the leadership of graduates from the Aspiring Principals Program (APP) with that of schools under other new principals.

Both groups had to have been placed as new principals and to have remained in their positions for three years. Of the 147 graduates in the 2004 and 2005 APP cohorts, 88 (60 percent) met the inclusion criteria. 371 non-APP principals met the criteria; of these, 334 were in schools with comparable grade configurations. So there were 88 APP principals and 334 comparison principals in the study.

The schools in the two categories were significantly different. Compared to other new principals, APP principals tended to be placed in lower-performing schools and schools trending downward in ELA and math. There were also demographic and geographic differences.

The study used two types of comparison: (a) a straightforward comparison of average achievement in both types of schools and (b) a regression analysis (controlling for various school and student characteristics). It was the regression analysis that suggested an APP edge in ELA (but not for math) for elementary and middle schools.

The findings

The study found that test scores at schools in both groups improved over the period of the study in terms of test scores– but not as much as schools in the rest of the city. More specifically, the regression analysis indicated that the ELA standardized scores of APP elementary and middle schools were relatively stable, compared to schools headed by new principals who were not APP graduates. In math, APP elementary and middle schools fared slightly worse than comparison schools in relation to the city, but the differences were not statistically significant.

At the high school level (not mentioned in the NYU press release or NYT article), the differences between APP and comparison schools were “minor and inconclusive.”

Unanswered questions

There are many questions that the study did not address. Only 88 out of 147 graduates in the 2004-2005 and 2005-2006 cohorts met the inclusion criteria. More than 18 percent of APP graduates were never placed as principals at all. The rest stayed in their positions for fewer than three years.

Is this a high or low number? The authors wrote that they did not have comparative mobility information for the non-APP principals, but they presumably could have reported the average attrition rate for New York City principals overall.

Also, the study only analyzed test score data – which alone are insufficient to fully evaluate a school’s performance. Wasn’t there other data that could have been examined? What about the parent and teacher surveys at APP-headed schools compared to schools run by other new principals?

Though the study compared the size of the schools for both cohorts (APP graduates on average headed smaller schools) they did not compare class sizes – or other school-level conditions that could have contributed to the relative performance of both groups.

Most intriguing is the finding that the relative test scores at both sets of schools continued to decline compared to the rising achievement of schools citywide, but schools headed by APP principals declined less –at least in terms of their ELA results:

“... relative student test performance falls modestly in the years following the installation of a new principal, in both APP and comparison schools…..we find a statistically significant negative relationship between new principals and achievement in both mathematics and ELA.”

It was only after doing a regression analysis, by controlling for various factors (including student background), that they found that the relative performance of APP schools was relatively stable while the comparison schools continued to decline. See this graph:

Thus, the reigning philosophy of the Klein administration – that new leaders properly trained in the methods propounded by the Leadership Academy will spark significant improvements in low-performing schools does not seem to hold true. Instead, these appointees may stabilize what otherwise would be expected to be continued decline resulting from a new principal.

As a way to deal with ongoing attrition and fill positions in elementary and middle schools, the Leadership Academy might be said to be “promising”. But as a way to “turn around” schools it does not seem to be promising at all.

Unmentioned in any of the news articles was the fact that the research organization Mathematica had originally been commissioned by DOE to do an in-depth, multi-year study of the Leadership Academy. Yet after several years of analysis, this study was cancelled by DOE, just months before the results were supposed to be released. What Mathematica might have been discovered about the program and its graduates will probably never be known.

For another close look at this study, see Aaron Pallas’ critique at Gotham Schools.

The Truth about Charter Schools

Come to a meeting on charter schools, sponsored by Sen. Bill Perkins; Wed. Sept. 2 at 6 PM. More info by clicking on the flyer at the right.

Many schools and students are being deprived of classrooms and equitable conditions because of the forced insertion of charter schools into their buildings.

How should we deal with the growing trend towards privatization that is ripping our communities apart?

The hiring crisis in our schools: what brought us to this point?

The Times performs a double whammy today by running an editorial in favor of the Race to the Top proposals; featuring a direct attack on the teachers unions for opposing tying the evaluation of teachers to standardized test scores: Editorial: Accountability in Public Schools.

The paper also features an article, Amid Hiring Freeze, Principals Leave Jobs Empty , about how principals are refusing to hire experienced teachers on ATR (absent teacher reserve) – of which there are nearly 2,000 -- despite 1800 teaching openings. Instead, principals are hiring new teachers as “permanent” substitutes, waiting out the hiring freeze that Klein announced a few months ago. What the article fails to discuss is how this is a crisis entirely of Joel Klein’s making, and represents one of the biggest management blunders of his career.

Klein has continued to pay for Teach for America and the Teaching Fellows program to recruit new candidates long after it was clear that a huge pool of experienced teachers was growing, who are being paid full salary and yet have no regular classroom assignments-- through no fault of their own. In the article, no mention is made of what led to this crisis: how Klein changed the school funding system, under the advice of Sir Michael Barber, a management consultant from McKinsey and Co., whose advice to the Blair administration to impose a similar scheme in the UK had earlier caused massive teacher layoffs and what was described as the most serious educational crisis in that nation’s post-war history.

NYC’s so-called “fair funding” system was specifically designed so that for the first time, principals would have to pay for their own staffing of teachers and for their full salaries, to give them an incentive to hire new, cheaper teachers rather than experienced ones. Many critics warned that given budget cuts to come, the refusal of DOE to fund teachers centrally -- as opposed to say, school achievement facilitators, data inquiry teams or parent coordinators, all of which is directly financed by the administration -- would lead to principals being forced to choose between larger classes and less experienced teachers, and this is exactly what has occurred.

See, for example, our exchange with Robert Gordon, who designed the funding system for Klein, in March 2007, and earlier comments from Noreen Connell and Gordon in a Times article from Jan. 27, 2007, Seeking Equity, School Chief Outlines a Financing Plan:

The expert, Noreen Connell, who leads the Educational Priorities Panel, a nonprofit group, said that the changes would initially make the budget system more complicated, and would be harmful long term by making it overly expensive for schools to retain veteran teachers.
While the new plan would provide money to schools and require principals to cover payroll and other expenses, Ms. Connell said in an interview that she preferred a system that seeks to calculate a school's staffing needs and then provides the dollars to meet them.

''The funding proposals,'' she wrote in commentary posted on the group's Web site, ''have the potential to do lasting damage for decades to come.'' In the interview, Ms. Connell also said the chancellor did not have time to carry out the plan before the end of Mr. Bloomberg's term in 2009. ''They won't be around to suffer the consequences,'' she said.

Robert Gordon, the Education Department's managing director for resource allocation, who is designing the new system, said it would maximize the amount of control that principals have over their budgets, allowing them ''to retain their most experienced teachers if that is what they want to do.''

Why should the school funding system be designed to force principals to choose between hiring or retaining experienced teachers and smaller classes – when these are among the few factors that have been proven to result in better schools ? Do public schools in the suburbs have to choose between these goals, or the private schools to which Klein and Bloomberg sent their kids?

The article also omits its own reporting of the fact that there has been a decrease of 1600 in the number of classroom teachers under this administration, with a concurrent rise in ten thousand out of classroom positions, including two thousand more school secretaries.

The only thing incorrect about Noreen Connell’s predictions in 2007 is that we may be saddled with Chancellor Klein for years to come, because of Bloomberg’s overturning of term limits. Robert Gordon has now moved onto the Obama administration, where he is probably designing similarly destructive funding schemes on a national scale. Sadly, the Educational Priorities Panel, one of the few objective monitors of DOE’s spending practices, is gone, apparently because the NYC foundation world didn’t see the point in supporting the sort of expert analysis that EPP was able to provide.