Hewlett Foundation Education Grant Making on School District Reform:

An Assessment

EXECUTIVE SUMARY

Adam Gamoran, University of Wisconsin

Paul Goren, Spencer Foundation

James Spillane, Northwestern University

We used three framing questions in our examination of the Hewlett Foundation’s education investments on school districts:

1. What did the program intend and what did the grantees do?

2. What is the promise of district-led instructional reform to raise student performance levels and reduce achievement gaps?

3. Given the multiple issues facing low performing students in public schools, is a district-focused reform initiative a promising and productive way for continued Hewlett Foundation investments?

To understand the Hewlett grant making approach, we placed its portfolio of work in the context of concurrent education reform, policy initiatives, and the work of other national foundations.  We looked closely at Hewlett annual reports, budgets, and strategic memos; grant proposals and reports; and products from the grants executed by the Foundation.  We also spoke with some grantees. This executive summary provides the context in which Hewlett executed this grantmaking program, a discussion on the progress made and challenges faced by the Foundation in this arena, and implications for the future.

Context

The Hewlett Foundation’s grant making on district-led improvements, occurred within a long and sustained focus on education reform at the state and national levels.  Since 1983, when the seminal report A Nation at Risk was published, standards-based education reform has remained a key policy issue for politicians, educators, social scientists, advocates, and citizens. Successive waves of reform moved the education enterprise from a focus on inputs to an emphasis on outputs and performance. After nearly a quarter of a century, current reforms include the development of content standards across curricular areas, assessments intended to align with these standards, and high-stakes accountability systems that address chronically failing schools. Common in the reform discourse are questions about reducing and eliminating student achievement gaps across racial and economic lines, attracting and retaining outstanding teachers and school leaders, improving the quality of teaching and learning, and addressing the increasing needs of English language learners and special education students. Hewlett’s emphasis on the district points to the organizational entity within public schools that can address and implement all aspects of standards-driven reforms, from the professional development of teachers and principals, to the development of instructional regimes intended to reach all students, to the closing of chronically failing schools at scale.

Over the years, several national foundations made investments parallel to Hewlett’s to improve public education, especially in urban centers serving low-performing students from families below the poverty line who are predominantly African American and Hispanic.  A variety of approaches were executed including the establishment of content, performance, and opportunity to learn standards; efforts to build state and local capacity; teacher and principal training and on-going professional development; the reform of middle schools and high schools; and attempts to align curriculum, instruction and assessment focused on improved student outcomes. The Hewlett Foundation’s Education Program stayed the course over a seven-year period, in contrast to many of the other foundations whose interests have waxed and waned. 

The district-focused part of the Hewlett education portfolio evolved over time, responding to opportunities in districts such as San Diego, Boston, Hayward and Ravenswood, in addition to “residual” grant making such as the long-term commitment to the Bay Area School Reform Collaborative (BASRC) that was part of the Annenberg match. The following statement by Paul Brest, president of the Hewlett Foundation, most succinctly characterized the Foundation’s philanthropic strategy and experience on district-led instructional reform.

Designing, implementing and monitoring a strategy to improve society or the environment is like planning and embarking on a perilous quest for an elusive goal. It involves lots of slogging, with periods of tedium as well as excitement; moments of fear as well as the satisfaction of being on course; opportunities for unexpected discoveries, both good and bad – and the possibility that you may not get there at all (Hewlett Foundation 2003 Annual Report).

Progress of Hewlett’s Work

Our review of the Hewlett portfolio suggests that district-led instructional reform shows both promise and challenges.  Overall, the portfolio of work supported by Hewlett over the past five years:

· Offers existence proofs that district-led instructional reform is feasible, even in large urban school districts.

· Identifies core elements of a theory of action for district-led instructional reform.  Still missing from most of the work, however, is a clear and precise articulation of how these elements fit together to enable instructional improvement and improve student achievement. 

· Offers some evidence that district-led instructional reform is sustainable over time, but no evidence that it is scalable to more than a handful of America’s school districts.

· Provides evidence of impact on student outcomes, but only from one case.

Theory of Action

Hewlett-supported initiatives ranged widely in the extent to which a strong theory of action was present that guided reform efforts from the outset while in some cases a more or less specified the theory of action emerged over-time.  Looking across the portfolio some combination of the following elements constitutes district-led instructional reform.

· Instructional improvement efforts initiated and supported by the school district, acting alone or in concert with an intermediary agency (e.g., BASRC or IFL)

· Investment in professional development for district and school administrators and classroom teachers

· Instructional coherence established through alignment of instructional guidance instruments with standards for teaching and learning

· Mobilizing incentives and sanctions for agents and agencies to improve

· Data-based decision-making

· Designing and deploying new organizational routines and tools to support the work of improvement.

Grantees (both research and development efforts) focused on some combination of these elements though the degree to which the elements were developed varied widely across sites. Further, district-led instructional reform supported by the Hewlett Foundation ranged from reasonably well-specified theories of action to weakly specified and/or evolving theories of action. 

Scaling Up Reform

By scale we mean whether district-led instructional reform can be implemented on a district-wide basis, and whether more than a handful of school districts can take up district instructional reform.  While the research and development projects supported by Hewlett offer existence proof that district instructional reform is possible, they offer much less evidence that school district instructional reform are scalable to most or even a majority of America’s school districts. This is to be expected considering Hewlett’s focus was on funding research and development efforts that enabled the design and specification of district-led instructional reform strategies rather than examining their scalability.

Sustaining Reform

By sustainability we mean whether district reform, be it orchestrated from within the school district or by some external provider or intermediary agency (e.g., IFL, New Teacher Center, BASRC), can be sustained over time. Overall, Hewlett’s portfolio of work offers mixed evidence on the question of sustainability. The evidence offered by the work underscores the tenuousness of district-led instructional reform.  Further, exogenous factors often beyond the control of the superintendent and district office staff (and external partners of various types) – state and federal policy, teacher and administrator mobility – are key considerations with respect to the sustainability of district-led reform. The Boston work offers evidence that school districts can stay the course with a broad strategy of reform over a decade or more.  The San Diego case suggests that a cohesive and relatively well-specified theory of action can be sustained for relative long periods (7 years), even in the face of substantial resistance from the teachers union and to a lesser degree from ethnic advocacy groups. At the same time, this case highlights the challenge of sustainability and the considerable financial and political resources necessary to stay the course.

Impact on Teaching and Learning

With respect to classroom teaching, although causal evidence is lacking due to the research designs funded by Hewlett, several studies suggest tenable hypotheses about the impact on classrooms based on correlational and pre-post research designs.  For example, the New Teacher Center’s (NTC) work in Ravenswood School District suggests that intensive novice teacher induction programs may help retain new teachers, with 84% of new teachers returning to the classroom in 2004/05 and 87% returning the following year, compared to 27% in the 2003/04 school year. Similarly, evidence from the BASRC project suggests that inquiry practices (when implemented well by school districts) predicted growth in teacher collaboration, increased instructional focus on developing and assessing students’ higher-order thinking and reasoning skills, and improvement in student performance (Wood & Talbert, 2007). 

With respect to student achievement, evidence of the impact of district-led instructional reform is limited because most of Hewlett supported studies were not designed to permit causal inferences about effects on student outcomes.  While one quasi-experimental study (MDRC study of BASRC) found weak and inconsistent effects on student achievement, research on San Diego using fixed-effects models did find robust evidence of impact on student achievement.  Specifically, the San Diego study shows that district-led reform, if implemented consistently and with fidelity to a well-specified theory of action, can raise student achievement and reduce achievement gaps particularly in elementary school, less so in middle school, and not at all in high school. The cumulative effects of gains in elementary school are quite pronounced:  the entire distribution was shifted upward, with about 10% of students moving up from the two bottom deciles.  While the evidence suggests a qualified success with respect to impact on student achievement, it does offer an existence proof that district-led instructional reform can produce improvement in student outcomes. Further, this evidence suggests, but cannot confirm, that a cohesive theory of action is an essential ingredient for impact on teaching and learning.

Implications for the Future

Substantial evidence, mainly from outside the Hewlett portfolio, points to high-quality teaching as the key mechanism for raising levels and reducing inequalities in student achievement.  Thus, instructional reform is clearly warranted; the question is whether the district is the right place to invest in it.

Considerations on Continued Investment in District-Led Instructional Reform

The main argument in favor of district-led instructional reform is that the district is the most effective leverage point in the education system for addressing issues of scale, sustainability, and capacity-building.  Only at the district level is it feasible to implement, at scale, instructional guides aligned with standards for teaching and learning, and to support these efforts with sustained professional development to improve leadership and teaching.  The state is too far removed from classroom instruction, and teacher development in one school at a time (let alone one teacher at a time) falls far short of the scale that is required to make a profound difference in a large district.  As an intermediary agency, the school district is ideally positioned to implement the blend of incentives, sanctions, and opportunities needed to build capacity for large-scale reform.

An opposing argument is that the district context is too complex and challenging for continued investment to pay off.  A district is beset by competing interests, from the federal and state regulatory environment, to labor unions, to advocacy groups, which make it difficult if not impossible to adopt a coherent and sustained reform policy.  Turnover of leadership and staff is another major factor that inhibits district-led reform. 

On balance, our view is that continued investment would be worthwhile.  The prevalence of unanswered questions, alongside few cases of clear success, indicate that future work would be worthwhile, if the interventions and research were designed to address the most pressing questions.  The fact that the urban district is an extremely difficult environment in which to enact change is not a sufficient reason for abandoning the strategy.

Priorities for Continued Investment in District-Led Instructional Continues

Our review indicates that a program of research and development on district-led instructional reform that focused on the following key areas would build on the existing knowledge base and, over the next five to ten years, make important contributions to the field:

• A program of work that synthesizes existing research and development to elucidate the theories of action on which different district-led instructional reforms operate. This work would need to go beyond simply identifying the key elements of district-led instructional reform to articulating the relations among these elements and specifying how these elements in interaction are hypothesized to improve student achievement.  Without such a systematic synthesis we fear that future work will simply re-invent the existing knowledge base on district-led instructional reform, occasionally re-labeling to give the illusion of novelty.  Further, absent clearly articulated theories of action, research on impact, scale, and sustainability is difficult if not impossible.

• A program of development and evaluation that results in interventions that can produce existence proofs for broad-based instructional improvement for schools and systems with the greatest needs. This work should systematically examine the sustainability and scalability of district-led instructional reform by not only generating hypotheses, but also by testing hypotheses related to scalability and sustainability.

• A program of research that systematically examines the practice of district-led instructional reform by moving beyond the identification of macro-level reform strategies (e.g., data-based decision-making, coaching) to examine how these strategies play out in the day to day practice of the school district.  Such work is essential for addressing the challenges of scale and sustainability.

• A program of research that is designed to assess the impact of district-led reform on school leadership and management, classroom instruction, and student achievement.  Impact can be rigorously assessed through statistical approaches such as the fixed-effects models), or through experimental approaches.