Similar Students, Different Results

From the latest Teacher's College Record.

It looks like a solid study, but I have one caveat. One of the findings is that successful schools are aligned with the State Standards and success is then measured by these standards. This does raise questions about the content of these standards. The creation of these standards has been highly political and in some cases the resulting standards leave much to be desired. For an earlier California story, see Gary Nash, Charlotte Crabtree and Ross Dunn's History on Trial.

http://www.amazon.com/gp/product/0679767509/002-1214435-6720800?v=glance&n=283155

TJM

http://www.tcrecord.org/Content.asp?ContentID=12299

Similar Students, Different Results by Trish Williams & Michael W. Kirst — January 25, 2006

Why do some California elementary schools serving largely low-income students score as much as 250 points higher on the state's academic performance index (API) than other schools with very similar students?

That’s the research question asked by a new, large-scale EdSource-led study that surveyed principals and teachers in 257 such schools across the state. What we learned is that the higher performing schools tend to have four interrelated practices at the core of their operation—prioritizing student achievement; implementing a coherent, standards-based curriculum and instructional program; analyzing student-assessment data from multiple sources; and ensuring availability of instructional resources.

Many studies have examined successful schools as a group, in an effort to understand their methods or best practices. This study—conducted by EdSource and researchers from Stanford University, the University of California, Berkeley, and the American Institutes for Research—took a different tack. Rather than looking at a specific performance zone, we examined elementary schools within a specific, fairly narrow socioeconomic and demographic band but across the full range of school performance.

Specifically, we garnered the participation of 257 out of the 550 elementary schools that fall within the 25–35th percentile band of California’s School Characteristics Index (SCI). This band of schools has high levels of students from low-income families, or from ethnic minorities, and/or who are learning English as a second language. Our dependent variable was the school’s Academic Performance Index (API), which is based annually for these schools on student scores in English Language Arts and math on the California Standards Tests in grades 2–5. We surveyed all the principals and 80 % of the K–5 classroom teachers at the schools in our sample, giving us 257 principal surveys and over 5,500 teacher surveys.

Our surveys included over 300 items each covering seven broad domains often associated with effective schools. The questions were updated to reflect California’s current standards based reform environment, asking specifically what principals and teachers were doing, and how frequently. Once the data was collected, we used a weighted analysis to assure that results were statistically representative of all public, non-charter elementary schools in the 25–35th SCI band. We then used several types of regression analyses to determine which school practices differentiated the highest performing schools from the lowest performing ones.

WHAT SCHOOLS DO CLEARLY MAKES A DIFFERENCE

We’ve heard people say that you can guess a school’s Academic Performance Index (API) score by the zip code of its students. But the 250-point difference in API between schools serving similarly disadvantaged students shows that’s not true. What do the higher performing schools do differently? From the school district through to the classroom, they focus on improving student achievement against the state’s academic standards and align all their practices around that goal. Of the seven domains we studied, these four had the biggest impact on a school’s API:

• Prioritizing Student Achievement. Where teacher and principal answers to multiple survey questions indicated higher expectations for students, their schools had, on average, higher API scores than schools whose staffs indicated lower expectations. In more successful schools, teachers and principals alike reported that their schools have well defined plans for instructional improvement and that they put priority on meeting the state’s API goals and the federal adequate yearly progress goals. Teachers and principals also report that their schools set measurable goals for exceeding the mandated API student subgroup growth targets for improved achievement.
  
  
• Implementing a Coherent, Standards-based Curriculum and Instructional Program. Teachers who report the following were more likely to be in higher performing schools: instructional consistency within grades; curricular alignment from grade-to-grade; classroom instruction guided by state academic standards; curriculum materials in math and language arts aligned with the state’s standards; in a district that addresses the instructional needs of English learners at their school. Principals were more likely to be in higher performing schools if they reported that the district has clear expectations for student performance aligned with the district’s adopted curriculum and that the district evaluates the principal based on the extent to which instruction in the school aligns with the curriculum.
  
  
• Using Assessment Data to Improve Student Achievement and Instruction. Strongly correlated with a higher school API was the extensive use of student assessment data by the district and the principal in an effort to improve instruction and student learning. For example, principals more often reported that they and the district use assessment data from multiple sources (including frequent curriculum program tests and annual California Standards Tests) to evaluate teachers’ practices and to identify teachers who need instructional improvement. Principals report using this data to develop strategies to follow up on the progress of selected students and help them reach academic goals. According to these principals, the district expects all of its schools to improve achievement, evaluates principals based on student achievement, and provides support for site-level planning related to improving achievement.
  
  
• Ensuring Availability of Instructional Resources. Where more teachers reported having regular or standard certification for teaching in California, schools had, on average, higher API scores. The same was true of schools where principals more often reported that their districts provide sufficient and up-to-date instructional materials as well as support for supplementary instruction for struggling students and for facilities management. Teachers with at least five years of full time teaching experience were more likely, on average, to be from schools with higher APIs. Principal experience was also correlated with higher school achievement.

The survey also included questions related to schools’ efforts to involve parents, teacher collaboration and development, and the enforcement of high expectations for student behavior. Although each of these types of practices made some contribution to a school’s API score, and were likely practiced by most of the schools in the sample, they were not nearly as strongly correlated with higher school performance as were the four key interactive, interdependent school improvement practices described above.

WHAT THE STUDY FINDS ABOUT TEACHING AND LEARNING

Our study underscores that there is no single silver bullet for school success; multiple factors and practices are at work. Yet it appears that in the context of standards based, accountability-driven reform, at least for elementary schools with large numbers of low-income students, a major implication is that the caliber of teaching and learning appears directly related to the level of system-wide coherence. The clearer the strategic through line from district to school to classroom, the more likely it is that higher achievement will result. Put another way, the more all the requisite pieces are aligned, seen as interdependent parts of a whole, and overtly structured to support the work of teachers in the classroom, the stronger the benefits to learning.

The study provided some glimpses of how this looks in action:

• District practices include setting clear expectations that schools meet API and AYP growth targets, including for subgroups, as well as providing schools with achievement data and evaluating principal performance and teacher practices based on that data. They also include ensuring that math and language arts curricula are aligned with state standards, that instruction is focused on achievement and that schools have adequate facilities and textbooks as well as resources for struggling students.
  
• Principal leadership is redefined to focus on effective management of the school improvement process. In general, API scores were higher in schools with principals whose responses indicate that they act as proactive managers of school improvement, driving the reform process, cultivating the school vision, and extensively and personally using student assessment data for a wide variety of school improvement areas of focus, including evaluation of teacher practice and assistance to struggling students.
  
  
• An effective teaching staff appears to be characterized by certain ingredients. In higher achieving schools, principals reported that a higher proportion of their teaching staff had, in order of priority: a demonstrated ability to raise student achievement, strong content knowledge, characteristics that made them a good fit with the school culture, training in curriculum programs, and the ability to map curriculum standards to instruction. These teachers were also supportive of colleagues’ learning and improvement, able to use data from student assessments, familiar with the school community, excited about teaching, and familiar with state standards.

ADDITIONAL RESEARCH QUESTIONS

An interrelated question we plan to follow up on is the relationship between school-wide use of specific curriculum packages, other curricular and instructional school practices, and school performance. In addition, our teacher survey included a robust set of questions around EL instruction that have yet to be analyzed.

Reference

Williams, T., Kirst, M., Haertel, E., et al. (2005). Similar students, different results: Why do some schools do better? A large-scale survey of California elementary schools serving low-income students. Mountain View, CA: EdSource.

The initial findings report, Similar Schools, Different Results: Why Do Some Schools Do Better?, including appendices with more details on the research methodology and the demographics of the school and student sample, can be found at www.edsource.org

Cite This Article as: Teachers College Record, Date Published: January 25, 2006
http://www.tcrecord.org ID Number: 12299, Date Accessed: 4/25/2006 3:24:06 PM

Posted by Thomas J. Mertz at April 25, 2006 2:26 PM
Subscribe to this site via RSS/Atom: Newsletter signup | Send us your ideas