Research to this point in time has clearly indicated that there are practices which do work to produce positive academic outcomes in urban districts (Council of the Great City Schools, 2002). Researchers are also voicing more firmly than ever that the secret to educational success has been uncovered (DuFour, 2004b; Schmoker, 2004b), and that practices such as teacher collaboration, reflective practice, professional learning communities, job-embedded professional development, and distributed leadership have produced positive academic outcomes when implemented consistently (DuFour, 2004b; Fullan, 2001; Schmoker, 2004a, 2004b; Timperley, 2005).

School reform initiatives continue to be a constant in the education system. Since the National Commission on Excellence in Education published the 1983 report, A Nation at Risk: The Imperative for Educational Reform, a variety of reforms has been implemented, some with limited success. (Fullan, 2001). The struggle to positively change high schools through initiatives such as smaller learning communities (SLCs) is often described in the media due to publicity of promoters such as Bill and Melinda Gates and Warren Buffett.

Accountability has become a required measuring tool with severe consequences at both the federal and state level for campuses and districts that do not meet required levels of student achievement. Focusing on implementing effective practices leading to student growth has become even more necessary. This age of accountability has focused attention on not only what is taught but also how it is taught (Jacob, 1997, 2004; Marzano, Pickering, & Pollock, 2001). There is a need for reliable assessments aligned to rigorous curriculum, and to know how to use information collected from assessments (Stiggins, 2004). Learning Points and Associates (2000) report that a process must be in place to help campuses understand how to read data and turn the information into an action plan:

When this process is absent, confusion reigns. Staff from (…) schools indicated that they did not see the connection among teacher-administered in-class assessments, their norm-referenced district test, and the large-scale state assessment. Nor did they know what to do with this information. (Learning Point Associates, 2000, p. 1)

The need to bring a collaborative assessment culture to a campus is apparent (Bernhardt, 1999; DuFour & Eaker, 1998; Holcomb, 2004; Schmoker, 2004b; Stiggins, 2004). The DCM process requested that campuses use a collaborative culture as a way to encourage teachers to meet regularly and discuss data and student work in a safe and productive manner. Encouraging professional dialogue, sharing of strategies, literature studies, a shared vision, and distributed leadership are components of a professional learning community (DuFour, 2004b). The significance of a professional learning community becomes evident when one takes into consideration the seriousness of the task to be completed. Teachers must also be prepared to engage in continuous learning throughout their careers by using job-embedded professional development practices. Those practices could include book/article studies, action research, peer mentoring and peer coaching, reflective logs and other practices.

Implication of Findings

The study of the data analysis model was designed to measure the impact of the Data Collaborative Model on a large urban school district by analyzing a criterion-referenced test. The state test, Texas Assessment of Knowledge and Skills (TAKS) was used to measure student achievement. The hypothesis predicted a statistically significant difference in TAKS math and reading passing rates after using the Data Collaborative Model for a three-year period of time. By the end of the third year, the high implementation group had a mean percent gain in reading of 7.19%. While not statistically significant, the mean percent gain was higher than the low implementation group (2.41%). The high implementation group’s gain in TAKS math was statistically significant at 8.51% compared to the low implementation group (2.72%). High implementing campuses also had higher means than low implementation campuses on their perception of DCM’s impact on achievement, understanding the DCM tools, and actually using DCM tools.

In the case of the statistical significance found for the high implementation group with TAKS math passing rates, one would have to complete further study to see what other factors were impacting teacher effectiveness at those campuses. One possible answer could be attributed to the math learning community’s pilot that began in 1998 on a few campuses and has since spread to many campuses across the district. If the significance can be traced to that initiative, then it would give credence to the research that suggest most new initiatives take several years to become sustained in the educational environment of the campus or district. Speck (1996) concluded from his research that for substantial change in educational practices to happen professional development must be conducted over several years. Significant change in school practice can take four to seven years, or longer. District administrators should take into account this longer time frame when considering the measured outcomes of recently implemented initiatives.

Twelve elementary campuses were selected based on their personnel’s survey responses. Although all 12 campuses in the study were exposed in varying degrees to the DCM and all campuses showed gains on the percent of students passing TAKS, ANOVA analyses failed to prove DCM implementation had an impact on TAKS gains. Completing the study after three years of DCM implementation may not have given the campuses enough time to fully implement the initiative.

There seemed to be a discrepancy when it came to DCM tool usage on the 12 campuses between the principals and the teachers. Although principals were more apt to say they understood and implemented the DCM process, teachers reported implementing it at a lower rate. This could be attributed to principals encouraging teachers to take part in the DCM process, but not actually taking part in the practice themselves. Of the principals who responded, 58% said they used and understood the DCM TAKS tools, but only 17% of teachers responded similarly. For example, 83% of the principals claimed they understood and used the pre-slugged template, a benchmark tool sent to the campuses after each district-formative assessment was scored, but only 20% of teachers said they used this same tool. This may be a reflection of principals who thought the tools were understood and used on their campus or they exaggerated their responses for positive survey results.

Change comes slowly to any organization (Marzano, Zaffron, Robins, Zraik, & Yoon, 1995), but it is especially hard in large bureaucracies. The 12 schools studied reflected urban trends seen by many large districts (Council of the Great City Schools, 2004). Trends such as initiatives not monitored to full implementation; high principal turnover; changes in sub-districts and/or feeder patterns; frequent general superintendent turnover; and a curriculum lacking in rigor were evident throughout the study. Seven of the 12 campuses studied had seen two or three leadership changes during the three years of this study. Six of the 12 campuses changed sub-area and or feeder patterns, which meant a change in focus as per the superintendent of that sub-area. In half of those six cases the sub-area changes occurred along with principal turnovers. New top leadership impacted initiatives and performance targets; and small gains in student achievement could not be attributed to any one cause of practice due to a lack of consistent implementation in a sub-district or single campus.

The district in this study did not have a rigorous and relevant curriculum in place long enough to make a difference in scores, nor did it have stable leadership, efficient management, and strong parental and community support. The district had “pockets of excellence” depending on how well read school leaders were on issues such as how to implement effective instructional practices on their campuses. But as Fullan stated in 2001, “stimulating, coordinating, and sustaining ‘coherent’ development across many schools is exceedingly difficult because it requires balancing top-down and bottom-up forces” (p. 170)

Research points to effective practices that work with urban schools. These practices are working for various reasons. The Council of the Great City Schools reports that progress is being made toward the performance benchmarks that states have established (2004). This can be attributed to, “higher standards; strong and stable leadership; better teaching; more instructional time; regular assessments; stronger accountability; parental and community support; and efficient management” (p. 65).

The research gives hope to urban districts. There are practices that can make a difference district-wide, but the execution of the process must be well designed and systemic. As Fullan wrote in 2001, “support of central administrators is critical for change in district practice. …general support or endorsement of a new program by itself has very little influence on change in practice (e.g., verbal support without implementation follow through)” (p. 81). In the case of the district in this study, although the superintendent at the beginning of the 2004-2005 and the interim superintendent of the 2005-2006 school year both endorsed the DCM initiative, little effort was made to supervise its implementation through the sub-districts.

Elmore wrote in 2005 that, “Improvement at scale is largely a property of organizations, not of the pre-existing traits of the individual who work in them. Organizations that improve do so because they create and nurture agreement on what is worth achieving” (p. 25). For a district to see change in its schools, it must begin with “reculturing towards interactive, accountable, inclusive professional learning communities” (Fullan, 2001, p. 180). Elmore (2005) adds that change in the organization itself, in its very culture, is also necessary. Organizations must put in place the internal processes to help people gradually learn how to implement the change in order to achieve the desired goal. In other words, helping people deal with change in a realistic manner along with implementing new practices should be a part of the internal process.

Policy Implications

Among the best practices hailed by researchers, such as DuFour (2004b) and DuFour, DuFour, Eaker, and Karhanek (2004), DuFour and Eaker (1998), Fullan (2005), Schmoker (2004a, 2004b), and others, is the professional learning community (PLC), which provides a collaborative culture on a campus. One of the major strengths of the PLC cycle is its design as a job-embedded professional learning process that is ongoing and results driven. Multiple correlation studies concerning teacher quality (Darling-Hammond, 2000; Darling-Hammond, Hightower, Husbands, LaFors, Young, & Christopher, 2003) indicate that higher levels of student achievement are associated with educators who participate in sustained professional development based on content-specific pedagogy. Continuous collaborative professional learning increases teacher outcomes which, in turn, impacts student academic outcomes in a positive manner.

Conclusion

Legislators tend to think of accountability as something new since NCLB, but schools have always been held to some level of accountability, dependent on the guiding policies within their district. What has not changed much within this period is that when there has been change, it has not been sustained. We must move from a culture in which the work of the organization is the sum of the work of its individuals to a culture in which individuals’ work is shaped by collective expectations, values, and commitments. This requires the exercise of agency at both the individual and collective level (Elmore, 2005). Elmore also notes that the current working model of accountability is flawed. Accountability needs to be the organizational response to the needed change, not just action toward compliance or implementation.

Contained within the process of the Data Collaborative Model are research-based best practices that have made other urban schools around the country successful. The district in this study should take a second look at the implementation process of best practices to see if additional resources, time, and attention would benefit its campuses. With the right environment, a process like the Data Collaborative Model could flourish and ultimately lead students, teachers, and administrators to new achievements followed by deeper learning.

Lessons Learned

Utilizing data to analyze and improve instruction is a daunting process with many pit falls. Stakeholders must be able to stay on course with the slow start up process. They must realize that the returns will be vast if they continue the process through a minimum of three to five years before seeing more than small academic results. Collaboration and practices in learning communities add positively to the dynamics of the process resulting in improvement.

How do we help teachers begin the data analysis process?

This is a question that often halts the improvement process. The answer to this question is to encourage teachers and administrators to keep current on what is working in other similar schools through the reading of educational journals and books. Reading and discussing practices that work in higher achieving schools keep people motivated to work harder than they might otherwise. Reading others’ stories also alerts them to the slow downs in progress that can impede the process of change. It also reminds them that change is slow; it cannot be rushed if sustainability is the desired outcome. As Fullan (2005) and Schmoker (2004b) have reminded educators time and again, the secret to helping teachers and students become successful learners is known…now they just need to be embraced.

References

Bernhardt, V. (1999). The school portfolio: A comprehensive framework for school improvement. New York: Eye on Education.

Council of the Great City Schools. (2002, September). Case studies of how urban school systems improve student achievement. Retrieved on March 17, 2006, from http://www.cgcs.org/reports/Foundations.html

Council of the Great City Schools. (2004). Beating the odds IV: A city-by-city analysis of student performance and achievement gap on state assessments: Results from 2002-2003 school year. Retrieved on March 17, 2006, from http://www.cgcs.org/reports/beat_the_oddsIV.html

Darling-Hammond, L. (2000). How teacher education matters. Journal of Teacher Education, 51(3), 166-173.

Darling-Hammond, L., Hightower, A., Husbands, J., LaFors, J., Young, V., & Christopher, C. (2003). Building instructional quality: "Inside-out" and "outside-in"-Perspectives on San Diego's school reform. Seattle, WA: Center for the Study of Teaching and Policy, University of Washington.

DuFour, R. (2004a). The best staff development is in the workplace, not in a workshop. Journal of Staff Development, 25(2), 63-64. Retrieved on March 19, 2006, from National Staff Development Council Web Site: http://www.nsdc.org/library/publications/jsd/dufour252.cfm

DuFour, R. (2004b). What is a professional learning community? Educational Leadership, 61(8), 6-11.

DuFour, R., & Eaker, R. (1998). Professional learning communities at work: Best practices for enhancing student achievement. Bloomington, IN: National Education Service.

DuFour, R., DuFour, R., Eaker, R., & Karhanek, G. (2004). Whatever it takes: How professional learning communities respond when kids do not learn. Bloomington, IN: National Education Service.

Elmore, R. (2005). Building a new structure for school leadership. Washington, DC: The Albert Shanker Institute.

Fullan, M. (2001). The new meaning of educational change. New York: Teachers College Press.

Fullan, M. (2005). Professional learning communities writ large. In R. DuFour, R. DuFour, & R. Eaker (Eds.), On common ground. Bloomington, IN: National Education Service. Retrieved on March 27, 2006, from http://www.michaelfullan.ca/Articles_05/UK_Ireland_preread_final.pdf

Holcomb, E. (2004). Getting excited about data second edition: Combining people, passion, and proof to maximize student achievement (2nd ed.). Thousand Oaks, CA: Corwin Press.

Jacob, H. (1997). Mapping the big picture: Integrating curriculum and assessment K -12. Alexandria, VA: Association for Supervision and Curriculum Development.

Jacob, H. (2004). Getting Results with Curriculum Mapping. Alexandria, VA: Association for Supervision and Curriculum Development.

Learning Point Associates. (2000, November). In using student assessment data: What can we learn from schools? Retrieved on March 18, 2006, from http://www.ncrel.org/ library/ publications/tools/tools2-03holl.cfm

Marzano, R., Pickering, D., & Pollock, J. (2001). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA: Association for Supervision & Curriculum Development.

Marzano, R., Zaffron, S., Robins, S., Zraik, L., & Yoon, L. (1995). A new paradigm for educational change. Education (pp. 162-173). San Francisco, CA: Landmark Education.

Schmoker, M. (2004a). Start here for improving teaching and learning. School Administrator. Retrieved on March 18, 2006, from http://www.findarticles.com /p/articles/mi_m0JSD/

Schmoker, M. (2004b). Tipping point: From feckless reform to substantive instructional improvement. Phi Delta Kappan, 85(6), 424-432. Retrieved on March 19, 2006, from http://www.pdkintl.org/kappan/k0402sch.htm#1a

Speck, M. (1996, Spring). Best practice in professional development for sustained educational change. ERS Spectrum, 33-41.

Stiggins, R. (2004, September). New assessment beliefs for a new school mission. Retrieved on March 18, 2006, from http://www.assessmentinst.com/documents/sdc.org/library/publications/jsd/stiggins202.cfm

Timperley, H. S. (2005). Distributed leadership: Developing theory from practice. Journal of Curriculum Studies, 37, 395-420.