INTRODUCTION

“Leadership is second only to classroom instruction among all school-related factors that contribute to what students learn at school” (Leithwood, Louis, Anderson, & Wahlstrom, 2004, p. 1). This theme is consistent in the literature on school leadership and the effects of the principalship on student learning. Principals matter, and good schools have good principals. However, over the past 20 years, principal training has been under scrutiny.

Numerous organizations, including the National Commission on Excellence in Educational Administration (NCEEA), the National Policy Board for Educational Administration (NPBEA), the Broad Foundation, and the Thomas B. Fordham Foundation, have documented concerns with educational administration programs including their recruitment practices, instructional leadership preparation, professional development, low licensure standards, and lack of real-world problems and experiences. Recommendations from these organizations have included the closure of hundreds of graduate programs in educational administration and/or the abolishment of them to be replaced with alternative programs (Meyer, 2003; Murphy & Forsyth, 1999; National Policy Board for Educational Administration, 1989; National Policy Board for Educational Administration, 1990). Most recently, Levine (2005) identified weak criteria for admissions, irrelevant courses, weak academic rigor, unskilled teachers, and incoherent curricula as problem areas in traditional principal training programs.

In addition to the quality crisis, a quantity crisis has been documented (Educational Research Services, 2000). There are fewer and fewer qualified candidates available to assume the role of principal in American schools (Educational Research Services, 2000). School systems around the nation are attempting to deal with this shortage of leadership at a time when standards and accountability demands are high, stress levels due to the job are high, pressures on local budgets are high, and salaries for the job are low.

If American schools of education are not adequately ensuring that there are quality candidates available to assume the principalship in American schools, then school districts must investigate ways to (1) effectively partner with schools of education as a form of quality control or (2) develop their own principal preparation programs to ensure excellence in every school building. An effective partnership with schools of education can be a practical and cost effective solution to this dilemma.

PROBLEM STATEMENT

With a shortage of candidates to assume the principalship, and with traditional preparation programs being criticized for not adequately preparing future administrative candidates, many school districts are attempting to develop their own principals through district-run programs. “Grow Your Own” principal preparation programs are becoming more common in large school districts, but the literature on grow your own principal preparation programs is scarce (Joseph, 2009; Miracle, 2006; Morrison, 2005). According to Glasman, Cibulka, & Ashby (2002), there are growing numbers of innovative leadership preparation programs around the country, yet there is little or no systematic evaluation of them. As school districts grapple with ways to increase the quality and quantity of principal candidates in their school districts throughout the country, investigations exploring the nature of district developed principal training programs are needed. The purpose of this study was to evaluate a secondary principal development program in a large, high-performing school district in a mid-Atlantic state.

RESEARCH QUESTIONS

The original study used Stufflebeam’s (2000) Context, Input, Process, Product (CIPP) evaluation model as a conceptual framework, and multiple evaluations were conducted in alignment with the framework. This article will focus on the process evaluation from the original study. The process evaluation intended to answer questions such as: Is the program aligned with “best practices” that have been identified in the literature on educational preparation programs? Who is involved in the program, and what is their function? How are program goals communicated and enforced with all members of the program? What systems are place to monitor implementation of the program? The research questions that guided the process evaluation were as follows:

CONTEXT OF THE STUDY

The study was conducted in a school district in the mid-Atlantic region of the United States. District M was the largest school district within its state during the study. District M had both suburban and urban characteristics, and it was one of the most diverse school districts in the state. The average SAT score for the district during the time of the study was 1624 when

averaging the scores on the critical reading, mathematics, and writing subtests. The total possible score on the SAT is 2400. The SAT was the most widely accepted college admissions test in the region in which this study was conducted. There were 200 schools within the school district, and it was highly diverse racially. The racial make-up of the school district during the time of this study was the following: 22.9% African American, .03% American Indian, 15.2% Asian, 21.5% Hispanic, and 40.1% White. One fourth of the students within the district received free or reduced-price meals.

District M began implementation of an Administrative and Supervisory Professional Growth System (A&S PGS) in the 2003-2004 school year with 50 principals. During the 2004-2005 school year, the implementation was expanded to all principals, school based administrators, and central office administrators. The school district described the purpose of the district’s professional growth system for administrators as the following:

The school district had developed a sequence of training programs to prepare future principals: the AP 1 program, the AP 2 program, and the AP 3 program (internship). All of the candidates in these principal training programs had their initial licensure to be an assistant principal in the state in which the district resides. The programs began in the early1990s, prior to the existence of the Administrative and Supervisory Professional Growth System (A&S PGS) in the 2003-2004 school, and have evolved over time due to budgetary constraints. Initially, cohort groups moved from the AP 1 program to the AP 2 program. AP 1s and AP 2s filled assistant principal vacancies within the district, and participants were paid as assistant principals. After completing the AP 2 program, administrative candidates were considered assistant principals within the district. Assistant principals who were deemed ready to assume a principalship were invited to participate in the AP 3 (internship) program.

The secondary AP1 and AP2 programs were for middle school and high school administrative candidates. Administrative candidates participated in a two-year program, which included participating in full-day monthly seminars as a cohort in addition to participating in a professional development team meeting with their principal, an outside principal consultant (mentor), and a central office supervisor (community superintendent or director of school performance). The professional development team met five times throughout the year for two hours each meeting. The AP 1 or AP 2 used this meeting to demonstrate proficiency on the school system’s principal standards by sharing a portfolio of his or her work and reflecting with veteran district administrators and a mentor on the portfolio and related administrative experiences. Upon successful graduation from the AP2 program, candidates deemed ready were invited to participate in the third phase of the program: the internship. This program was for experienced assistant principals, and it assisted these administrators with preparing for the principal interview process within the school district. The program also included a four-week internship program in which the administrative candidates assumed the responsibilities of the principalship.

RESEARCH DESIGN AND METHODOLOGY

Process evaluation is the collection of evaluation data once a program has been designed and put into operation (Gall et al., 1996). Stufflebeam et al. (2000) wrote, “Process evaluation is an ongoing check on a plan’s implementation plus documentation of the process, including changes in the plan as well as key omissions and/or poor execution of certain procedures” (p. 294). According to Stufflebeam et al., the process evaluation should report how observers and participants judge the quality of the process. In this study, the process evaluation included data to understand the systems that were put in place to monitor implementation of the program. The evaluation also included information detailing who was involved in the program and describing how program goals were communicated and enforced with all members of the program. A process check between what the educational literature identifies as best practices and the actual elements of the program was also conducted.

This component of the study relied on three primary sources of data: individual interviews; focus groups of principals, outside consultants, and AP 2s; and document reviews. Interviews with executive staff members of the school district, including the district superintendent, deputy superintendent, chief financial officer, associate superintendent for human resources, associate superintendent for organizational development, the former associate superintendent for organizational development, the chief performance officer, and the former chief performance officer were conducted. In addition, personal interviews were conducted with the administrative union president, three AP 3s (interns), and the director of secondary training. Focus groups were conducted with principals that had trained principal candidates in previous years; and with outside consultants, who were former principals and supported AP 1s and AP 2s as mentors throughout the process. In addition, internal documents and program descriptions were analyzed. Data were coded, chunked, and triangulated to search for patterns and draw conclusions. Table 1 summarizes the research questions, data collection methods, and analysis procedures for the process evaluation.

Initially, the researcher conducted the analysis of the data sources looking for pre-identified themes that emerged from previous literature. Secondly, the researcher organized and identified patterns that were not seen in previous literature. Table 2 summarizes the pre-existing literature that was identified by the researcher prior to the data collection and analysis phase for the context, input, process, and product evaluations.

RESULTS/FINDINGS

Empirical studies supported the following components of an effective principal development program: A) Entrance standards aligned with the realities and duties of the principalship B) A substantive internship C) Based upon clear standards D) Emphasizes reflective practice E) Provides extensive mentoring F) Learning in a cohort model. These factors were used to evaluate the process of implementing the program.

The entrance standards for the administrative pool in District M, to a limited extent, aligned with the realities and duties of the principalship. A former executive staff member who was responsible for the training unit of the school system stated that one of the major improvements the program needed to make was strengthening the entrance requirements for the program. The ISLLC standards, the national standards that District M based its administrative evaluation system upon, delineated a number of areas that are aligned with the demands of the principalship including vision, sustaining school culture, management, collaboration, continuous improvement, and influencing political, social, economic, legal, and cultural context. The program’s present entrance requirements did not align with the standards that candidates were eventually evaluated upon. Executive staff members and program implementers expressed concern with the overall quality of the pool of applicants. All applicants were not accepted into the pool; however, it appeared that the standards could be more rigorous and selective as described in the research literature. Participants should be required to submit a portfolio of their experiences that is verified by their administrators that demonstrated substantive experiences in the areas identified by the district’s standards. Interviews should also be a component of the entrance requirements.

Similarly, the internship had been implemented within the school district to a limited extent. Financial and political factors had limited the number of internships and the length of the internships at the secondary level. Numerous executive staff members and stakeholders recognized that secondary principals were not very receptive to the idea of a secondary internship and opposition was recognized. This was a school district within a major media hub, and there were concerns about potential incidents occurring in the absence of the principal. At the time of this research, secondary interns assumed the role and the responsibilities of the principalship within the school district for 4 weeks at a secondary school. Program participants and executive staff members asserted that this length of time was not extensive enough to provide interns with an authentic principal experience. AP 3’s commented that the time wasn’t enough for them to get an “authentic” principal experience. One AP 3 shared that she felt that some of the people would simply wait for her one month to be over to have the tough decisions answered by the principal when he returned!

District M’s secondary leadership development program, to a great extent, was based upon standards. The ISSLC standards were used to create and evaluate performance of candidates within the secondary leadership development program. These national standards served as a foundation for the training program. The school district adapted some of the standards to make them more relevant for the uniqueness of the school system, but most of the standards were the same as the ISSLC standards.

This study found that there was, to a great extent, an emphasis on reflective practice within the secondary leadership development program. The program provided opportunities for extensive reflection through the professional development team meetings, through working with outside consultants, and through the monthly seminars when all of those structures functioned as designed. Participants were required to reflect in writing and orally, and participants perceived these structured opportunities for reflection to be beneficial to their growth as administrators. Despite some of the positive responses that were given about the nature of their opportunities for reflection, AP 1s, AP 2s, AP 3s, principals, and the administrative union president questioned the relevance of the reflective opportunities provided in the monthly seminars. Time that was not spent specifically developing administrative candidates’ technical skill set was deemed as “fluff”, “a waste of time”, and “irrelevant” by different stakeholders.

In addition, the secondary leadership development program provided an extensive mentorship program through experienced administrators. Participants were provided with a number of opportunities to be mentored by veteran administrators. Outside principal consultants were assigned to all participants in the program. In addition, a community superintendent or a director of school performance met with participants on five different occasions during the course of the school year to support participants’ growth. Participants were also introduced to a number of administrators from different offices through participation in monthly seminars. In addition, participants received support from the director of secondary leadership development.

Relationships were fostered by design as participants in the secondary leadership development program participated in cohort groups. Over the course of the 2 or 3 years participants were in the program, they met monthly and trained together. Participants were given opportunities to share best practices and offer suggestions to one another. They viewed their ability to come together to learn and share as a strength of the program. Many participants that ascended to principalships commented on how the relationships that were fostered through their experiences as a cohort member in the secondary leadership development program continued to be beneficial to them in their roles as principals. All stakeholders described the lasting, trusting relationships that were developed through the use of cohort learning models.

District M’s secondary leadership development program was not consistently being implemented as designed. There were concerns expressed about the development of the content of the monthly seminars. The program received an annual review, but principals, administrative participants, and the union president did not believe participants were consistently being trained on relevant topics. The director of secondary leadership development felt that the annual review of the program and the program’s content was adequate, but stakeholders, including principals, the administrative union president, AP 1s, AP 2s, and AP 3s did not feel that there was enough input and communication regarding the content of the monthly seminars. The need for collaboration in the development of seminars was identified as a concern for the program and its implementation.

Participants cited an implementation concern regarding the principal’s ability to serve as primary trainer. Although the principal accounted for 88% of the training of an administrative participant, there was limited training provided to principals. As a result, there was variance in the extent to which administrative participants were trained, as well as the opportunities administrative participants were given, based upon their work location and the experiences of the principal. Further, stakeholders including the administrative union president and the executive staff member responsible for the office that evaluates schools and principals shared that some principals did not want to serve as principal trainers but were required to do so simply because they had an administrative vacancy in their schools. The executive staff member responsible for the office that oversees schools and principals shared that District M was beginning to rethink how administrative candidates are assigned to trainers and his office was interested in ensuring that his best principal trainers work with administrative candidates. Yet, at the time of this study, vacancies dictated who the principal trainer was for administrative candidates.

Participants and stakeholders saw the value in having principal consultants, but they reported inconsistency in how the job was performed. Some principal consultants communicated effectively with participants and principals and some did not. There was also inconsistency regarding the amount of time principal consultants provided program participants. Although principal consultants should work 5 hours per month with AP 2s and 8 hours per month with AP 1s, stakeholders reported inconsistency with some outside consultants giving more hours than was required while other outside consultants worked less that the required amount of hours.

With regard to implementing the professional development team meeting process, there was inconsistency regarding acceptable standards of performance for administrative participants. The administrative union president, principals, and outside principal consultants reported instances in which underperforming candidates’ were allowed to exit the program with a rating of proficient. Performance outcomes of the program were evaluated subjectively, thereby causing a variance in expectations and performance between different development teams.

IMPLICATIONS FOR PRACTICE

The need to strengthen traditional university-based administrative training programs has been documented (Jackson & Kelley, 2002; Levine, 2005), and the limited research that has been conducted on district “grow your own” principal development programs suggest that “grow your own” district run programs can be strengthened (Joseph, 2009; Miracle, 2006; Morrison, 2005). School districts and schools of education have the power to collaborate to create substantive, meaningful experiences that can adequately give principal candidates relevant theory and relevant experiences to lead schools of excellence. The following are some suggestions that can support a true partnership between university personnel and school district personnel.

Memorandums of understanding should be developed between school districts and regional graduate schools of education to support each institution’s efforts to increase the quantity and quality of principal candidates. Graduate schools of education can provide support to school districts attempting to develop district level principal training programs by assisting districts to ensure that their programs are aligned with best practices noted in research findings. The district in this study had a full-time director of leadership development who researched, designed, and implemented the program. A partnership with local universities would ensure that leadership development programs are not based solely upon the efforts and expertise of one individual within a school district. Faculty from educational leadership departments could serve as members of leadership development advisory boards that would meet frequently to discuss appropriate educational theory that should be introduced during cohort seminars.

This study also found that some principals within District M did not feel adequately prepared to serve as trainers of administrative interns. Serving as a trainer requires a different subset of skills than serving as a principal. This suggests that there may be broader concerns regarding principals’ self-efficacy. Faculty from educational leadership departments may have the capacity to support school districts whose principals needed additional training to perform their primary duties or their duties as a trainer of administrative leaders.

In addition, graduate schools of education can support local school districts in providing objective measures to evaluate progress on identified outcomes within school district run principal development programs. To limit the subjectivity of performance based assessments, rubrics could be developed and sent to university evaluation staff for objective feedback. The school district in this study, and research findings from previous research (Miracle, 2006; Morrison, 2005), found that grow your own programs are not receiving the critical evaluation and feedback that they need to continuously improve. One challenge may be that many districts do not have evaluation/accountability departments that are capable of effectively evaluating the effectiveness of programs. This is an area where there are opportunities for graduate schools of education to partner with local school systems to ensure programs are receiving the critical feedback they need to improve.

Concomitantly, graduate schools of education can work with local school districts to systematically provide administrative candidates with significant opportunities to reflect through exposing administrative candidates to leadership theory. An understanding of critical theory and its application to new situations is critical for administrators. The reality of the principalship is that new challenges and new situations arise daily. It is unrealistic to assume that any training program can provide a “cook book” of technical skills that will ensure administrative candidates’ success. As was the case in this study, many administrative participants do not understand the significance of “moving to the balcony” and reflecting to acquire new learning. School districts, which may be limited by human resources, may not have the capacity to effectively research and train administrative candidates on current, relevant theories of leadership using an interdisciplinary approach.

An unfortunate reality is that most educational theory that is introduced to administrative candidates is taught while the candidates are not serving as administrators. As a result, many candidates may not critically apply theory to their current practice. If this is true, it points to a weakness in the design of current educational administration programs. Adult learning theorist have communicated the importance of timing and relevance in an adult’s learning process (Brookfield, 1985; Mezirow, 1998) Reintroducing relevant theory when a candidate is in an administrative role is a promising practice that can result in substantive, reflective learning. The opportunity for an administrative candidate to understand and apply theory in a real educational setting has the potential to increase candidates’ learning.

School districts, partnering with graduate schools of education, can help graduate schools strengthen their programs. Administrative candidates need access to authentic experiences if they are going to develop the competencies that educational administrative jobs demand prior to obtaining an assistant principalship or a principalship. School districts can create more substantive internship experiences for administrative candidates by partnering with universities to identify and define extended administrative internship opportunities. Graduate school of education syllabi can be developed in conjunction with school districts and school district needs can be used to define required internship experiences for students. Graduate school faculty partnering with practitioners to develop internship requirements for courses will strengthen the credibility and worth of internship experiences. Internship opportunities should extend beyond the walls of a candidate’s school and/or school district to provide administrative candidates with meaningful, substantive experiences. The practice of completing internship experiences after work at a candidate’s school with no consideration of the quality and/or demands of the experience must be reconsidered if graduate schools of education want to ensure a quality experience for administrative candidates. The findings from this study reaffirmed the need for an extended amount of time for an internship experience and a quality experience during the internship period regardless of whether the internship occurs prior to obtaining an administrative experience or after an administrative experience is obtained.

Services rendered between university personnel and school district personnel should be cost neutral. Graduate School of Education faculty should be encouraged to support school district goals so that they are able to gain access to authentic educational settings to conduct meaningful investigations which promote scholarship and improve student outcomes. School district personnel should eagerly allow university faculty access to programs and research opportunities in return for gaining faculty expertise on school district issues. Memorandums of understanding should seek to leverage concepts of mutual interests in lieu of financial benefits for either party.

CONCLUSION

There are numerous benefits for school districts and graduate schools of education to partner to support the development of principal candidates. If graduate schools do not seek opportunities to collaborate and provide authentic experiences to their students, they will continue to be criticized for not addressing the realities of the profession. Similarly, school districts may continue to develop “grow your own” principal preparation programs to try to improve the quality of post-graduate school administrative candidates, but with shrinking budgets, shrinking central offices, and increased demands, their ability to effectively execute will be limited. A continuum of training beyond initial licensure that partners school districts and graduate schools of education has the potential to provide benefits for all stakeholders including the most important of all: students.

REFERENCES

Alexander, G. C. (1992). The transformation of an urban principal: Uncertain times, uncertain roles. (ERIC Document Reproduction Services No. ED 348461)

Bottoms, G., O’Neill, K.; Frye, B., & Jacobson, A. (2004). Progress being made in getting a quality leader in every school. Atlanta, GA: Southern Regional Education Board. Retrieved October 12, 2007, from http://sreb.org/main/Goals/Publications/04E12_Quality_Leader.pdf

Brookfield, S. (1985). A critical definition of adult education. Adult Education in the Health Professions, 36(1), 44-49.

Browne-Ferrigno, T. (2001, November). Preparing school leaders: Case study summary and implications. (ERIC Document Reproduction Services No. ED465226)

Cooley, V. E. & Shen, J. (2003, March). School Accountability and professional job responsibilities: A Perspective from secondary principals. NAASP Bulletin, 87(634), 10-25.

Cox, H. S. (1998). Effectiveness of principal preparation in Florida as perceived by selected superintendents, first-year principals and other key informants.Dissertation Abstracts International. ( UMI No. 9838576)

Educational Research Services. (2000). The principal, keystone of a high achieving school: Attracting and keeping the leaders we need. Washington, DC: National Association of Elementary School Principals and National Association of Secondary School Principals.

Educational Research Services. (1998). Is there a shortage of qualified candidates for openings in the principalship? An exploratory study. Washington, DC: National Association of Elementary School Principals and the National Association of Secondary School Principals.

Gall, M. D., Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. White Plains, NY: Longman Publishers.

Glasman, N., Cibulka, J., & Ashby, D. (2002). Program self-evaluation for continuous improvement. Educational Administration Quarterly—Special Issues: Ensuring the Capacity of University-based Educational Leadership Preparation,38(2), 257-288.

Goodwin, R. H. (2002). On the edge of chaos: A Delphi study of the changing role of the secondary principal. Dissertation Abstracts International. (UMI No. 3055917)

Herr, N. J. (2002). Addressing shortages in the principalship: Perceptions of novice administrators in Pennsylvania. Dissertation Abstracts International. (UMI No. 3097229)

Jackson, B., & Kelley C. (2002, April). Exceptional and innovative programs in educational leadership. Educational Administration Quarterly, 38, 192-211.

Joseph, S. (2009). A comprehensive evaluation of a school system’s grow your own principal preparation program. Dissertation Abstracts International. (UMI No. 3344118)

Lauder, A. (September, 2000). The new look in principal preparation programs. NASSP Bulletin,84(617), 23-28.

Leithwood, K., Louis, K. S., Anderson, S., & Wahlstrom, K. (2004). How leadership influences student learning. Retrieved November 15, 2007, from http://www.wallacefoundation.org/KnowledgeCenter/KnowledgeTopics/EducationLeadership/HowLeadershipInfluencesStudentLearning.htm

Levine, A. (March, 2005). Educating school leaders. Retrieved November 18, 2007, from http://www.edschools.org

Love, D. W. (2000). The availability of qualified candidates for the secondary principalship in Arkansas. Dissertation Abstracts International. (UMI No. 3010168)

Maryland Task Force on the Principalship. (2000). Recommendations for redefining the role of the principal; recruiting, retaining, and rewarding principals; and improving their preparation and development. Baltimore, MD: Maryland State Board of Education.

McCarthy, M. M. (1999). The evolution of educational leadership preparation programs. In J. Murphy & K. S. Louis (Eds.), Handbook of research on educational administration (2^nd ed.) (pp. 119-139). San Francisco: Jossey-Bass.

McFadden, C., Mobley, D., Burnham, J., Joyner, R. L., & Peel, H. A. (2003). Are national ISLLC standards important to job performance? An Eastern North Carolina perspective. In F. C. Lunenburg & C. S. Carr (Eds.), Shaping the future: Policy, partnerships, and emerging perspectives (pp. 389-400). Lanham, MD: Rowman & Litchfield.

Mercado, L. P. (2002). The secondary principalship academy. A critical ethnography of the Houston Independent School District and the University of Houston’s innovative principal preparation program. Dissertation Abstracts International. (UMI No. 3067773)

Meyer, L. & Feustrutzer, E. (2003). Better Leaders for America’s Schools. (ERIC Document Reproduction Services No. ED479070)

Mezirow, J. (1998). On critical reflection. Adult Education Quarterly, 48(3), 185-198.

Miracle, T. L. (2006). An analysis of a district-level aspiring principals training program. Dissertation Abstracts International. (UMI No. 3228666)

Morrison, H. (2005). A critical evaluation of a school system’s effort to develop and implement a grow your own principal preparation program.Dissertation Abstracts International. (UMI No. 3178557)

Murphy, J. (1994). Transformational change and the evolving role of the principal: Early empirical evidence. (ERIC Document Reproduction Services No. ED374520)

Murphy, J., & Forsyth, P.E. (1999). Educational Administration: A decade of reform. Corwin Press: Thousand Oaks

Newman, C. A. (2004). Interim evaluation of the UNT/Dallas Public School Leadership Development Program: A working model.Dissertation Abstracts International. (UMI No. 3126581)

National Policy Board for Educational Administration. (1989). Improving the Preparation of School Administration: An Agenda for Reform. (ERIC Document Reproduction Services No. ED 310493)

National Policy Board for Educational Administration. (1990). Alternative certification for school leaders. (ERIC Document Reproduction Services No. ED 326969)

Portin, B. S. (1997). Complexity and capacity: A survey of principal role change in Washington state. (ERIC Document Reproduction Services No. ED414 624)

Stufflebeam, D. L., Madaus, G. F., & Kellaghan, T. (2000). Evaluation models (2^nd ed.). Boston: Kluwer Academic.

Wendel, F. C. (1992). Reforming administrator preparation programs. (ERIC Document Reproduction Services No. ED366807)

Wilmore, E. L. (2002). Principal leadership: Applying the new Education Leadership Constituent Council (ELCC) Standards. Thousand Oaks, CA: Corwin Press, Inc.