Knowledge versus experience is often a consideration in employment decisions regardless of the field. When it comes to educational leadership, finding qualified and knowledgeable candidates with credentials is much easier than finding those with experience in leadership activities necessary in modern schools (Hale & Moorma, 2003). The best predictor of what a person will do in the future is what they have done in the past. Practice may not make perfect, but it improves the odds for eventual success. But how, in a non-internship program, does one practice being a school leader when one's range of experience is most often limited to classroom teaching?

The opportunity and design of meaningful experiences can vary; whether through internships or fieldwork, situated learning in authentic workplaces has been the hallmark of exemplary school leadership preparation programs (Browne-Ferrigno, 2003; Jackson & Kelly, 2002; Perez, Uline, Johnson, James-Ward, & Basom, 2011). In a year-long study of 18 students participating in a principal licensure program, Browne-Ferrigno (2003) found that informal and informational experiences helped participants frame their concept of the principalship; “…the purpose of field-based learning guided by leadership practitioners is to begin initial socialization into a new community of practice (p. 495).”

Many notable principal preparation programs assign concurrent course-specific field experiences within classes across the duration of programs so that students acquire practical experience (Dishman & Redish, 2011; Jackson & Kelly, 2002; Kirkpatrick, 2000; Orr, 2006; Perez, et al., 2011). Researchers have found that it is through immersion in real school problems that preservice principals gain meaningful problem solving skills (Browne-Ferrigno & Muth, 2004; Browne-Ferrigno, 2007; Perez, et al., 2011). In their study to analyze changes to a principal preparation internship program, Risen and Tripses (2008) found that expert problem solving was central to strengthening future school leaders.

The purpose of principal preparation programs is to develop leadership skills and capacities within preservice principals (Dishman & Redish, 2011; LaPointe & Davis, 2006; Orr, 2010). A key component of this preparation is administrative experience that reflects what actually occurs in today’s schools. In California, Preliminary Administrative Services credential programs must meet 15 Standards for accreditation. Standard 7 - Nature of Field Experiences, and Standard 8 -Guidance, Assistance and Feedback, provide criteria for planning and evaluating required program elements. Standards 10-15, closely aligned with the California Professional Educational Leadership Standards (CPSELs), describe the skills, knowledge, and dispositions expected of entry-level administrators.

The fieldwork experiences component designed and facilitated by the Department of Educational Leadership at California State University, Fullerton (CSUF) consists of four semester units spread throughout the five-term program. Instructors guide students in the development of fieldwork plans, which identify specific leadership tasks and responsibilities to be completed under the direction of their mentors, who are experienced administrators, typically principals or assistant principals at their sites. Activities must align with the CPSELs, and during their final semester, students submit a portfolio of artifacts to support their written analysis of their experience and progress toward competency in each of the standards.

In 2008, CSUF Educational Leadership faculty convened a task force to review the department’s fieldwork standards and procedures. Faculty responsible for evaluating administrative competency had expressed concern that the leadership experiences that students plan and complete are often the result of convenience and opportunity, rather than need. Although students collect substantial evidence regarding their fieldwork accomplishments and are able to articulate the alignment with the leadership standards, faculty questioned whether the activities addressed the wide range of skills and expertise demanded of today’s educational leaders.

Instructors concluded that fieldwork may need to be more prescribed and individualized, congruent with the California Professional Standards for Educational Leadership (CPSELs) but also designed to expand the experiential base of each student. Students enter the administrative credential program with varied educational and professional backgrounds. Some are veteran educators, with 20 or more years in the profession while others are neophytes, with as few as three years of experience. Some are department chairs, team leaders, or teachers on special assignment (TOSAs) while others have developed expertise in a specific area of instruction, such as special education. Fieldwork, we concluded, should force students to move out of their comfort zones and into new arenas of experience. Toward that goal, we developed a tool. The Candidate Inventory of Personal Leadership Competence was designed to help students identify their areas of relative strengths and weaknesses, an assessment that could be used to drive their fieldwork plans and to measure growth. The purpose of this study was to evaluate the impact of candidates’ field experience in leadership based on the pre and post implementation of that tool.

This study is based on an analysis of candidates’ pre and post self assessment of their own level of experience in activities aligned with the six CPSELS. The instrument, The Candidate Inventory of Personal Leadership Competence, was designed around the six CPSELs which state that an instructional leader promotes the success of every student by…

Collaboratively, we “unwrapped” the standards and operationalized each one in terms of specific leadership behaviors and experiences. We asked ourselves, “What, specifically, do leaders need to be able to do?” Second, “What experiences and knowledge do our students already have and which do they lack?”

At its inception, we believed that this forced-choice instrument would accomplish three things: (a) familiarize students with the standards and program expectations, (b) identify a starting place for developing meaningful fieldwork targeted to the areas of greatest need, and (c) provide baseline data to measure perceived growth and program effectiveness.

The pre assessment was implemented in Fall 2008, and the first post-assessment took place in Spring 2010 when those initial cohorts of participating students completed the preparation program. Each pencil and paper assessment was administered within the confines of a classroom, directed by the fieldwork course instructor.

The Candidate Inventory of Personal Leadership Competence consists of six pages, one page devoted to each CPSEL with a total of 58 items. For each of the professional educational standards, there are 9 - 11 specific leadership activities or roles for which students are to indicate their current level of preparation based on two dimensions: experience and knowledge. Responses were on a four-point scale, with 4 representing “high” and 1 representing “low.” The activities listed under each standard were collected from various print and online sources. Examples of actual specific leadership activities by CPSEL Standard contained in the Inventory include the following:

These examples represent only 11 of the 58 items contained in The Candidate Inventory of Personal Leadership Competence.

Question 1: What degree of change occurred in students’ pre and post self assessment of their level of experience for each of the CPSELs, and how similar or different were the changes from pre to post self assessment among the six standards?

Table 2 summarizes the pre and post mean responses and computed difference (post test M – pre test M) illustrating the average change in candidates’ self reported level of experience for each CPSEL. As shown below, the mean differences between pre and post assessments ranged from .8 to 1.3 scale points. All differences were significant based on paired sample T-tests (p=.001). The largest gain (1.3) in self reported experience was for CPSEL 6 - influencing the political, social, economic, legal, and cultural context. The smallest gain (.8) was for Standard 2 - culture, instructional program.

Table 2

A Comparison of Pre and Post Self Assessments of EDAD Students over a Two-Year Period According to Difference in Mean Responses (Scale = 4 [high] to 1 [low] with N=82)

Question 2: Did the degree of reported gain/loss in leadership experience vary among individual students?

Mean differences are helpful in making group comparisons, but means can hide individual differences, and we were curious to know whether the reported gains/losses tended to be consistent among all students or varied.

To answer this question, we conducted a matched score comparison and computed the frequency distributions of individual student’s mean response differences for each standard. We asked ourselves, what was the magnitude of this change and was growth evenly or disparately distributed? As shown in Table 2, the range of total group growth was from .8 to 1.3 full scale points. In order to discriminate more precisely in the matched case comparison, we calculated growth in .5 scale score point increments giving us a range of <.5 to ≥2.0 full scale points for analysis. Table 3 displays the findings from this analysis. The first statistical column (<.5) lists the percentage of students whose self reported level of experience grew less than one-half (.5) scale score points; the fifth statistical column (≥2.0) lists the percentage of students whose self reported level of experience was equal to or greater than two scale score points. Because a full scale point gain on the 4-point scale represents a25% gain, we appended an additional column. The last column is summative and displays the total percentage of candidates who reported gains of one full scale score point or more (≥1.0).

Table 3

Matched Score Comparison of Pre and Post Self Assessment of Experience by Scale Score Points

Table 3 displays the six standards from highest to lowest according to the percentage of fieldwork participants indicating growth of at least one scale score point (1.0) between pre and post self-assessment. With this representation, three clusters of standards surfaced relative to growth of at least one scale score point (1.0). More students (70%) reported significant growth in their experiences understanding, responding to, and influencing the political, social, economic, legal, and cultural context of their environments (Standard 6) than any of the other five standards. Approximately six of every 10 participants indicated at least a one scale score point improvement in Standards 5, 3, 1, and 4.

The fewest number of participants (48%) self-assessed their growth as one scale point or more in pre post fieldwork experience in Standard 2 which states that an instructional leader promotes the success of every student by advocating, nurturing, and sustaining a school culture and instructional program conducive to student learning and staff professional growth. This finding is consistent with that shown in Table 2 which displayed total group mean growth.

Question 3: How similar or different were the changes from pre to post assessment according to the total years of professional experience (TYE) of the candidates?

The final question in this analysis focused on the relationship between reported gains/losses in perceived level of experience and TYE (total years experience, see Table 1). To answer this question, we compared average gains according to the following range of TYE, based on self reported data at the time of enrollment in the principal preparation program: 1-4 TYE; 5-9 TYE; or 10 or more TYE. Table 4 displays the average reported gains in experience for each standard.

Table 4

Pre and Post Self Assessment Mean Gains on Six CPSELS by Total Years of Experience

As shown in Table 4 and congruent with earlier comparisons, the least gains for all groups was relative to Standard 2 – culture, instructional program, although those with 10 or more years of experience reported growth approaching one scale points (.97). With 18 points of comparison, three for each of the six standards, only three comparisons differed by more than two-tenths of a scale point. Each of these comparisons show candidates with the greatest number of years of experience (10+) reporting marginally greater growth than did those with fewer years of prior experience. The most experienced candidates reported greater growth than those in either of the two comparison groups, in Standard 3 – organizational management; Standard 4 – collaboration…diverse community needs; and Standard 5 – fairness, integrity…ethics; – and Standard 6 - influencing political, social, economic, legal, and cultural context. A one-way ANOVA indicated no statistical significant differences among the three groups, based on an alpha of <.05.

Comparisons of the pre and post self assessments of the 82 participants in this study support the conclusion that over the course of their administrative program participation, candidates perceived gains in their level of leadership experience. In that respect, our principal preparation program is fulfilling its purpose to provide these ‘administrators to be’ with experiences deemed appropriate for educational leaders in California. The greatest growth based on both mean growth differences and matched score comparisons, was for Standards 6, 5, and 3. Standard 1 - vision of learning - was slightly ahead of Standard 4 - collaboration…diverse community needs - in the matched score comparison (Table 3) but somewhat behind relative to group mean growth as shown in Table 4. Our students appear to be saying, “I am growing in my ability to…

In terms of meaningful differences, however, it was Standard 2 - culture, instructional program - that stood out. In every ranked comparison, Standard 2 was last, the only standard with a total mean difference of less than one scale point (.8) and the only standard with fewer than half (48%) of the candidates reporting growth of at least one scale score point . One explanation for the disparate results for Standard 2 may be found in the standard itself; it is the only standard to focus almost entirely on the instructional program: An instructional leader promotes the success of every student by advocating, nurturing, and sustaining a school culture and instructional program conducive to student learning and staff professional growth. Virtually all of the 82 participants were classroom teachers or had positions (e.g., teachers on special assignment) that focused on instruction. One can reasonably conclude that students enter the principal preparation program with more instructional experience than experience in any of the other five areas. Student learning is central to a teacher’s work and remains central to those who aspire to be instructional leaders in schools.

Implications. Findings from this analysis support the value of a pre and post assessment instrument as a measure of candidate growth. Although dependent on the participants’ ability to self assess, these data provide useful information for program improvement purposes and compiled longitudinally will be increasingly valuable. Our second purpose in developing this tool was to individualize students’ fieldwork experiences based on their relative strengths and weaknesses. In order to come closer to that goal, future program enhancements might include the development of an interim assessment instrument to measure candidate progress through the program; analyzing candidates’ growth in experience somewhere in the middle of the program may provide us with more direction in counseling our students to become more experienced in other standards. We may also want to evaluate the experiences our candidates report when working with their mentors. We need to ensure that our candidates are provided opportunities based on need and not convenience.

A next step is to test the validity of the Inventory by convening an expert group of practicing school leaders to review the items and make recommendations relative to the importance of their inclusion/exclusion and possible addition of other activities.

Limitations. We acknowledge three limitations to this study: a) Accuracy of the findings are dependent on the validity and reliability of the instrument, the Candidate Inventory; b) As a self assessment, the data reflect the subjective judgment of each candidate; and c) It is not possible to separate the actual impact of candidates’ fieldwork experience from the influence of their participation in other components of the administrative preparation program or from professional experiences unrelated to their credential preparation experiences.