The Academy, through the use of research and best practices, designs creative and innovative approaches to establish learning communities in public schools by developing human resources and by modeling and advocating collaboration, support, shared decision making, team learning, risk taking, and problem solving. Partners commit to changing their organizations to support system improvement. (About Us section, para. 2)

The situations professional nurses find themselves usually are so complex that the notion of routine practice should be precluded. To engage in competent nursing practice, nurses need to consider the contextual variables in each interaction, mindful that each interpersonal encounter is to some extent unique and that there is usually more than one desired outcome.… They might need to question the routine, explore alternatives and transform previous ways of understanding. (p. 30)

Knowing-in-action is evident when a professional practitioner uses knowledge that they have accrued from prior experience and applies it to a current situation (Schön, 1987). For many new principals, the knowledge discharged in knowing-in-action might often be based on the theoretical knowledge that has been learned through professional socialization sources or formal pre-service training experiences. For example, school principals typically go to a university educational leadership program to learn the theories of educational leadership and how to apply it to a practice situation. With this knowledge, there are preconceived assumptions that are made as to how approach a particular problem (Mezirow, 1997). While making decisions and solving problems using prior knowledge and theoretical frameworks may assist in the success of the daily routines of being a principal, seldom does it lead to significant transformative change in the performance of the individual or the organization (Mezirow, 1997).

Reflection-in-action, on the other hand, addresses the learning of knowledge based on an individual learner’s experience in the profession. As the individual’s experience is ongoing for the duration of time they are in the profession, reflection-in-action also is continuous and is seen as a cornerstone to producing new knowledge that assists the practitioner in their everyday work. Schӧn (1987) explained:

Reflection-in-action leads to learning that addresses the contextual aspects of the work. Ongoing reflective practice and peer learning may or may not use prior knowledge to anchor thoughts, generate new ideas, or solve problems. It is the influence of a particular situation, organization, or culture that leads to the questioning of the assumptions that are present in broad-stroke theoretical concepts often found in knowing-in-action (Schön, 1987).

Professional knowledge should be acquired from both knowing-in-action and reflection-in-action (Cervero, 1988). Having a base of theoretical knowledge allows professional practitioners to have a platform to continue to grow in their learning as well as their practice through experimenting and questioning the status quo. It could also be argued that the contextual nature of education and the often spontaneous and unique human interaction experienced by school leaders requires reflection-in-action and collective inquiry to play significant roles in the continuing development of principals – development that is essential in pursuing changing organizational culture, practice, and performance through communities of practice or professional learning communities (DuFour, DuFour, & Eaker, 2009).

There are a diverse set of definitions of reflection ranging from simply thinking about something to engaging in a more crafted and strategic practice of looking at a particular problem or phenomenon. Reflection has been defined as a thinking process that involves a deliberate pause accompanied by an open perspective that examines beliefs, goals, and practices to gain new insights and understanding (York-Barr, Sommers, Ghere, & Montie, 2001). Loughran (2002) offered, “it is important that the nature of reflection be identified in such a way as to offer ways of questioning taken-for-granted assumptions and encouraging one to see his or her practice through others’ eyes” (Loughran, 2002, p. 33). As a strategy for professional adult learning, Schön (1987) identified reflective practice as being crucial in overcoming the “technical rationality” (p. 3) that pervades the learning in higher education institutions.

Whatever the definition used, it is perhaps most important to recognize reflective practice as being an integral part of the “conceptual characteristic” of the professionalizing process as proposed by Houle (1980). According to Houle:

Through reflective practice, school principals develop a conceptual understanding of what their job should be as opposed to an assumed role that is molded by public opinion or historical perceptions. Never has this been more evident than in today’s world of high-stakes accountability where principals are asked to take on the role of an instructional leader while continuing to maintain the managerial responsibilities of running a school.

Although the research on reflective practice in school leadership is sparse, Day’s (2000) study of twelve highly effective principals found that there were five different kinds of reflection:

Day’s five types of reflection suggest that reflection can be used for different purposes ranging from personal development to developing a school-wide vision.

York-Barr et al. (2001) proposed four different experiential contexts that reflection can occur in schools. These include individual reflection, reflection with a partner, reflection in small groups or teams, and school-wide reflection. It can be feasible to match these various contexts within which reflection occurs to the multiple types of reflection proposed by Day (2000). For example, in holistic reflective practice, a school-wide approach might be suitable; in pedagogical and interpersonal reflection practices, reflection with a partner might be most suitable; in intrapersonal reflection, individual reflection would be most appropriate, and so forth.

Drago-Severson (2009) identified the following collegial inquiry practices seen by principals as highly effective and meaningful to their development: reflection through the process of writing, reflection through dialogue, reflection through decision making, and reflection through helping and advising others. It would be feasible to match Drago-Severson’s collegial inquiry practices to the types of reflection identified by Day (2000) and contexts of reflection identified by York-Barr et al. (2001) as portrayed in Figure 2.

Figure 2. Identifying the similarities between various types of reflection as presented by York-Barr (2001), Day (2000), and Drago-Severson (2009).

Peer learning, a bi-product of effective collegial inquiry or “collaborative reflective practice” (Drago-Severson, 2009, p. 154), involves individuals “learning from and with each other in both formal and informal ways” (Anderson & Boud, 1996, para. 6). According to Boud (1999), “Peer learning involves a group of people taking collective responsibility for identifying their own learning needs and planning how these might be addressed” (p. 4). The above definitions of peer learning offer such terms as collaborative, collective responsibility, learning from and with each other. All of these support the premise of self-directed learning and social interactions set forth in adult learning theory. Adult learning is enhanced through social interactions between individuals that focus on a common problem, and self-directed learning is one of the pillars of adult learning theory (Knowles, 1992). When adults are given the opportunity to work collaboratively with little intervention from facilitators, self-directed learning occurs (Mezirow, 1997). Participants must value collaboration and embrace the opportunity to share perspectives in a safe environment where it is acceptable to make mistakes and to seek assistance and clarification when needed (Anderson & Boud, 1996; Boud, 1999). In addition, the learning environment must encourage and nurture reflective dialogue (York-Barr et al., 2001; Drago-dSeverson, 2009).

One form or structure that supports peer learning is the community of practice. According to Hansman (2001), “Communities of practice are self-organized and selected groups of people who share a common sense of purpose and a desire to learn and know what each other knows” (p. 48). Communities of practice provide an environment where competence and experience combine to allow for the acquisition and the creation of knowledge (Wenger, 1998). To further distinguish communities of practice, Wenger (2009) suggested, “Communities of practice are groups of people who share a concern or a passion for something they do and learn how to do it better as they interact regularly” (p. 1).

A distinction between communities of practice and professional learning communities should be made as groups of educators who meet to discuss issues about instruction and learning, continue to learn about their craft, and share ideas with others are often referred to as professional learning communities. We see professional learning communities as being a type of community of practice with the fundamental difference being the communities of practice acquire and generate new knowledge as part of a general social learning theory while learning communities tend to concentrate on only acquiring new knowledge (Wenger, 1998). Professional learning communities can be seen as a form of professional development implying the acquisition of knowledge, while communities of practice can be seen as both professional development and professional transformation as knowledge is both acquired and created.

The Arkansas Leadership Academy is a community of practice that incorporates both reflective practice and peer learning networks as significant components to its Master Principal Program (MPP). A major goal of the MPP is to develop school leaders who are capable of implementing and managing meaningful change within their schools and school systems.

Supporting the MPP’s community of practice, peer learning networks are in place to accommodate the knowledge-building between participating principals. While the phrase peer learning network is most often associated with technology-driven online learning opportunities, the ALA defines peer learning networks as any opportunity for the participants in the MPP to get together, virtually or physically, to enhance their own learning through the interaction with others.

How do the written accounts of the ALA MPP participants represent their reflective thoughts about their professional work as a principal?

The research team was comprised of four members, three of whom had strong associations with the ALA. One of those three was directly involved with the Master Principal Program. Because of the involvement of the three ALA-affiliated research team members, it was important to recognize that there might be bias to the success of the ALA MPP program. For this reason, this study was not evaluative in nature. Instead, the study was designed to examine the reflective process of the program participants and to determine if there was a relationship between reflective practice, involvement in peer learning support networks and performance on the portfolio assessments.

The team was aware of the potential for researcher bias and took measures to protect the research process as discussed in the methods section; however, it was also recognized that there was value in the subjectivities of the researchers as they had an intimate understanding of the Master Principal Program (Maxwell, 2012). Therefore, the research team did not identify their potential bias as a threat to the study as long as measures were taken to acknowledge their own subjectivities and to approach the study with a sound methodology.

Study participants were 59 principals from throughout the state of Arkansas who applied for participation in MPP Phases 2 and 3 from 2005 through 2010. Principals with portfolio scores for Phase 2 and subsequent Phase 3 applications were included in the analysis to allow for repeated measures analysis. Of the 59 participants, 85% were female and 93% were Caucasian. Urban, suburban and rural schools were represented. Out of the 59 participants in the quantitative study, there were 14 portfolios chosen using random purposeful sampling as the total number and length of narrative texts available was too large for review for in-depth qualitative analysis (Creswell, 2013).

The 59 principals in this study were the population of principals that completed the MPP through Phase 3 application between 2005 and 2010 and whose Phase 2 and Phase 3 records could be linked for analysis. These participants constitute 91% of the 65 MPP Phase 3 graduates. Given the eligibility requirements for participation in MPP, it follows that Phase 3 principals would have a minimum of 4 years of experience as a principal. However, there are five principals with less than 4 years. These principals may have been admitted to Phase 1 as new principals or assistant principals on a case-by-case basis. The years of experience of the participating principals can be found in Table 1.

Table 1

Years of Experience as Principal

The MPP portfolio is required to be submitted by applicants desiring to participate in MPP Phases 2 and 3. Phase 2 and 3 portfolios are scored using the MPP rubric. The MPP rubric provides detailed indicators which delineate the knowledge, skills and abilities of principals along a 5-point continuum of effectiveness. The MPP rubric was designed to differentiate applicants’ readiness for further development as a Master Principal. Applicants are scored on the 5-point scale for each of six performance areas resulting in a Total Point Score (TPS) out of 60 possible points. Portfolios are scored by ten independent scorers (i.e., two scorers for each performance area). The team of scorers were selected from the following groups:

Team members received rubrics and scoring guidelines two weeks prior to the scoring activity. At the beginning of the two-day scoring session, there was a guided practice session which led scorers through a sample portfolio assessment. Through this process,inter-rater reliability was established prior to independent scoring of the portfolios.

While participants in the MPP submitted school performance data as part of their portfolio, the narratives were written describing their school, the relationship of their school and community, their own practice as principal related to the six performance areas of the rubric, and the overall progress of their school.

Two repeated measures analyses of ALA MPP principals’ portfolio scores were conducted to test the aforementioned hypotheses:

Repeated measures designs provide more power for detecting meaningful differences because idiosyncratic differences are removed from the error term (Stevens, 2002). The increased power of repeated measures is particularly important in cases where small sample sizes are unavoidable.

For the two-way repeated measures ANOVA principals were grouped into three categories based on the number of additional ALA Institutes that principals participated in prior to or during participation in Phase 2 of the MPP. These ALA institutes focused on developing various school improvement related knowledge and skills through the use of the RPPLS learning approach. The categories were classified using the following frequencies.

Following the quantitative analysis, a qualitative analysis was conducted of Phase 2 and Phase 3 applicants’ narratives provided in their portfolios. An informal approach to the analysis of the written narratives was used. Perӓkylӓ (2008) explained:

Perӓkylӓ (2008) also suggests that an informal approach to analyzing text is most often seen as an acceptable way to process narrative writing when the textual analysis is seen as a complementary or supporting component of the larger research design. The goal of the analysis was to discover any change in the written reflections of the program participants with the intent to determine if the level of reflection by the participants changed between the writing of the Phase II portfolio narrative and the writing of the Phase III portfolio narrative.

The portfolio exercise was designed, in part, as a reflective writing exercise where MP candidates were asked to reflect on their experiences as a school leader in relation to the principles and standards of the MP program. Given that the data was pre-existing, and not collected specifically with the intent of uncovering the reflective practices of the participants, it can be determined that more work might be done to gain a better understanding of the reflective process experienced by participants in the ALA MPP.

Phase II and Phase III portfolios produced between 2008 and 2010 were read. A post-positivist approach (Creswell, 2013) to the analysis initially involved a deductive analytical approach using the theories presented in Figure 2. The first cycle of descriptive coding (Saldaña, 2009) led to the realization that emerging themes could be categorized using van Manen’s (1995) temporal dimensions (see Table 2). Therefore, a second cycle of deductive coding was used to further reduce the data and further define emerging themes.

Table 2

Temporal Dimensions of Reflection

The data analysis approach was conducted by a one member of the research team. Once the coding was done and themes identified, the analysis was vetted to the other members of the team through peer debriefing to establish credibility to the analysis (Marshall & Rossman, 2011). This approach was used to reduce researcher bias during the qualitative analysis as the researcher who actually conducted the initial analysis was not associated with the ALA.

While retrospective and anticipatory dimensions are the most common, contemporaneous reflection suggests the most powerful type of reflection (Schӧn, 1983; van Manen, 1995). It is through contemporaneous reflection or reflection-in-action that practitioners surpass the simple application of theory or what Schӧn would define as technical rationality. Contemporaneous reflection seizes the moment and allows for a deepening of awareness that is driven by the context of the present situation the practitioner finds themselves. Raelin (2001) explains:

It is the contemporaneous reflection that is the most difficult to identify, particularly when using existing data from narratives that were written for purposes other than current study. However, by pursuing the identification of contemporaneous reflection in the narratives, the findings illuminated the thought that while reflection-in-action was difficult to determine, the shift to more holistic reflection was evident.

Results of the dependent samples t-test indicated principals applying to Phase 3 of the MPP scored significantly higher on their Phase 3 portfolios than their Phase 2 portfolios (t = 8.12, p < 0.0001). Principals scored 37.14 (σ = 5.46) points on average for Phase 3 application portfolios compared to 30.20 (σ = 6.12) for Phase 2 portfolios. The effect represents a large or meaningful difference (d = 1.06) in principals’ knowledge and skills from the beginning of Phase 2 to the beginning of Phase 3 MPP.

The two-way repeated measures ANOVA with portfolio scores as the repeated measure resulted in a significant interaction between principals’ frequency of participation in RPPLS and the change in portfolio scores from Phase 2 to Phase 3 applications (F_(2,56) = 4.22, p = 0.02). Table 3 provides the summary table for the two-way repeated measures ANOVA.

Table 3

Summary Table for the Two-Way Repeated Measures ANOVA

Figure 3 illustrates the interaction between the number of RPPLS Institutes attended by principals and the change in portfolio scores from Phase 2 to Phase 3 applications. Principals involved in Category 3, 5 or more RPPLS Institutes by the completion of Phase 2 MPP, achieved the highest mean portfolio scores; whereas principals attending fewer than 5 ALA Institutes demonstrated lower average performance by the end of Phase 2 MPP.

Figure 3. Mean rubrics scores of participants’ portfolio assessment in each of the three categories for Phase 2 and Phase 3.

The voices of the participants suggested that Master Principals had developed into more holistic reflective practitioners as they moved from Phase II to Phase III of the MPP. The narratives of the Phase II portfolios presented rich data, the reflections included almost exclusively retrospective and anticipatory reflective thought. There were more instances of broad realizations of their professional experience as a principal displayed in the Phase III portfolio reflections.

Many of the participants’ reflections in the Phase II portfolios could be described as being reports of actions taken in the past (i.e, retrospective reflection). For example, one elementary principal wrote:

This “look into the past” is a form of retrospective reflection that might help this particular principal identify where her school was at one point in time. By doing so, she may shed light on how much the organization has changed its meeting practices, but there is little to suggest how her practice as principal has changed.

Anticipatory reflection was also evident in the Phase II portfolio narratives. This same participant offered, “Once new vision is established, we will work to align our policies and procedures to our core beliefs.” Here, the principal gives some ideas of what she is envisioning is going to happen in the future related to extending the school’s vision. One participant anticipated the need to continue working on instructional issues:

Yet another Phase II participant exhibited anticipatory reflection about instructional matters through the statement:

The anticipatory reflections of the Phase II participants represent planning based on current or prior experience and projecting what lies in store in the future (Raelin, 2001, van Manen, 1995).

In the Phase III portfolio narrative, these same principals revealed anticipatory and retrospective reflections – much like the reflections in their Phase II portfolios. However, there were instances of a more holistic reflection. These reflective statements might not necessarily be identified as contemporaneous reflections; however, they do have characteristics of intrapersonal and holistic reflection as described by Day (2000).

In the Phase III portfolio, this one participant wrote about her school’s growth in sharing a vision and mission:

The phrase “I have realized…” implies an intrapersonal and holistic reflective thought – one of realization not only of the importance of a culture that has a cohesive belief system, but also one of the realization of progress being made. Another participant stated that she had come to the realization that “We [the faculty] had turned into a family…” while another expressed, “we are never satisfied with the status quo.” The statement, “I realize now that I have been an agent of change!” illuminates an intrapersonal reflective thought (Day, 2000). The data from all analyzed portfolios generally followed the trend found in Figure 4.

Figure 4. The trend of participants’ reflective thoughts from retrospective and anticipatory reflection to a more robust and reflective process with the addition of holistic and intrapersonal reflective thought.