Skip to content Skip to navigation

OpenStax_CNX

You are here: Home » Content » Standards-based Assessment for Principal Interns

Navigation

Lenses

What is a lens?

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

This content is ...

Endorsed by Endorsed (What does "Endorsed by" mean?)

This content has been endorsed by the organizations listed. Click each link for a list of all content endorsed by the organization.
  • NCPEA

    This module is included inLens: National Council of Professors of Educational Administration
    By: National Council of Professors of Educational Administration

    Click the "NCPEA" link to see all content they endorse.

Recently Viewed

This feature requires Javascript to be enabled.
 

Standards-based Assessment for Principal Interns

Module by: Glenn Koonce, Ralph Causey. E-mail the authors

Summary: The Framework for School Leaders, an architecture derived from the Interstate School Leaders Licensure Consortium (ISLLC) Standards, is utilized in the design of the Principal Internship Mentor’s Assessment (PIMA). PIMA outcomes are reported for average scores for each standard and investigated as a measure of ISLLC Standards achievement and for predictive scores on the School Leaders Licensure Assessment (SLLA). Findings include consistency of variance within the PIMA and its constituent items. No positive relationships were found between SLLA scores and individual PIMA items. In addition, findings indicate reliability for the instrument.

The International Journal of Educational Leadership Preparation, Volume 6, Number 1, January - March, 2011, ISSN 2155-9635

ncpealogo.gif

Note:

This manuscript has been peer-reviewed, accepted, and endorsed by the National Council of Professors of Educational Administration (NCPEA) as a significant contribution to the scholarship and practice of education administration. In addition to publication in the Connexions Content Commons, this module is published in the International Journal of Educational Leadership Preparation, Volume 6, Number 1 (January - March, 2011), ISSN 2155-9635. Formatted and edited in Connexions by Theodore Creighton and Brad Bizzell, Virginia Tech and Janet Tareilo, Stephen F. Austin State University.

Sumario en espanol

La Armazón para Educa a Líderes, una arquitectura derivada de la Escuela Interestatal Líderes Licensure Consorcio (ISLLC) Estándares, son utilizados en el diseño de Evaluación del Principal Puestos de interno Mentor (PIMA). Los resultados de PIMA son informados para cuentas medias para cada estándar e investigados como una medida de logro de Estándares de ISLLC y para cuentas predictivas en la Escuela Líderes Licensure Evaluación (SLLA). Las conclusiones incluyen consistencia de variación dentro del PIMA y sus artículos constituyentes. Ningunas relaciones positivas fueron encontradas entre cuentas de SLLA y artículos individuales de PIMA. Además, las conclusiones indican la certeza para el instrumento.

Note:

Esta es una traducción por computadora de la página web original. Se suministra como información general y no debe considerarse completa ni exacta.

Introduction

“The internship is considered by many practicing principals to be the most valuable component of their preparation program, as schools provide the laboratories where the connection between educational leadership theory with practice and application can best be made” (Bost, 2009, p. 15). In the quest to build and maintain high quality educational leadership preparation “all programs struggle with ways to provide robust internships” (Darling-Hammond, Myerson, LaPoint, & Orr, 2010, p. 74). Finding time for assessing the intern is among the greater issues facing busy mentoring principals in the field. Few have time for an elaborate evaluation process. Despite this reality, university internship supervisors seek evidence that the intern has met sufficient “performance of core school leaders responsibilities based on clearly defined performance standards” (Fry, Bottom, & O’Neill, 2005, p.7).

The instrument used for assessing interns becomes a critical component of the principal preparation program and requires evidence that it is linked to clearly defined performance standards. The Interstate School Leaders Licensure Consortium (ISLLC) Standards are recommended for performance standards in school leadership preparation programs (Teacher Education Accreditation Council, 2005). ISLLC standards are utilized for items on this study’s internship assessment instrument.

Literature Review

The ISLLC Standards “effectively reflect the wide variety of responsibilities associated with the principalship” (Bost, 2009, p. 5). “There is no consistent set of program strategies in place across the country” (Darling-Hammond, Myerson, LaPointe, & Orr, 2010, p. 159). By utilizing the ISLLC framework for assessing program outcomes, states are “beginning to create more systematic approaches” (p. 159). “ISLLC standards have been used for guidelines in principal preparation programs and they have sharpened the focus of principal training considerably” (p. 150). For this reason, the ISLLC standards were selected for the Principal Internship Mentor’s Assessment (PIMA) examined in this study.

The Regent University Education Leadership Program adopted the six ISLLC standards for learning outcomes. These standards were also selected by Program faculty for national accreditation with the Teacher Education Accreditation Council (TEAC). The Program was accredited on January 9, 2009. TEAC is currently one of two national accreditors currently recognized by the U.S. Department of Education (USDE) and by the Council of Higher Education Accreditation (CHEA) (NCATE/TEAC, 2010). The other national accreditor is the National Council for Accreditation of Teacher Education (NCATE). NCATE adheres to the Educational Leadership Constitutes Council (ELCC) standards for programs in educational leadership, ISLLC and ELCC standards are identical except that ELCC standards have an additional seventh standard; 7.0 internship (National Policy Board for Educational Leadership, 2002).

Standard 7.0 Narrative Explanation: This standard addresses the importance of structured, sustained, standards-based experiences in authentic settings. The internship is defined as the process and product that results from applying the knowledge and skills described in the previous standards in a workplace environment. Application of standards based knowledge, skills, and research in real settings over time is a critical aspect of any institutional program. The provision of graduate credit allows institutions to underscore the importance of this activity (p. 18).

States vary greatly in their program internship requirements. For example, “92 percent of New York’s principals had had an internship as part of their program preparation, whereas only 8 percent of Delaware’s principals had” (Darling-Hammond, Myerson, LaPointe, and Orr, 2010, p. 152). Internships across the country “range from a certain number of acquired credit hours that can be satisfied at least partially at a candidate’s home school or during school breaks, to year long, paid assignments working side by side with a mentoring principal” (Martin, Wright, Danzig, Flanary, & Brown, 2005, p. 134). It is likely there are as many internship assessments as there are different leadership preparation programs in the nation. Since no evidence was found in the literature for a common principal internship assessment, there are likely a large number of different instruments.

There are examples of assessments for principal internships in the literature. Dallas Baptist University (2004) employs an instrument titled “Supervising Administrator Evaluation Form” (p. 14). This assessment is brief, employs the ISLLC standards and a Likert-type scale. It is very similar, although not as extensive an instrument as the one being addressed in this study. In Dallas’s instrument there is only one indicator for each of the six ISLLC standards. An instrument using ISLLC standards can be found in School Leadership Internship (Martin, Wright, Danzig, Flanary, & Brown, 2005), but this assessment focuses only on dispositions.

Another instrument, MSEd (Principal/Building Level) Practicum Internship Assessment, from the University of Nebraska at Kearney (2003) is a one-page checklist of 13 questions and one open-ended question. It does not appear to be aligned with ISLLC nor any other standards. Florida Atlantic University (2009) uses a Principal Internship Activity Log, with a box for standards on the form, where students fill in the log and the mentor initials and can make comments. Other principal intern assessments reviewed were more comprehensive and included: reflective journals and journey mapping (Cooner & Dickmann, 2006); proficiencies exhibited and explained in the Applied Principal's Portfolio (Indiana Wesleyan University, 2010); work log and mentor’s assessment (Cunningham, 2005).

A Framework Derived from the ISLLC Standards

This study makes application of “The Framework for School Leaders”, an architectural design derived from the ISLLC Standards (Hessel & Holloway, 2002, p. 26). The Framework translates theory into practice and defines the elements of appropriate practice for school leaders (Hessel & Holloway, 2002). The Framework presents each of the six standards as a set of “four components of professional practice for school leaders, or specific aspects of the Standards” (p. 26). The Principal Internship Mentor’s Assessment (PIMA) is the result of synthesizing the framework into an instrument used to assess interns by mentoring principals in the field (Appendix A). The goal for the university program in this study was the design of an assessment that was researched-based, linked to the ISLLC standards, and practical, in terms of time, for mentoring principals completing the assessment.

A rubric (see Appendix B), designed to identify performance levels and specific activities for scoring each of the four components in each ISLLC standard, provides a common set of parameters for principals completing the assessment. The PIMA contains five performance levels displayed in a Likert-type scale within the assessment.

The PIMA is utilized as a summative evaluation for the internship. “These performance indicators are used by the university instructor to designate the intern’s level of performance on each of the components of the dimensions of school administrative factors” (Cunningham, 2007, 22). The outcomes provide evidence required for a grade in the principal internship course and data for accreditation evidence and program improvement.

The School Leaders Licensure Assessment

The School Leaders Licensure Assessment (SLLA) “measures whether entry-level education leaders have the standards-relevant knowledge believed necessary for competent professional practice” (Education Testing Services, 2010, p. 1). The content of the SLLA is “keyed to the ISLLC standards and the knowledge, skills, dispositions, and performances they reflect” (Jones & Kennedy, 2008, p. 5). This study examines scores prior to the new revised assessment that went into effect September 2009. The format for the six-hour assessment utilized in this study is divided into four sections:

  • Evaluation of Actions I (1 hour) - Ten short vignettes covering situations a principal might encounter. Candidates respond to a focused question that asks for next steps, factors influencing a decision, or possible consequences of an action.
  • Evaluation of Actions II (1 hour) - Six longer vignettes. Each presents a dilemma based on typical school issues. Candidates analyze the circumstances and respond to a focused analytical question that requires prioritizing action steps or articulating the relevant issues.
  • Synthesis of Information and Problem-Solving (2 hours) - Two case studies involving teaching and learning issues. In each case, candidates propose a course of action to address a complex problem, referring to a set of documents, and a short scenario describing a school and its community.
  • Analysis of Information and Decision Making (2 hours) - Seven documents that relate to teaching and learning issues. Candidates answer two questions about each document (Virginia Tidewater Consortium For Higher Education, 2005, pp. 12 & 13).

Situations presented on the SLLA are designed to be real-life in the day-to-day activities of a principal. The examinee responds to each question and is evaluated with a scoring rubric grounded in the ISLLC standards.

Many states use the SLLA for licensure or endorsement in educational administration and utilize a cut score for candidates to attain before a license/endorsement/certificate is awarded. Many university educational administration preparation programs use candidate scores on the SLLA as a measure of program success.

Statement of Purpose

The purpose of this study is to examine the utility of PIMA as a measure of ISLLC Standards achievement, and as a predictor of SLLA scores. The ultimate outcome is improving principal preparation programs as required by accrediting agencies and state program approval regulations.

Research Question

  1. Can performance on the PIMA predict success as measured by the SLLA?

Methods

Participants

Fifty-nine Educational Leadership program completers were studied. The participants were uncompensated and were not interviewed, tested or surveyed beyond the normal program requirements. All were licensed and experienced educators prior to program commencement. The participant group consisted of all personnel that completed the leadership program including internship and who also took the SLLA between September 2005 and August 2009. The final data draw for this study was August 2009 because a new SLLA began in September 2009.

Apparatus

The PIMA is a 24-item Likert-type scale instrument derived from the ISLLC standards. There are four items per standard with each item being rated on a 5-point Likert-type scale (see Figure 1).

Figure 1. Performance Levels of the PIMA Likert Scale

figure1.png

The four items per standard are taken from “Components of Professional Practice for School Leaders” (Hessel & Holloway, 2002, p.27) (see Figure 2). The rating form reflects the 24 Components of Professional Practice for School Leaders derived directly from the ISLLC standards (Hessel &Holloway, 2002, p. 27). Each item was directly linked to an ISLLC standard. The items and the instrument were reviewed for content validity by the program faculty. The form was piloted in the 2006-2007 academic year. The Educational Focus Group (Cannizzaro, 2007) provided feedback on the form and confirmed its content validity since participants were practicing experts in the field. The Pilot Study consisted of interns that completed an internship and a recorded score for the SLLA (N = 4). The investigators reasoned that the small sample and low variability among scores hampered statistical inference. The N (N=4) is small because the University did not require an internship before May 2007. To address inter-rater reliability, sets of two raters used the form and discussed the outcomes in the Focus Group and the four teams of practitioners rated the PIMA similarly (Cannizzaro, 2007). Subjective scoring (Inter-rater-reliability/consistency between tests) is helped when usable guidelines for scoring are developed such as the scoring rubric for the PIMA.

Procedure

Collection of data included principals’ completion of the PIMA each semester for each student completing an internship. The final date for collection of data was August 2009 because the new SLLA format began in September 2009. These data was reported in an ongoing recorded history of program internship data. PIMA scores were extracted from this spreadsheet. SLLA scores were extracted from the university data base. A Pearson’s correlation was performed for average scores for each ISLLC Standard. Another correlation was performed using each individual item and SLLA scores.

Figure 2. Components of Professional Practice for School Leaders

figure2.png

Results

Overall, the implementation of PIMA was consistent with the program’s standards. Internship mentors are defined as principals practicing in the field during the time of the internship. The numbers of students who completed internships (N = 78) is relatively small since internships were not required until August, 2007 (Virginia Department of Education, 2007). Additionally, SLLA scores were not available for some individuals bringing the number of students available for study down to 59 participants. Aggregate data indicate that all average scores exceeded the 75% benchmark (3.00) set by the program faculty. The faculty reset the benchmark to 80% in 2008. Results are reported in Table 1.

Table 1: Summary of Average Scores for Each ISLLC Standard 2005-2010
               
  1a 1b 1c 1d      
Mean 3.68 3.66 3.64 3.69 Total Average of Standard: 3.66  
S.D. 0.54 0.49 0.51 0.49 Total Average of Deviation: 0.54  
               
  2a 2b 2c 2d      
Mean 3.81 3.67 3.70 3.68 Total Average of Standard: 3.71  
S.D. 0.43 0.54 0.49 0.50 Total Average of Deviation: 0.49  
               
  3a 3b 3c 3d      
Mean 3.71 3.69 3.60 3.75 Total Average of Standard: 3.66  
S.D. 0.52 0.50 0.53 0.51 Total Average of Deviation: 0.51  
               
  4a 4b 4c 4d      
Mean 3.76 3.67 3.64 3.76 Total Average of Standard: 3.71  
S.D. 0.42 0.47 0.65 0.44 Total Average of Deviation: 0.49  
               
  5a 5b 5c 5d      
Mean 3.81 3.75 3.81 3.83 Total Average of Standard: 3.80  
S.D. 0.39 0.60 0.47 0.42 Total Average of Deviation: 0.59  
               
  6a 6b 6c 6d      
Mean 3.67 3.59 3.54 3.53 Total Average of Standard: 3.58  
S.D. 0.66 0.57 0.65 0.58 Total Average of Deviation: 0.59  

Note:

100% of candidates meet or exceed score of 3.00; 3.2% after 2008.

Table 1 reports the average scores from the mentoring principal’s assessment for each of the four items under ISLLC Standards 1-6. All average scores exceeded the 75 % (3.00); 80% (3.20) after 2008 standard set by the program faculty.

The investigators performed a correlation analysis on mean scores for each of the six standards, the overall PIMA and scores on the SLLA. There were strong to very strong, statistically significant relationships between the means for each of the 6 standards and the PIMA overall (see Table 2).

Table 2: Pearson Correlations among ISLLC Standard sections, the PIMA and SLLA Scores
                 
  Standard 2 Standard 3 Standard 4 Standard 5 Standard 6 PIMA SLLA  
                 
Standard 1 .666** .651** .704** .640** .813** .856** .072  
Standard 2   .721** .801** .769** .663** .876** -.002  
Standard 3     .727** .703** .673** .845** .065  
Standard 4       .758** .777** .904** .001  
Standard 5         .722** .857** -.053  
Standard 6           .892** -.004  
PIMA             -.029  

Note:

A second correlation study was performed using SLLA Scores and the individual responses on the PIMA. As with the correlation noted for the means by standard, moderate to strong relationships were found between individual items.

For each standard the constituent items correlated robustly with the standard overall. Pearson’s r values ranged between .779 and .980. All findings fell within the .01 level of significance. It was also noted that individual items correlated significantly with the other items grouped within the same standard.

Discussion of Results

The average scores for each standard ranged from 3.58 to 3.80 on a scale from 0.00 to 4.00. Standard deviations for the standards fell between 0.49 to 0.59. The strong to very strong relationships evidenced by the Pearson’s correlation analysis indicate that the six mean standard scores as measured by the PIMA are consistent in variation. This is also true of the individual items with each standard.

The precise relationship between the ratings on the PIMA and SLLA scores is of primary interest to the researchers. Although a relationship seems likely, this initial analysis lacked the statistical power to support the research hypothesis. The apparent internal consistency and significant relationship between PIMA individual items and standards suggests that further study is warranted. The researchers recommend the use of the PIMA in a longitudinal study to further describe the relationship between PIMA ratings and SLLA Scores.

One limitation of the study is that interns completed a formal leadership preparation program designed primarily with the ISLLC Standards as the foundation and external validity may not be extended beyond interns is this particular program. There may be some generalization to other programs that use the ISLLC as the basis for their performance standards. Because the findings are limited in scope, the researchers recommend additional study using the ISLLC Standards to examine if similar or more favorable results can be obtained.

This study is important in the use of internships and particularly standards-based measures used in internship assessment. Literature suggests that although not universal, internships are beneficial and should be robust. This includes the assessment of the intern by mentoring principals. The contribution of this study is to assist practitioners, both principals and university supervisors in the field, to better assess interns. It is recommended that the program continue to utilize the PIMA and compare the findings to the new SLLA outcomes that, as of fall 2009, report scores on each individual ISLLC Standard. Although adjustment of the PIMA is justified, the researchers suggest the PIMA construct also be used as the basis of further research. This will enhance validity of the instrument and provide outcomes that can be applied to specific program improvement.

Conclusions

Since the researchers are interested in validity and reliability of the instrument, it is recommended that the PIMA be reviewed for redesign. A change in Likert-type scale would eliminate Zero and Not Applicable and would change to a true Likert scale, using the common 1 to 5 intervals. Additionally, the wording of the items may be altered to reflect the mentor’s perception of the intern’s ability to do a task rather than assessment of actual task performance. The wording of PIMA items would be changed to reflect how candidates would make decisions as future building principals. This is aligned with the measures found in the School Leaders Licensure Assessment (SLLA) where candidates simulate decision making.

Recommendations for further study include: when feasible, have enough numbers to do a confirmatory factor analysis; a comparison of PIMA data triangulated with a university supervisor assessment, and a self-assessment utilizing the same assessment items; comparisons of results in this program with results in another university preparation program; and, once the PIMA is revised, compare results with half the sample using the original PIMA and half using the new version of the PIMA (see Figure 3).

A final recommendation is to repeat the study utilizing the revised Principal Internship Mentor’s Assessment and the new SLLA format that went into effect on September 2009. The new format is shorter (4 hours as compared to 6 hours in the previous format). The new format also has been reduced to two sections.

Section I contains 100 multiple-choice questions and Section II, seven constructed- response questions that call for written answers based on scenarios and sets of documents that an education leader may encounter. In answering the questions, candidates are required to analyze situations and data, propose appropriate courses of action, and provide rationales for their proposals. (Education Testing Services, 2010, p. 1)

Figure 3. Principal Internship Mentor’s Assessment Revised for Standard 1

figure3.png

The first section is completed by the candidate on-line and the seven constructed response questions are completed by the candidate at a testing center. The examinee responds to each multiple choice question (vignettes) and each constructed response question (case-studies) and is “evaluated with a scoring rubric grounded in ISLLC standards” (p. 5).

Click Here for Appendix A: Principal Internship Mentor’s Assessment

Click Here for Appendix B: Principal Internship Mentor’s Assessment (PIMA) Rating Rubric

References

Bost, D. N. (2009). An Examination of the Effect of State Level Policy in Changing Professional Preparation: A Case Study of Virginia Principal Preparation Programs and Regulatory Implementation. Unpublished doctoral dissertation, Virginia Commonwealth University.

Cannizzaro, S. V. (2007). Executive Summary: Focus Group of Practitioners in Educational Leadership. Paper presented at Regent University, School of Education, Virginia Beach, VA.

Cooner, D., & Dickmann, E. (2006). Assessing Principal Internships and Habits of Mind: The use of Journey Mapping to enhance reflection. Innovate 2(4). The Fischler School of Education and Human Services, Nova Southeastern University.

Cunningham, W. G. (2007). A handbook for educational leadership interns: A rite of passage. New York: Allyn Bacon.

Dallas Baptist University. (2004). Principal Internship Handbook. Dorothy M. Bush College of Education, Dallas, Texas.

Darling-Hammond, L. Myarson, D. LaPointe M. & Orr, M. (2010). Preparing principals for a changing world: Lessons from effective school leadership programs. San Francisco: Jossey Bass.

Florida Atlantic University. (2009). EDA 6942-Principal Internship activity log-Fall 2009. Department of Educational Leadership. Boca-Roton, FL. Retrieved from: http://www.leadership.fau.edu/Documents%20for%20EDA%206942/EDA%206942%20-%20Internship%20Activity%20Log.pdf

Fry, B., Bottoms, G. & O’Neill, K. (2005). The principal internship, how can we get it right? Atlanta, GA: Southern Regional Educational Board.

Hessel, K. and Holloway. (2002). A framework for school leaders license; Linking the ISLLC standards to practice. Princeton, NJ: Educational Testing Services.

Indiana Wesleyan University. (2010). Principal Licensure Program. Retrieved from: http://caps.indwes.edu/Bulletin/0708/825.htm

Jones, L. & Kennedy, E. (2008). Passing the leadership test: A study guide for the school leaders licensure examination. Lanham, MD: Rowman and Littlefield.

Martin, G. B., Wright W. F., Danzig A. B., Flanary, R. A., & Brown, F. (2005). School leader internship: Developing, monitoring and evaluating your leadership experience. Larchmont, NY: Eye on Education.

National Policy Board for Educational Leadership. (2002). Standards for Advanced Programs in Educational Leadership for Principals, Superintendents, Curriculum Directors and Supervisors.

NCATE/TEAC. (2010). [Online Report] Report of the NCATE/TEAC Design Team to Our Perspective Board of Directors. Retrieved January 3, 2011 from: http://www.teac.org/wp-content/uploads/2010/10/Design-Team-Report.pdf

Teacher Education Accreditation Council. (2005). Guide to accreditation. Washington, DC.

University of Nebraska at Kearney. (2003). Graduate Program Assessment: MSEd (Principal/Building Level) Practicum Internship Assessment. Retrieved from: http://www.unk.edu/academicaffairs/assessment.aspx?id=4886

Virginia Department of Education. (2007). Licensure Regulations for School Personnel. Richmond, VA.

Virginia Tidewater Consortium For Higher Education. (2005). Principal/Administrator orientation and preparation study guide: A self-paced tutorial for the school leaders licensure assessment. Virginia Beach, VA.: Crown Educational Services, Inc.

Content actions

Download module as:

PDF | EPUB (?)

What is an EPUB file?

EPUB is an electronic book format that can be read on a variety of mobile devices.

Downloading to a reading device

For detailed instructions on how to download this content's EPUB to your specific device, click the "(?)" link.

| More downloads ...

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks