Skip to content Skip to navigation

OpenStax-CNX

You are here: Home » Content » Development of Research-Based Protocol Aligned to Predict High Levels of Teaching Quality

Navigation

Recently Viewed

This feature requires Javascript to be enabled.
 

Development of Research-Based Protocol Aligned to Predict High Levels of Teaching Quality

Module by: Gary Schumacher. E-mail the author

Summary: This study proposes a research-based teacher selection protocol. The protocol is intended to offer school district hiring authorities a tool to identify teacher candidates with the behaviors expected to predict effective teaching. It is hypothesized that a particular series of research-based interview questions focusing on teaching behaviors in four key areas (i.e., classroom management, organizing for instruction, implementing instruction, and monitoring student progress and potential), can be used to predict high levels of quality teaching, which lead to enhanced student performance.

Development of Research-Based Protocol Aligned to Predict High Levels of Teaching Quality

Gary Schumacher, PhD

Assistant Professor

University of Houston-Clear Lake

2700 Bay Area Boulevard

Houston, TX 77058

Schumacher@uhcl.edu

Bettye Grigsby, PhD

Assistant Professor

University of Houston-Clear Lake

2700 Bay Area Boulevard

Houston, TX 77058

grigsbyb@uhcl.edu

Winona M. Burt, PhD

Assistant Professor

University of Houston-Clear Lake

2700 Bay Area Boulevard

Houston, TX 77058

burtwm@uhcl.edu

Abstract

This study proposes a research-based teacher selection protocol. The protocol is intended to offer school district hiring authorities a tool to identify teacher candidates with the behaviors expected to predict effective teaching. It is hypothesized that a particular series of research-based interview questions focusing on teaching behaviors in four key areas (i.e., classroom management, organizing for instruction, implementing instruction, and monitoring student progress and potential), can be used to predict high levels of quality teaching, which lead to enhanced student performance. This study addressed the following research question: Which teaching behaviors identify effective teachers? Hiring authorities can use this empirically based protocol knowing that research supports specific response patterns by effective teacher candidates, which in turn can lead to high levels of student achievement. The findings of this research study suggest a series of research-based interview questions that focus on teaching behaviors in four key areas to predict high levels of quality teaching. The research findings may provide valuable information to administrators and central office personnel regarding interview questions and their link to predicting qualities of effective teaching during the hiring process. Hiring authorities can utilize the research-based questions in the selection process. As a result of the findings from this research, they can be confident that teachers, upon initial hire, possess key teaching behaviors reported to result in high levels of student achievement.

Research has identified teacher quality as the single most important determinant of student achievement (Gentry, 2007; Kaplan & Owings, 2002; Salinas & Kristsonis, 2006; Stronge & Hindman, 2003). However, Stronge and Hindman (2003) conclude that the common denominator in school improvement and student success is teaching quality. Teaching quality, not just the characteristics of the teacher, is closely aligned to student academic achievement (Strong & Hindman, 2003).

Competent, qualified teachers are critical to student learning (Clement, 2009). Pillsbury (2005) maintains that the most effective approach to school improvement is one that focuses on teaching quality. A critical performance responsibility for principals is the hiring of quality teachers (Grove, 2009). Based on the research cited above, principals need to be certain they are hiring teachers who possess exemplary teaching skills to ensure optimum student learning.

Teaching quality can be defined differently depending on the circumstances, such as in hiring processes (Goe, 2007). Current research suggests that with the continuing emphasis on accountability, hiring authorities need to focus on teaching quality (Festencaher & Richardson, 2005). Pillsbury (2005) maintains that the most important decision that principals make is hiring teachers. Clement (2009) suggested, “in today’s era of accountability, we have high-stakes hiring, as a weak hire may impact student achievement, lower overall school performance, and lower morale of colleagues” (p. 22).

Research has suggested that a reasonable strategy for school districts seeking to improve student performance is to align human resources management practices to instruction (Grove, 2009; Heneman & Milanowski, 2004). One very important aspect of human resources management is the selection and hiring process of teachers. Many school districts decentralize aspects of this process, allowing school principals to interview teacher candidates for vacancies at their campuses. Interviews are the most popular predictor for hiring teachers, are used extensively, and are generally a required component of any hiring process (Castetter & Young, 2000; Delli & Vera, 2003). Rutledge, Harris, Thompson, and Ingle (2008) found that principals placed a great deal of emphasis in the hiring process on teacher interviews – on teacher quality, as they perceived it, in the interview process. “The majority of organizational research on employment interviews, spanning more than eighty years, has underscored the importance of structuring interviews to maximize their reliability as a decision-making tool” (Delli & Vera, 2003, p. 138).

Most hiring authorities would agree that teacher characteristics (the teacher as a person) are important factors in making effective hiring decisions. There are several commercially produced, structured (scripted and scored) instruments available. The developers of these instruments maintain that the use of these in the selection and hiring process can assist in assessing teacher characteristics and that these instruments bring a level of objectivity to hiring processes. Over 2,000 school districts use one of these commercially marketed instruments (Delli, 2001). One of the most widely used instruments, developed by Gallup, Inc. is the TeacherInsight (TPI). The TPI is an online interview protocol where teacher candidates respond to a series of statements using a Likert scale as well as a number of open-ended questions. These protocols are based on themes (teacher characteristics such as mission, empathy, activation, etc.) that the Gallup organization claims identify good teachers. If candidates respond affirmatively according to Gallup’s research-based “correct” responses when completing this instrument, Gallup asserts that there is high predictability of these candidates being “good” teachers.

However, the TPI, and other similar commercially-marketed instruments, do not measure the effective teaching performance but rather merely identify teacher candidates who possess traits, values and dispositions of effective teachers (Metzger & Wu, 2008). Other psychological influences in teacher interviews, such as attractiveness, likability, and similarity, have been studied and would support Metzger and Wu’s assertion (Delli & Vera, 2003). Metzger and Wu (2008) propose that the TPI primarily captures beliefs, values, and attitudes that are similar to those of the administrators who are using the instrument in the hiring process.

Metzger and Wu suggest that this instrument “seems to measure something, but we are not convinced that what it measures relates meaningfully to what matters for teaching effectiveness” (p. 933).

Although attempts to identify effective teachers using commercial instruments which assert to identify “good” teachers based on their personal characteristics, traits, values, and dispositions may not be a waste of time as these characteristics may be very important; they may not be useful in predicting effective teaching performance. Rather, research would suggest that a greater focus on teaching characteristics (teaching behaviors) in hiring practices can lead to high levels of achievement for all students (Goe, 2007).

“Questions must be asked about differences between beliefs, attitudes, and values that are relevant to pedagogical effectiveness” (Metzger & Wu, 2008, p. 934). Therein lays the key difference in this research study. Rather than focus on teacher characteristics to predict effective teaching in the hiring process, as most commercially developed products focus on, this research focuses on teaching behaviors to predict effective teaching, as defined by high levels of student performance, in the hiring process.

This study addressed the following research question:

“What are the emerging themes of effective teaching resulting from the interview protocol?”

Methodology

Participants

Study participants included a convenience sample of 222 working teachers who reported 7.2 years of teaching experience, on average. Sixty-eight percent of the teachers taught at the elementary school level, 19% taught at the middle school level, and 13% taught at the high school level. Subjects taught by the participants included language arts, reading, writing, music, social studies, mathematics, science, algebra, foreign languages, English as a second language, and physical education. All respondents were employed in two large public school districts, one in metropolitan Houston and one in metropolitan Milwaukee. These two districts were selected because the researchers have had prior research experience with them and have developed relationships with district administrators. Approval to conduct this research was obtained from the researchers’ university institutional review board and from each district.

Research Design

The study unfolds in two phases and phase one is reported on here. A qualitative research design was used to address the research question in phase one of the study. In phase two of the study, qualitative teacher responses from phase one will be quantified using a seven-point Likert-scale. The resulting quantitative data will be used as independent variables in a multiple regression analysis to predict average student gains for each teacher.

Procedures

Open-ended protocol questions were piloted one week prior to full implementation by principals chosen from each district’s human resource personnel. Principals were asked to announce the opportunity for participation in the study during a staff meeting. Principals followed up the announcement with an email invitation to teachers. Participation was voluntary and respondents were asked to complete the interview protocol online. The full interview protocol is presented in the Appendix. All teachers on the campus were given the opportunity to individually answer or decline to answer the protocol questions. Each district electronically delivered responses from participating teachers to the researchers. The open-ended responses will be used to generate Likert-scale protocol-items for phase two of the research study.

Instrumentation Development

The interview questions for this research study were developed by the researchers, in cooperation with the principals participating in the study. The questions addressed key qualities of effective teachers (teacher behaviors) as defined by Stronge, Tucker and Hindman (2004). There were four teaching behaviors selected:

1. Classroom management and organization.

Included in this category are student behavior expectations, classroom organization (physical environment), and classroom climate.

2. Organizing for instruction.

This category includes focus on instruction, maximizing instructional time, high student expectations, and instructional planning and preparation.

3. Implementing instruction.

This includes effective utilization of instructional strategies, communication of content knowledge, contextual understanding, questioning strategies and student engagement.

4. Monitoring student progress and potential.

This final category includes collecting evidence of student learning, assessment design and strategies, aligning instructional strategies to assessment, and differentiating instruction (Stronge et al., 2004).

The above four areas were the foundation for developing a set of proposed “behavior-based” interview questions for the teacher research interview protocol. Behavior-based interview questions have been identified as being very effective in predicting a candidate’s future behavior based on their responses to past performance (Clement, 2009). To aid in the further development of an instrument with strong predictability in identifying high levels of teaching performance during the hiring process, round table discussions occurred with each participating district. A three-hour workshop was conducted with principals and the assistant human resource director in a large, metropolitan Houston school district. During this dialogue, each question was read to determine readability, meaning, and format. The same type of forum ensued during a three-hour session with principals, superintendent, assistant superintendent, special education director, and the director of human resources in a large, metropolitan Milwaukee school district. After both sessions, the questions were revised according to recommendations and comments.

Two of the researchers presented the revised questions to 30 existing teachers in their graduate-level, administrative preparation courses for critical feedback for improving the quality and efficiency of each question. Students were given the opportunity to answer the questions so researchers could determine the amount of time it took to complete the protocol. Upon completion, a class discussion on question ambiguity, question difficulty, and order of questions took place. Based on student dialogue and responses, some questions not answered as expected were re-worded. Pertinent information on the magnitude of variation of responses emerged during this process as well.

After revisions based on multiple forums of input, the interview protocol utilized in this study consisted of nine questions under classroom management and organization, two questions for organizing instruction, three questions under implementing instruction, and five questions for monitoring student progress and potential (see Appendix).

Analysis

During this first phase of research, open-ended questions in the four identified areas were presented to existing teachers from participating districts in the form of an online questionnaire. Themes were identified from the open-ended questions. The reoccurring themes that emerged are identified in the following section.

Findings

Displaying quality teaching behaviors in classroom management and organization, organizing instruction, implementing instruction, and monitoring student progress and potential are all important components of an effective teacher. Teachers have a profound effect on the students under their tutelage. Therefore, it is imperative that teachers exhibit behaviors that will lead to student success (Stronge, 2007).

Classroom Management and Organization

The first few days of school are crucial when developing effective classroom management and organization throughout the entire school year (Wolfgang, 2009). Just as an effective teacher carefully designs a quality lesson, the same time and preparation must be applied when organizing the classroom. Organization of the classroom includes such areas as room arrangement, discipline, and creating routines (Stronge, 2007). Arthur-Kelly, Lyons, Butterfield, and Gordon (2006) developed a classroom management model with strategies classified into two categories: (a) creation of a positive learning environment in order to yield positive student behaviors, and (b) handling student disruptive behaviors.

Successful teachers in our study utilized one or more of the following strategies when handling the area of classroom management and organization. The students as a community of learners decided the guidelines for acceptable behaviors and consequences. Appropriate expected behaviors were modeled, taught, and re-taught along with visual reminders at the beginning and throughout the year. Classroom contracts containing academic and behavioral expectations were issued to students for parent signatures. These teachers also organized the classroom so that at the beginning of class students were accustomed to entering class, gathering necessary materials, taking their seats, and working on the bell ringer while the teacher performed daily administrative tasks. The last five minutes were left for wrap-up, filling out their personal conduct sheets, and completing an exit ticket to assess understanding of the lesson’s objective.

Organizing Instruction

Taking time to plan engaging and effective lessons is a critical piece of successful teaching. Effective lesson planning allows teachers the opportunity to critically think about how to deliver instruction, develop varied activities so students can comprehend the subject matter, and determine how individual students can best master the content (Shen, Poppink, & Cui, 2007). Two important activities effective teachers engage in prior to the delivery of instruction are material preparation and time organization (Stronge, 2007).

In the area of organizing instruction, key elements such as the objective, individual or group activities, and assessments were included in the daily lesson plan. The structure of the lesson delivery, the different learning styles of students, and critical concepts and skills students needed to learn by the end of the lesson were considered when planning a lesson.

Implementing Instruction

Once a teacher has organized the classroom and developed the lesson, teaching must occur. How a teacher presents the material to a student is critical to the success of a student. Research supports that one teaching strategy does not fit all students (Gregory & Chapman, 2002; Marzano, Pickering, & Pollock, 2001; Silver, Strong, & Perini, 2000). Therefore, an effective teacher must implement a variety of activities that takes into account different learning styles and multiple intelligences.

In this research, teachers utilized various instructional strategies when implementing instruction to challenge all learners, accommodate different learning styles, and to ensure student success. These strategies included small group instruction, cooperative grouping, various activities that incorporate visual, tactile, and audio intelligences, think-pair-share, peer tutoring, manipulatives, video clips, and choices on forms of assessments and projects.

Monitoring Student Progress and Potential

Constantly determining if a student understands the content and finding a way for a student to express understanding are behaviors effective teachers exhibit daily. It is imperative in the age of high stakes testing that teachers know the strengths and deficits students possess on taught content. This knowledge can only be gained by developing strategies to consistently monitor student progress.

Successful teachers in this study monitored student progress and potential through observations, portfolios, weekly assessments, disaggregated data from testing, exit tickets, higher order questioning techniques, peer explanations (teaching), summaries in learning logs, and graphic organizers.

Conclusion

The behaviors exhibited by teachers determine their effectiveness in the classroom and, ultimately, the impact they have on student achievement (Stronge, 2002). The thought process and practices effective teachers utilized while organizing their classrooms, determining expected behaviors and consequences, preparing for instruction, implementing instruction, and monitoring students progress and potential are key to student success. “Merging teacher selection practices and research regarding qualities of effective teachers can help to ensure that effective teachers are hired” (Grove, 2009). This longitudinal study will provide hiring authorities with research-based protocols that have proven to predict high levels of teaching quality, which research has shown to be the single most important determinant of student achievement.

Recommendations for administrators

  1. Incorporate interview questions in the hiring process that address the four research based areas identifying quality-teaching behaviors.
  2. Provide struggling existing teachers with professional development in the four areas.

Recommendations for teacher preparation programs

  1. Include the four researched based quality-teaching behaviors when preparing teacher candidates.
  2. Professors should model these behaviors during lessons and allow students the opportunity to visit local teachers that exhibit these behaviors in their classrooms.
  3. Provide teacher candidates an opportunity to model the research-based behaviors.

References

Arthur-Kelly, M., Lyons, G., Butterfield, N., & Gordon, C. (2006). Classroom management: Creating positive learning environments. 2nd ed. South Melbourne Australia: Thomson.

Castetter, W., & Young, I. P. (2000). The human resources function in education administration. Columbus, OH: Prentice Hall.

Clement, M. C. (2009). Hiring highly qualified teachers begins with quality interviews. Kappan. 9(2), 22-24.

Delli, D. A. (2001). Criteria for adopting commercial interviews for teacher selection. The AASA Professor, 25(1), 30-34.

Delli, D. A., & Vera, E. M. (2003). Psychological and contextual influences on the teacher selection interview: A model for future research. Journal of Personnel Evaluation in Education, 17(2), 137-155.

Festencaher, G. D., & Richardson, V. (2005). On making determinations of quality in teaching. Teachers College Record, 107(1), 186-213.

Gentry, R. (2007). It takes 2 to produce a quality teacher. The Third Annual Reaching Out to Mississippi Education in Action (ROMEA) Conference, 1-14. Cleveland: Delta State University.

Goe, L. (2007). The link between teacher quality and student outcomes: A research synthesis. Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved June 28, 2009, from http://www.ncctq.org/publications/LinkBetweenTQandStudentOutcomes/pdf.

Gregory, G. H., & Chapman, C. (2002). Differentiated instructional strategies: One size doesn’t fit all. Thousand Oaks, CA.: Corwin Press, Inc.

Grove, S. (2009). A comparison of elementary, middle, and high school principals' teacher selection practices and perceptions of teacher effectiveness. (Doctoral dissertation, The College of William and Mary, 2009). (UMI No. 3340948)

Heneman, H. G., & Milanowski, A. T. (2004). Alignment of human resource practices and teacher performance competency. Peabody Journal of Education, 79(4), 108-125.

Kaplan, L. S., & Owings, W. A. (2002). Teacher quality, teaching quality, and school improvement. Bloomington, IN: Phi Delta Kappa.

Marzano, R.J., Pickering, D.J., & Pollock, J.E. (2001). Classroom instruction that works: Research-based strategies for increasing student achievement. Alexandria, VA.: Association for Supervision and Curriculum Development.

Metzger, S. A., & Wu, M. J. (2008). Commercial teacher selection instruments: The validity of selecting teachers through beliefs, attitudes, and values. Review of Educational Research. 78(4), 921-940.

Pillsbury, P. (2005). Only the best: Hiring outstanding teachers. Leadership. 35(2), 36-38.

Rutledge, S. A., Harris, D. N., Thompson, C.T., & Ingle, W. K. (2008). Certify, blink, hire: An examination of the process and tools of teacher screening and selection. Leadership & Policy in Schools. (7)3, 237-263.

Salinas, R., & Kristsonis, W. (2006). National challenge: teacher quality and student achievement in public schools. National Journal for Publishing and Mentoring Doctoral Student Research, 1(November), 1.

Shen, J., Poppink, S., & Cui, Y. (2007). Lesson planning: A practice of professional responsibility and development. Educational Horizons. 85(4), 248-258.

Silver, H.F., Strong, R.W., & Perini, M.J. (2000). So each may learn; Integrating learning styles and multiple intelligences. Alexandria, VA.: Association for Supervision and Curriculum Development.

Stronge, J.H. (2002). Qualities of effective teachers. Alexandria, VA: Association for Supervision and Curriculum Development.

Stronge, J.H. (2007). Qualities of effective teachers. 2nd ed. Alexandria, VA: Association for Supervision and Curriculum Development.

Stronge, J. H., & Hindman, J. L. (2003). Hiring the best teachers. Educational Leadership. 60(5). 48-52.

Stronge, J. H., Tucker, P. D., & Hindman, J. L. (2004). Handbook for qualities of effective teachers. Alexandria, VA: Association for Supervision and Curriculum Development.

Wolfgang, C.N. (2009). Managing inquiry-based classrooms. Science Scope 32(9). 14-17.

Appendix

Teacher Research on Interview Protocol – Interview Questions

This survey instrument consists of questions in the four areas considered qualities of effective teaching. Please respond to each question based on your experience in the classroom this school year.

Classroom Management and Organization

  1. How do you develop your academic expectations for students at the beginning of the year?
  2. How do you communicate your academic expectations to students at the beginning of the year?
  3. How do you develop your behavioral expectations for students at the beginning of the year?
  4. How do you communicate your behavioral expectations to students at the beginning of the year?
  5. Describe your system of classroom management.
  6. Describe the learning environment you want to create in your classroom.
  7. How do you create this environment?
  8. Describe what your students are doing the first five minutes of class.
  9. Describe what your students are doing the last five minutes of class.

Organizing for Instruction

  1. What key elements do you include in the daily lesson plan?
  2. What do you consider when beginning your lesson planning for a class?

Implementing Instruction

  1. How do you differentiate instruction (challenge struggling learners and advanced learners) within the same class?
  2. Describe your teaching style.
  3. How do you ensure that students experience success?

Monitoring Student Progress and Potential

  1. How do you monitor student progress?
  2. How do you evaluate student progress?
  3. What learning behaviors do you look for during the lesson to determine individual student mastery of the concept?
  4. Name ways individual students within a group can demonstrate they understand and can apply the concept.
  5. Explain how you use assessments (classroom and state) as a way to improve student learning.

Demographic Information

  1. What are your current years of teaching experience? (including the current year) ____________
  2. What subjects are you currently teaching? ____________
  3. What grade level are you currently teaching? __________

Content actions

Download module as:

Add module to:

My Favorites (?)

'My Favorites' is a special kind of lens which you can use to bookmark modules and collections. 'My Favorites' can only be seen by you, and collections saved in 'My Favorites' can remember the last module you were on. You need an account to use 'My Favorites'.

| A lens I own (?)

Definition of a lens

Lenses

A lens is a custom view of the content in the repository. You can think of it as a fancy kind of list that will let you see content through the eyes of organizations and people you trust.

What is in a lens?

Lens makers point to materials (modules and collections), creating a guide that includes their own comments and descriptive tags about the content.

Who can create a lens?

Any individual member, a community, or a respected organization.

What are tags? tag icon

Tags are descriptors added by lens makers to help label content, attaching a vocabulary that is meaningful in the context of the lens.

| External bookmarks