RESEARCHING EDUCATIONAL LEADERSHIP: A CONCEPTUAL FRAMEWORK FOR DOCTORAL PROGRAM RESEARCH

Module by: Dale Johnson, Tod Allen Farmer

Summary: The NCPEA Handbook of Doctoral Programs in Educational Leadership: Issues and Challenges, Chapter 5, authored by Dale M. Johnson, Danna M. Beaty, and Tod A. Farmer.

This chapter is devoted to a description of a conceptual framework for the investigative component of an EdD program. During the last two decades of the 1900s and the first part of this century, the research arena has experienced controversies on topics ranging from the philosophic roots of scientific inquiry to methodologies and implementation techniques. The state of graduate level education research is complex, changing, and, too often, confusing. The framework presented in this chapter serves one doctoral program as a way of viewing doctoral inquiry complexities in a logical and integrated format.

Following up on Berliner’s (2002) sobering view of the educational research scene, Eisenhart and DeHann (2005) reinforced experiences familiar to faculty and administrators grappling with the chaotic research arena of education doctoral programs by noting that “graduate training programs in education research face a more diverse and challenging set of obstacles to socializing students to the norms of scientific inquiry than are faced by traditional programs” (p. 18). The complexity of investigative components of graduate education has been recognized by many researchers and scholars and includes defining appropriate roles of research in PhD and EdD programs (Boote & Beile, 2005; Eisenhart & DeHaan, 2005), a clash between scholar and practitioner cultures (Barger & Duncan, 1986; Labaree, 2003), a division between objective and subjective research (Ercikan & Roth, 2006), and a struggle “to strike a balance between the practice in education and research in education” (Shulman, Golde, Bueschel, & Garabedian, 2006, p. 26). Shulman, Golde, Bueschel, & Garabedian (2006) argued that confusion surrounds the purposes of preparing scholars and practitioners and as a result, neither is satisfactorily accomplished.

There is no shortage of suggestions for addressing the problem of preparing educational researchers in graduate school (Barger and Duncan, 1986; Berliner, 2002; Booate & Beile, 2005; Ercikan & Roth, 2006; Labaree, 2003; Schoenfeld, 1999; Shulman, Golde, Bueschel, & Garabedian, 2006; Siegel, 2006;). However, a common thread appearing throughout the literature is the need for integration and combining investigative resources to address well defined research problems. Eisenhart and DeHann (2005) advocated an approach socializing doctoral students to a “culture of science” and “doctoral training that is consistent with a broad definition of scientifically based research” (p. 3).

Background and Context

Culminating a five year process, in 2001 a regional university was approved to offer the first doctoral program in the history of the institution. The degree was an EdD in Educational Leadership and Policy Studies and was structured around coursework in the leadership core, tools of inquiry, and an elective specialization support area. Because the design of the doctoral program was based on a scholar-practitioner paradigm, the tools of inquiry were envisioned to be formally taught in inquiry classes but were to be consistently reinforced and integrated throughout the core and specialization curricula.

The unsteady state of doctoral research described in the literature permeated initial discussions, long-range planning, and program definition as the EdD program was started. This state of affairs illuminated the need to organize doctoral inquiry into a format that would guide thoughts, discussions, and actions regarding the program. Starting with the scholar-practitioner model as a guide, the students in the program were expected to perform scientific investigation using accepted scholarly methodology to address applied and practical problems. Scholarly endeavors would address needs identified by problematic conditions in the field; and conversely, practitioners would be guided by research-based best practices. Within the parameters of the scholar-practitioner model, the faculty developed a conceptual framework that addresses the call from literature for integrated methodology.

The Conceptual Framework

While no model fully represents the complexity of reality it represents, the framework functions to better define the inquiry domain of the doctoral program. Therefore, it serves as a framework for clarifying program structure, for long-range planning, for defining the domain for program accreditation purposes, and for delineating “acceptable” areas of investigation for doctoral dissertations. The conceptual framework for the investigative dimension of the doctoral program is based on three foundational content areas: 1) Scientific Research, 2) Statistics, and 3) Program Evaluation.

Scientific Research

Rather than attempt to perpetuate arguments found in the literature about methodology for defining “research”, the definition selected was a traditional one that is often cited in modern research texts. The scientific research anchor for the conceptual framework was taken from Kerlinger and Lee (2000): “Scientific research is systematic, controlled, empirical, amoral, public, and critical investigation of natural phenomena” (p. 14). The definition starts with specifying that research is systematic and controlled. In order to satisfy these two conditions in varying degrees, designs in scientific research approaches include issues of randomization, attention to moderator and mediator influences, philosophies of science, and an approach known as the “scientific method.” Encouraging systematic procedures and processes during the doctoral dissertation stage minimizes the chances that inquiry will be haphazard and provides impetus for well prepared proposals. Further, research is empirical which implies that data are gathered that can be confirmed by an outside independent source as credible evidence. Research is subject to peer review, and one way to involve others in a critical review process is to make the works public. According to the definition, research results are not scrutinized through the lens of moral evaluation. Scientific research is concerned with testing theory, generating theory, addressing major problem areas and may be quantitative or qualitative, or some mixed-method classification.

When viewed as a sequential pattern of milestones for conducting research, the starting point is the existing body of knowledge. This body consists of information about what is known in the field of interest along with propositions, definitions, and theoretical perspectives about relations among constructs or concepts. The starting position for the research sequence requires familiarity with the current literature and current thinking in the field which are demonstrated in the review of literature activity. The second milestone is a statement of a problem which is derived from a theoretical perspective in the body of knowledge. Research questions or hypotheses are generated by the statement of the problem with the assistance of operational definitions. The statements of the research questions or hypotheses identify the data necessary for the research effort; consequently, the next step is to specify an appropriate research design to secure the necessary data or observational evidence. Once collected, data are analyzed appropriately to provide results which are in turn generalized to form conclusions which become part of the body of knowledge—the starting point for the research process.

Statistics

A second foundational content area for the doctoral conceptual framework is the field of statistics. Continuing to rely on foundational time-tested concepts, development of the model reverted back to a classical definition of statistics: “Statistics is the science dealing with organizing, summarizing, and analyzing, and interpreting numerical data” (Johnson, 1989, p. 7). Central to the definition is the term data. Data refers to a set of facts; more specifically, in the science of statistics, data refers to facts expressed as numbers. The numbers may be quantitative (numerically scaled as ratio, ordinal, or interval) or qualitative (nominal). Organizing the numbers into systematically arranged formats facilitates examinations of the evidence. Summarizing data describes numerical distributions while analyzing data implies that numbers are going to be arithmetically manipulated using various techniques for the purpose of extracting desired information from the numerical distributions. Finally, interpreting data includes arriving at results and probability-based conclusions drawn from results of data summaries and data analyses.

Typically, statistics is subdivided into descriptive (summarizing data that have been collected) or inferential (probabilistic conclusions about data not collected). As the name implies, descriptive statistics serve to describe numbers representing measurements or variables. Descriptive statistics are generally expressed visually as displays of graphs or tables and are expressed analytically with summary values for central tendency, dispersion, or relationships. Inferential statistics take on many forms depending on the hypotheses to be tested or research questions posed, but many times are expressed as confidence intervals (estimation) or significance levels (p-values). Modeling as an inferential technique often is accompanied with visual diagrams showing relations among latent or measured variables. Statistics as part of the foundation tripod glean information from numerical data and provide various ways of presenting meaningful results of data analysis.

Program Evaluation

A viable area of inquiry with roots ranging back to Ralph Tyler and efforts to achieve more efficient productivity during World War II achieved significant recognition in the educational field in the early 1970's through the work of such evaluators as Michael Scriven, Daniel Stufflebeam, Malcolm Provus, Egon Guba, Robert Stake, Blain Wortham, Eva Baker, Peter Airasian and James Sanders (Fitzpatrick, Sanders, & Worthen, 2004). Program Evaluation, the third foundational area for the conceptual framework, is generally recognized as a sequence of processes involved with delineating needs, goals for addressing the identified needs, the program designed to achieve the goals, and judgments about the effectiveness of the processes and products of the program with respect to standards or critical competitors. Evaluation is typically focused on a local program or project and may be qualitative or quantitative in nature.

As with the other two foundational areas of the framework, program evaluation takes on a variety of methodologies depending on the model employed, needs of the major audiences or stakeholders, and the philosophic orientation of the evaluation. However, the process generally begins with a needs assessment based on a discrepancy between the current status and the desired status. Or, if the program or project has been initiated, the first step in an evaluation is usually to identify the goals and objectives of the program – i.e., what the program is intended to accomplish. An evaluation plan describes both formative (process) and summative (product) schemes (Scriven, 1967) for collecting useful evidence on the effectiveness of the program. The evidence is synthesized and analyzed to produce findings. The findings are compared to predetermined standards and judgments about the value of the program from the conclusions. The conclusions are framed as recommendations and reports for stakeholders and program managers. Unlike scientific research, program evaluation is aimed at a local situation and does not require testing of theoretical propositions; rather, evaluation activities culminate in value judgments about the effectiveness of a specific program in a specific locale.

Building on the three foundation content areas of scientific research, statistics, and program evaluation, the interdependence among the three areas is visually displayed using methodology attributed to a 19th century mathematician John Venn (Johnson, 1989). Specifically various intersections or overlapping sections of the sets represented by scientific research, statistics, and program evaluation are portrayed in a Venn diagram. Much of the coursework and dissertation activity in most doctoral programs are focused in the intersection of the sets. The intersection represents a combination of foundational content areas and is viewed as integrated and interdisciplinary rather than polarizing.

Quantitative Research

The section in Figure 1 representing the large overlapping portions of the content fields of scientific research and statistics delineates a sub-domain called Quantitative Research.

Quantitative Research is consistent with the positivist philosophy in research. This intersection is a marriage of research design, measurement, and analysis. Numerical data are collected by the process of measurement which is part of the design or plan for the investigation. The data are subjected to statistical analysis for the purpose of describing numerical distributions, testing hypotheses or research questions, revealing confidence intervals, testing models, presenting results, and forming inferential conclusions about the object of study. The almost symbiotic nature of research and statistics is a result of research design producing data that need analyzing; and, statistical techniques requiring data in order to perform their function.

Qualitative Research

Qualitative Research is often viewed as post positivist research reflecting its philosophical foundation. The intersection of scientific research and program evaluation as shown in Figure 2 is conceptualized as the field known as qualitative research. Qualitative research has roots in many of the disciplines from the social sciences including anthropology, sociology, history, philosophy, and psychology. Traditions of qualitative research vary greatly in procedures for inquiry, purpose and popularity among researchers (Creswell, 1998; Gall, Gall & Borg, 2003). However, in keeping with the intent of the conceptual framework, emphasis is placed on the traditions of biography, phenomenology, ethnography, case study, and grounded theory. Carried out in natural settings where subjects are behaving naturally and data (evidence) are collected via field notes, interviews, surveys or other kinds of direct observation, this research is often undertaken in an effort to give voice to those in the field who have traditionally been discussed in terms of statistical representation and/or as a collective group rather than as individuals. This provides the opportunity for the researcher to examine multiple dimensions of a problem or issue and discuss it in all its complexity (Creswell, 1998). Furthermore, an understanding of qualitative inquiry provides students with the skills needed to conduct evaluative research within their own districts and on their own campuses.

Although qualitative research often employs rigorous, systematic procedures for inquiry, in contrast to quantitative research, qualitative inquiry outlines a general approach to a study which allows for the researcher to address emerging issues in the field. A set of philosophical assumptions guide the study and speak to our understanding of knowledge—knowledge of self, knowledge gathered and knowledge of the context in which it is studied. The researcher in a qualitative study differs significantly from that of a quantitative study in that the researcher is an active learner telling the story from the participants’ view rather than that of the “expert” who makes a judgment based on a data set (Creswell, 1998). This method of inquiry allows the scholar-practitioner to study and discuss problems or issues of the educational community through the use of language and narrative rather than numerical data.

Data-Driven Decision Making

Joining the elements of statistics and evaluation (Figure 3) forms the core for Data-Driven Decision Making. Data-driven decision making involves the use of data and data analysis to inform when deciding courses of action involving policy and procedures (Picciano, 2006). As a major player in policy and procedure determination, data-driven decision making training in a doctoral program in Educational Leadership seems particularly apropos. Data-driven decisions rely to a large degree on statistical description, disaggregation of data, and aggregation of numerical data from local sources for assisting in making judgments about school or community issues.

Five principles of data-driven decision making posited by Love (2002) are also guiding assumptions of statistics and/or evaluation – they are: 1) data-driven decisions fuels school reform, 2) such inquiry relies on rigorous use of data, 3) the inquiry is collaborative, 4) the focus of inquiry is improving student learning, and 5) improving student learning requires a systematic approach (pp. 6-9). Systems theory provides the major framework for the topic of data-driven decision making in schools and “the major components of a system, input, process, and output – and their interrelationship—are generally accepted as fundamental to all aspects of information system development, as well as to decision processes” (Picciano, 2006, p. 9). These three components outlined by Picciano can be shown to be fundamental to both program evaluation and scientific research thereby helping to explain the labeling of the intersection of the two sets.

Doctoral Inquiry

The conceptual framework for the inquiry dimension of the doctoral program is portrayed by the Venn diagram in Figure 4; specifically by the interlocking circular intersections of the three major domains of scientific investigation. Using approaches and methodologies from the three major foundations provides a foundation for maintaining the credibility and trustworthiness of doctoral inquiry.

The intersection defining doctoral inquiry draws strengths from the traditional and well-defined content areas of scientific research, statistics, and program evaluation. The doctoral inquiry dimension in the diagram also conveys the notion of an integration of approaches called for in the literature (Eisenhart & DeHaan, 2005; Barger & Duncan, 1986; Siegel, 2006) or as a hybrid program that emphasizes connections between theory and practice (Labaree, 2003).

Conclusions

The conceptual framework establishes a shared vision of the doctoral research program for faculty, students, accreditation entities, prospective students, and administrators. It fully represents and embraces scientific inquiry norms from the behavioral sciences, social sciences, and education. Although it encompasses major domains of scientific inquiry, it reduces the bewildering variety of factors explicated in the current literature to a few key concepts. The framework provides parameters within which the inquiry knowledge-base can be defined, serves as a guide for collective action by leaders and stakeholders, and provides a setting for decision making on curriculum issues.

For doctoral students, the conceptual framework defines “good” research and permits the alignment between expectations in the inquiry domain of the doctoral program and candidates’ performances. To most students, the field of “research” appears as an intimidating mass of quasi-related facts mostly tied to conflicting methodologies. By establishing a conceptual framework, students visualize a set of coherent concepts organized in a manner that facilitates communication among students and faculty. Thus, the framework reduces the wildly complex arena of inquiry to a conceptually simple visual that captures the essential components of inquiry while opening opportunities for students to use their innate mental and scholarly capabilities for mapping out more specific inquiry activities.

While the particular conceptual framework model presented in this chapter may not appeal to, nor be accepted by, all educational leadership doctoral programs, the activities and interaction among faculty members during the process of developing the paradigm have proven beneficial for communication among faculty and between faculty and doctoral candidates. This suggests that not only do summative conceptual frameworks in their final form have potential for enhancing the vision for a doctoral research component, the process and interaction has the potential to provide profitable experiences for faculty and students during the formative stage of development. Finally, the conceptual framework remains subject to revision and modification. The present form is an improvement over the initial model created by the faculty, and will undoubtedly be expanded, reduced, or altered during the next five-year tenure of this infant doctoral program.

As the national demand for doctoral programs continues to increase, a conceptual framework for doctoral program research can assist doctoral program development committees with the challenge of bringing purpose and precision to doctoral inquiry. By moving what remains for many in the realm of the abstract into a concrete, integrated format, the conceptual framework provides a common frame of reference for both faculty and students. Research activities become purposeful and constructive. Doctoral inquiry culminates into an apex of seamlessly integrated scholarly activities. Finally, the conceptual framework articulates the doctoral program research activities.

References

Barger, R. R., & Duncan, J. K. (1986). Creativity in doctoral research: A reasonable expectation? The Education Forum, 51(1), 33-43.

Berliner, D. C. (2002). Education research: The hardest science of all. Educational Researcher, 31(8), 18-20.

Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15.

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks: Sage.

Eisenhart, M., & DeHann, R. L. (2005). Doctoral preparation of scientifically based education researchers. Educational Researcher, 39(4), 3-13.

Ercikan, K., & Roth, W. M. (2006). What good is polarizing research into qualitative and quantitative? Educational Researcher, 35(5), 14-23.

Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd ed.). Boston: Pearson Education, Inc.

Gall, M. D., Gall, J. P., & Borg, W. R. (2003). Educational research: An introduction (7th ed.). Boston: Allyn and Bacon.

Johnson, D. M. (1989). Probability and statistics. Cincinnati: South-Western Publishing Co.

Kerlinger, F. N., & Lee, H. B. (2000). Foundations of behavioral research (4th ed.). Florience, KY: Thomson Learning.

Labaree, D. F. (2003). The peculiar problems of preparing educational researchers. Educational Researcher, 32(1), 13-22.

Love, N. (2002). Using data/getting results: A practical guide for school improvement in mathematics and science. Norwood, MA: Christopher-Gordon Publishers, Inc.

Picciano, A. G. (2006). Data-driven decision making for effective school leadership. Upper Saddle River, NJ: Pearson.

Schoenfeld, A. H. (1999). The core, the canon, and the development of research skills: Issues in the preparation of education researchers. In E. C. Largemann & L. S. Shulman (Eds.), Issues in education research: Problems and possibilities (pp. 166-202). San Francisco: Jossey-Bass.

Scriven, M. (1967). The methodology of evaluation. AERA Monograph Series on Curriculum Evaluation, No. 1. Chicago: Rand McNally.

Shulman, L. S., Golde, C. M., Bueschel, A. C., & Garabedian, K. J. (2006). Reclaiming education’s doctorates: A critique and proposal. Educational Researcher, 35(13), 25-32.

Siegel, H. (2006). Epistemological diversity and education research: Much ado about nothing much? Educational Researcher, 35(2), 3-12.

Author Biographies

Since 2003, Dale Johnson has served as Research Professor in the Department of Educational Leadership & Policy Studies at Tarleton State University.  His higher education experience spans four decades of teaching, research/scholarship, grants administration and academic administration.  Johnson was a 2001 inductee into the Oklahoma Higher Education Hall of Fame. 

Danna M. Beaty is an assistant professor in the Department of Educational Leadership and Policy Studies at Tarleton State University. Her research interests include leadership preparation, mentorship of women and minorities, and leadership for social justice.

Tod Allen Farmer is an assistant professor in the Department of Educational Leadership and Policy Studies at Tarleton State University.  Dr. Farmer has extensive K-12 administrative experience at both the building and central office levels.  His primary research interest is in higher education administration.

Comments, questions, feedback, criticisms?

Send feedback

• E-mail the authors of the module, RESEARCHING EDUCATIONAL LEADERSHIP: A CONCEPTUAL FRAMEWORK FOR DOCTORAL PROGRAM RESEARCH

Footer

This work is licensed by Dale Johnson and Tod Allen Farmer under a Creative Commons Attribution License (CC-BY 2.0), and is an Open Educational Resource.

Last edited by National Council of Professors of Educational Administration on Jul 19, 2007 3:54 pm GMT-5.