Survey Methods

Pilot and group discourse. For the survey “‘The Greatest’ Living Scholars of Our Time” (that some quipped to be a “tough assignment”) recipients were asked to “take a moment to make a difference by answering this question,” to nominate one person, and to “briefly jot down the reason(s) for your choice.” The form alluded to the necessity of being able to make an informed judgment (“this survey assumes that you’re in the field of educational leadership”). It was established that any feedback would be anonymously reported. A pilot sampling, conducted in 2002 at a doctoral research-extensive metropolitan university in Florida, involved eight educational leadership professors. The question was openly tested and the interest level gauged. This process further verified the value/importance of the question itself, so I broadened my data collection efforts. I also learned that some faculty wanted either to nominate more than one person or to vote with reservations, an outcome that foreshadowed an emerging pattern on a larger scale for some respondents.

Fuller sampling and distribution. Turning to listserves of professional associations, I accessed those most relevant to my study, including university-based educational leadership departments located through Internet searches. Conference councils and educational leadership editorial teams also received the survey. My goal was to obtain 200 surveys—214 (6%) complete responses were received; additionally, 19 electronic messages were returned explaining why a nomination was not possible. In all, 233 (7%) responses were analyzed. Approximately 3,500 individuals would have received the survey (some more than once), but this number proved impossible to accurately track.

While the goal set for the completed surveys was met, the very low return rate needs to be addressed, especially when one considers that a good response rate of 50 to 60% is generally considered accepted for survey research (Diem, 2002). However, at least one social science research team has found that its traditional paper survey yielded a higher response rate (60%) than the same questionnaire distributed electronically, which dropped significantly to 27%; this led them to question whether e-surveys are a reliable means of collecting data from a targeted population (Fraze, Hardin, Brashears, Smith, & Lockaby, 2003). In the long list provided by Newman (2002) for increasing one’s response rate from surveys in general, I used most of the suggestions—provide a salient question, indicate why the respondent’s answer matters, keep the survey short and simple, use university identification and a personalized note, and follow up with a reminder. I did not make many telephone calls or offer monetary incentives, both optional strategies for maximizing one’s response rate.

While I do not know why for certain why the response rate was low, several possibilities do come to mind. For one thing, the tensions evident in some of the responses (and non-votes) received suggest that the nature of the topic itself is controversial. Asking who the most exceptional scholar in educational leadership is today may seem confounded for those who question the very notion of “greatness,” especially among one’s living contemporaries, or those who can think of more than one person, or those who can think of no one at all deserving of such status. As Renzetti and Lee (1993) acknowledge, researching sensitive topics poses conceptual, methodological, and imaginative challenges, an explanation that describes my own experience.

As another possibility, there is little control that can be exerted over a study that is conducted electronically and where the recipients, although from a targeted population, may question the survey focus or the researcher’s motives. This is why I spent additional time collecting the data, re-stating the purpose of the study, eliciting some responses in-person, and emailing reminders to non-respondents, all as strategies for seeking a higher response rate.

Specifically, the survey was electronically circulated to the American Educational Research Association’s (AERA) Division A (Administration), which had 820 members in 2002, and AERA’s Division K (Teaching & Teacher Education), which had 1,004 (http://www.aera.net). Additionally, representatives of UCEA’s executive council and member institutes were recipients (http://www.ucea.org), along with NCPEA’s 1,622 members (http://www.ncpea.org). Thirty-five leadership professors also represented the Florida Association of Professors of Educational Leadership Association (FAPEL).

Deviation from Kiewra–Creswell study. Unlike Kiewra and Creswell (2000), who generated a list of names based on AERA’s Division C (Learning and Instruction) membership, I did not preselect scholars to be rated. I strove to avoid tying the results to particular associations and their “star” leaders, which could have limited and even biased the data. And I did not want to presume what “living scholar” might mean to others, so I avoided defining this term. Those respondents who forwarded the names of two exceptional scholars, explaining their reasoning for this decision, had both votes counted. Also, in contrast with Kiewra and Creswell’s focus on cognition and learning for their survey and recipient pool, my own form provided no premapping or compartmentalization relative to educational leadership. Reasons for selecting any particular area, such as supervision or policy, seemed arbitrary, serving only to privilege one at the expense of another. I opted for openness, hoping this would promote greater inclusion or representation of the educational leadership field and hence provide a rich data set of interest to readers.

Finally, the Kiewra and Creswell results were based on 41 (out of 113) responses. The 233 I received from nominators also compares with the 105 that Murphy (1999) collected.

Saturation and data analysis. The scholars recognized as exceptional in this study achieved this status once the data (i.e., votes cast and reasons provided) revealed clear patterns. Also, the response data (reasons given for selections) were coded, and key words and phrases highlighted, in search of potential themes, applying Miles and Huberman’s (1994) qualitative procedures for data management and analysis.

Discussion of Survey Results

In supplying the reason(s) for their choice, respondents generally emphasized areas of importance, specific contributions, and lines of work, even areas that personally influenced their own scholarship. Others noted publications and further scholarly contributions.

Four scholars have been identified as exceptional in this article. Although many more names were forwarded, some with obvious and consistent support by the voting body, a demarcation became evident based on the sheer number of tallies over time. A top tier consisting of four individuals had emerged in addition to a second and third tier.

Those finalists identified here are all white males. A number of females and a few minorities were nominated (and some were nominators), but not to the point of selection. Although there has been a dramatic increase of women in educational leadership, school leadership certification programs, and leadership positions in national level associations (e.g., UCEA, NCPEA, and AERA), male scholars may be publishing more frequently (Engstrom, 1999). As McCarthy (1999) speculated, women as a group have been faculty in this field for fewer years than men, and so have “not yet hit their stride” in terms of scholarly productivity and impact (p. 202). As another possibility, male scholars may be receiving greater recognition for works disseminated and scholarly efforts made. The same trend probably applies to minority scholars. Obviously, such trends and possibilities should be closely examined in the context of equity, ethics, and policy development in academe, as recommended by Haring (1998) and other scholars.

Different Readings of the Question

The survey question was interpreted in various ways. While some viewed it from a national policy perspective, others considered contribution to the scholarly field, schools, or novice leaders.

Selection Criteria and Reasons

The criteria of selection and reasons given were numerous; these are summarized in Table 1. The entries (appearing on the left) represent the themes that emerged from the response data and each quote (on the right) provides support from two or more respondents for the corresponding theme; these all characterize the “typical” comments received.

The four scholars selected as leading academics do not evenly fulfill all of the criteria listed, perhaps because each is known for particular spheres of influence (e.g., leadership standards, administrator program preparation reform, ethics and moral leadership, K–12 institutional partnership). Nonetheless, all were associated with such dimensions as significant and broad impact on scholarship and the field, national spheres of public influence, and mentoring and multi-authoring systems. The reasons provided extend beyond the traditional association with publication rates and impact via frequency of citation as markers of scholarly excellence. The proliferating criteria also fit with some of the emerging contemporary trends in our field, such as the increase since 1986 of university faculty committing to improving relationships with schools and practitioners, developing field-based components in preparation programs, and focusing more on ethics in professional practice (McCarthy, 1999; Mullen, Gordon, Greenlee, & Anderson, 2002).

However, the recurring reasons that were forwarded virtually bypassed contributions in the areas of diversity and social justice as well as alternative paradigms, such as feminism, critical theory, and postmodernism. Perhaps these and other philosophically critical locations will emerge in a more exhaustive sampling or a future one. A critical reader of a draft version of this article asserted that the results “reflect a chasm in the field, which is still very traditional while moving ahead.”

Going Wide/Deep as Reformers

Those who functioned broadly in their work and impact received more tallies within the discipline than scholars who functioned more narrowly. This pattern suggests that those receiving an abundance of votes were perceived as having a higher value. However, those who have made significant inroads in an educational leadership domain, such as administrator program preparation reform, were simultaneously associated with specific change agendas. Going wide/deep was a salient pattern, then, associated with “living legend.” Generally speaking, nominees had appeared to construct their own meaning of “exceptional,” seeing this as a comprehensive effort closely related to particular reform agendas. Fullan (1999) explains that “large scaleness” is only possible where human contact has been fully established and a “multilevel system” has been managed on a “continuous basis” (p. 74). Although Fullan was addressing large-scale reforms per se rather than particular reformers, these can be linked as I have done in this discussion.

Tensions in the Data Analysis

As is evident from Table 1, the results proved productive for identifying criteria that some academics currently associate with outstanding scholarship. In contrast, a minority (19 individuals) offered powerful insights into why nomination was simply not feasible to them. For a few, the very use of “our field” in the survey question was problematic: “I see a problem with your question vis-à-vis your use of the label ‘our field.’” Respondents doubtless thought about the heroes in their own areas of study.

Granted, the concept of “field” is very tricky. English (2003) critiques “field” and its cousin “knowledge base” as leftovers from modernism, denying a plurality of realities, truths, and interpretations. Postmodernism brings context, human agency, and multiplicity into the foreground: Educational leadership, similar to leadership studies, incorporates “a broad range of perspectives,” from descriptive to social scientific to humanistic and drawing upon the interpretive methods of history, literature, philosophy, and education (Johnson, 1996, p. 13). Another perception is that educational leadership is changing: The intent to bridge theory with practice has created an “emerging discipline that transcends the academy precisely because it is more than mere scholarship; it is scholarship plus” (Born, 1996, p. 47).

The hybrid or “borrowing” nature of educational administration has produced a composite field, arousing concern. Someone shared, “I’ve been deeply troubled by the many contradictions between American democratic ideals and the theories and notions borrowed from business, the military, and the social sciences being subsumed within the field of education administration with little scrutiny.” Because of the increasingly amorphous nature of educational leadership, another argued in favor of actually creating a “field” that has boundaries and a distinct identity: “Without clear, substantive differences from other academic departments, educational administration as a field of serious scholarly inquiry has no legitimate grounds on which to defend its continued existence within academe, particularly while higher education is being downsized.”

For still others, the use of “living scholar” understandably incited confusion and controversy, as thoughtfully articulated: “In good conscience I must say that I find the term living scholar something of an oxymoron. My definition may be out of style these days, but I believe before one can be considered a scholar that person’s writings or orations must have withstood the tests of time.” Interestingly, this decision evoked “considerable difficulty. Every time I visit your email the same questions prevent an answer—does ‘greatest living’ mean most frequently cited by other scholars? Most followed by practitioners? Contributed the greatest theoretical insights about leadership?

Done most to redirect the field of study? Or, added most to the empirical base for understanding leadership?”

My reaction to all of these pivotal concerns is that while the survey question is laden with datable, slippery concepts (i.e., “the field,” “living scholar,” “exceptional,” and even “educational leadership”), so is the profession itself. Further, the question solicited valuable information—it is useful to see the multiple, disjointed, and even contradictory viewpoints taken. Areas of consensus also surfaced from this mixed response, as captured in Table 1. Accounting for the feedback of non-voting members as I have done here has made visible issues of contention. Those who provided critiques about the nominating process and suggestively about its validity performed a probing hermeneutic deconstruction that was treated as data and thematically analyzed, with some attention given here.

Contextual Influences and Background Issues

One crucial insight of survey respondents was, “Who is outstanding in educational leadership and administration or any scholarly field is really framed by the times and the needs.” In many respects, this resounding message has greater worth than the criteria and even the participants’ selections. Certainly, context matters, a reality that keeps the idea of “living legend” and practice of hero-worshipping in perspective. This admission of temporality and contextuality contrasts with the view that the living legend “finalists” represent a static, noncontroversial choice.

The results, inevitably debatable from almost any angle, are also influenced by the methods I have selected and the venues surveyed. Regardless of my attempt to appraise the educational leadership field as comprehensively as possible, a disjointedly configured domain required piecemeal, “pick and shovel” sampling. Because no single “repository” exists to which all leadership professors belong, it is currently not possible to communicate with the complete constituency and at one time. Such systemic barriers make it clear that any such study should not be construed as the last word on the subject of exceptional scholarship.

In addition to systemic barriers to data collection, other contextual issues included political alliances, decision-making challenges, and generational biases. Some scholarly communities hold tight allegiances, making it difficult to know the extent to which nominations were influenced by loyalty rather than informed judgment. In a few instances, junior professors “confessed” that they had nominated their former major professors. Perhaps more exhaustive sampling procedures would have better monitored the influence of political entanglements; on the other hand, these seem inherent in the psyche of any discipline. Other contextual influences underscore how challenging it proved for some respondents to make a single selection. This struggle emphasizes just how demanding this decision-making process can be as well as—this is the good news—the high number of outstanding leaders from which to select. A few participants even postulated that no such scholars currently exist, except as experts within their own domain. But most persons did provide a nomination, even where disclaimers had been announced, an admission that supports the contentious notion that leading scholars for contemporary times can in fact be identified, even where tensions and uncertainty are embedded in the conclusions and where debate is inevitable and ongoing.

As mentioned, a generational bias entered into the results but once again the degree of influence is unknown. More senior professors generally know the older or more established generation of scholars, whereas those younger have familiarity with the newer “stars.” Several nominees addressed this phenomenon, as in: “The more scholarly respondents might tend to select someone who is treasured within their field for the writing they have done. Personally, I am not as up-to-date with names because my own heroes are mostly retirees; in fact, I am ignorant of the mid-career hotshots who are making good waves.”

Final Remark

The issues of complexity raised herein suggest that while nominations of “living legends” seem possible, especially where constituents have formulated criteria, deeper issues prevail. Nominators forwarded useful and revealing criteria supporting their decision making—a process thoughtfully undertaken, particularly by those sharing reflections and caveats. And the dissenting critiques proved invaluable as well. Nominators provided clues about the patterns of educational leadership they most value, the individuals to whom they have looked for guidance, the status of the field, and emergent trends.

Further research is needed that continues work on the controversial topic of exceptional scholarship in educational leadership. Debate is also encouraged about the topics of significance raised: The critical tensions explored herein that capture the thinking of some university faculty in addition to the self-identifying criteria for the votes cast would benefit from a community-wide response.

References

Born, D. (1996). “Leadership studies”: A critical appraisal. In P. S. Temes (Ed.), Teaching leadership: Essays in theory and practice (pp. 45-72). New York: Peter Lang.

Culbertson, J. (1995). Building bridges: UCEA’s first two decades. University Park, PA: The University Council for Educational Administration.

Diem, K. G. (2002). Maximizing response rate and controlling nonresponse error in survey research. New Brunswick, NJ: Rutgers Cooperative Extension/Resource Center Services. [Online]. Available: http://www.rce.rutgers.edu.

English, F. W. (2003). The postmodern challenge to the theory and practice of educational administration. Springfield, IL: Charles C. Thomas Publisher.

Engstrom, C. M. (1999). Promoting the scholarly writing of female doctoral students in higher education and student affairs program. NASPA Journal, 36(4), 264-277.

Fraze, S., Hardin, K., Brashears, T., Smith, J. H., & Lockaby, J. (2003). The effects of delivery mode upon survey response rate and perceived attitudes of Texas agri-science teachers. Journal of agricultural education, 44 (part 2), 27-37.

Fullan, M. (1999). Change forces: The sequel. London: Falmer.

Haring, M. J. (1998). Response to “A woman's name: Implications for publication, citation, and tenure.” Educational Researcher, 27(8), 43.

Johnson, P. F. (1996). Antipodes: Plato, Nietzsche, and the moral dimension of leadership. In P. S. Temes (Ed.), Teaching leadership: Essays in theory and practice (pp. 13-44). New York: Peter Lang.

Kiewra, K. A., & Creswell, J. W. (2000). Conversations with three highly productive educational psychologists: Richard Anderson, Richard Mayer, and Michael Pressley. Educational Psychology Review, 12(1), 135-161.

McCarthy, M. M. (1999). The “changing” face of the educational leadership professoriate. In J. Murphy & P. B. Forsyth (Eds.), Educational Administration: A decade of reform (pp. 192-214). Thousand Oaks, CA: Corwin.

Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.

Mullen, C. A., Gordon, S. P., Greenlee, B., & Anderson, R. H. (2002). Capacities for school leadership: Emerging trends in the literature. International Journal of Educational Reform, 11(2), 158-198.

Murphy, J. (1999). The reform of the profession: A self-portrait. In J. Murphy & P. B. Forsyth (Eds.), Educational Administration: A decade of reform (pp. 39-68). Thousand Oaks, CA: Corwin.

Newman, M. E. (2002). ‘Rounding up’ responses to mailed questionnaires. [American Evaluation Association/Extension Education Evaluation]. [Online]. Available: http://danr.ucop.edu/eee-aea/AEA_HearItFromTheBoardJuly2002.pdf

Renzetti, C. M., & Lee, R. M. (Eds.). (1993). Researching sensitive topics. London: Sage.

Author Notes

The respondent quotes appearing on this chart (and in this article) have been synthesized and slightly altered, rendered gender-neutral where possible and anonymous, protecting both the nominees and the nominators. I am grateful to the faculty nominators who generously shared their perceptions. Also, I appreciate the helpful critique provided by the editor and the two reviewers.

Acronyms for national standards used by nominators:

Educational Leadership Constituent Council (ELCC); Interstate School Leaders Licensure Consortium (ISLLC; National Council for Accreditation of Teacher Education (NCATE)