The Academy’s Zeitgeist—Standards of Scientific Investigation:Exploring the Impact of Scholarly Work 1.1 2006/03/02 12:45:57.612 US/Central 2006/03/09 11:12:43.148 US/Central Carol A. Mullen cmullen@coedu.usf.edu Janice R. Fauske jfauske@sar.usf.edu Carol A. Mullen cmullen@coedu.usf.edu Janice R. Fauske jfauske@sar.usf.edu educational administration educational reform scholarly publication scientific inquiry standards The academy’s zeitgeist—standards of scientific investigation—has recently come to the fore in the national arena as the dominant moral and intellectual framework for educational research. In this article, we explore the re-emergence of standards of scientific investigation as a significant shaping force in education and the scholarly culture, particularly in regard to the fields of leadership and administration. With the recent advent of politically based decrees of quality defined exclusively by traditional standards, alternative approaches to exploring human issues, however rigorous they might be in the qualitative realm, tend to be marginalized. Traditional, experimental studies that involve large-scale statistical research design and randomization have been authorized, making single-subject research, naturalistic inquiry, self-study, and other qualitative research practices unlikely candidates for federal funding. For this discussion on “authorized” and “unauthorized” perspectives of research, we explore the impact of regulatory practices within the academy. Note: This MODULE has been peer-reviewed, accepted, and sanctioned by the National Council of Professors of Educational Administration (NCPEA) as a scholarly contribution to the knowledge base in educational administration. Defining the Problem The U.S. federal government has recently codified standards for scientific investigation. Leading initiatives feature the National Research Council’s (2002) publication Scientific Research in Education, as well as the No Child Left Behind Act (NCLB) of 2001 (U.S. Department of Education, 2004). Such seminal works decidedly favor traditional, experimental scientific inquiry in educational research, policy, and practice, radically narrowing the scope of what counts as quality and rigor. This is particularly problematic in the social science disciplines, where the exploration of human behavior dominates research agendas. While difficulty in quantifying human behavior, interaction, and perception within educational settings is historically well documented, the enhancement of our collective theoretical and practical knowledge in the social sciences nonetheless continues to be highly valued (Becher, 1989). We argue, then, that the new governmental publications’ standards for educational research have restricted the kinds of inquiry that are viewed as legitimate in our field. Moreover, these have encouraged regression in our thinking by diminishing the importance of qualitative study and mixed method design. Consequently, educational researchers and pubic educators must dramatically increase our investment in traditional quantitative research and diversify grant-seeking strategies within this arena if we are to remain viable in seeking federal grants. As one serious effect of this trend, alternative voices and distinct methodologies are overlooked, privileging certain knowledge forms over others. A Retrospective of Scientific Inquiry in Education Favoring certain methods, foci, and forms of research is not a new phenomenon. In the past millennium, scholars in many disciplines have successfully perpetuated certitudes for controlling the scientific process and empirical investigation of a range of human circumstances (Becher, 1989; Lakatos, 1999). These scholars, characterized as modernists and structuralists, interpreted the rules of conduct for scientific investigation in an earnest attempt to assure standards and rigor in research (Feuer, Towne, & Shavelson, 2002). Gardner (1999)claims that the development of such standards and disciplines has allowed systematic investigation and discovery of knowledge that has profoundly affected as well as improved the existence of humankind. By describing and classifying, experimenting and replicating, scientific investigation and its patterns of logic and order have produced classification systems that perpetuatea view of the world that, in essence, standardize a single theory or norm (Feuer, et al., 2002). Principles of structuralism manifest, for example, in bureaucratic theory, are so pervasive in modern thought that they often go unquestioned and unacknowledged, in part because they offer comforting promises of order, organization, and certainty (Cherryholmes, 1988). In effect, the doctrine of traditional scientific investigation has been a “sanctuary” of stability to researchers across disciplines, making education one of its principal homes. In the past 30 years, postmodernists in education (e.g., English, 2003; Peters & Lankshear, 1996; St. Pierre, 2002) have challenged the prevailing modernist/structuralist views. Postmodern scholars see this worldview as promulgating a web of asymmetrical power that favors certain groups and ways of knowing. Becher (1989) describes modernist/structuralist scholars as having a tenacious hold on scientific investigation, much the same as elite designers set the standards for current fashion—regardless of whether the garments fit the wearers, the form and style are strictly dictated and widely accepted. Individual expressions and alternative styles are hence rendered unpopular and even objectionable; similarly, alternative voices and innovative research have been deemed “unauthorized” in light of trend-setting governmental publications. Some social scientists argue that the “scientific nature” of research has been debased, resulting largely from the overpowering of social and critical inquiry by conventions of scientific investigation(e.g., Eisner, 1997; Feuer, et al., 2002). The ensuing tension has been exacerbated by the definitions of research appearing in highly influential governmental works. Reactions have ranged from skepticism (Berliner, 2002), to critique (English, 2004), to fear (St. Pierre, 2002). Controversy will likely become even more vehement with the new federal pronouncements of what “counts” as legitimate inquiry in education. Calls for conceptual diversity in educational scholarship (Eisner, 1999;English, 2004) embody the growing unease with empiricism, modernism, and structuralism as paradigms restricting educational scholarship. Influential Governmental Publications First, regarding the NRC report (2002), scholars in science, engineering, and medicine serving on the Committee on Scientific Principles for Education Research created a “lighthouse” model that is expected to guide the work of educational researchers. The committee sought, in its own words, to depict “what constitutes good science” and “‘scientifically based’ education research for the policy communities [committed to] improving education policy and practice” (National Research Council, 2002, Foreword, p. vii). The NRC Committee’s report recommended six “scientific principles” as standards of quality for research: (1) Pose significant questions that can be investigated empirically; (2) Link research to relevant theory; (3) Use methods that permit direct investigation of the question; (4) Provide a coherent and explicit chain of reasoning; (5) Replicate and generalize across studies, and (6) Disclose research to encourage professional scrutiny and critique (pp. 3-5). The language chosen by the NRC does, in fact, encourage the selection of methods appropriate to the research question and does not specifically exclude qualitative research. However, these criteria have been formulated as rules for acceptable scientific investigation without recognizing the constructed and subjective nature of their very making, in addition to the conformist, if not coercive, effects of their enforcement (English, 2004). We believe it is problematic that a group of mostly noneducators had been charged with creating perspectives for the educational research community. English (2004) further argues that the mindset at work in the NRC report is teleology, the doctrine of final causes. Teleologymeans that not only causation but also ultimate purposes (e.g., the common good) drive an individual or culture (New Webster’s Dictionary and Thesaurus of the English Language, 1993, Lexicon Publications). Such a stance promotes noncritical or self-affirming thinking about the value systems inherent in scientific standards for quality research. English’s (2004) deconstruction of the NRC’s premises highlights its alliance with logical empiricism: The model itself establishes nonnegotiable rules for scientific inquiry, governing, in a nutshell, issues of significance, coherence, and replication in the development and implementation of studies. Similarly, the language of the NCLB supports only one view of research activity—that it is to be quantitatively based and that it will satisfy the “measurable objectives” outlined in the legislation. Whether in the context of applications from state agencies for school–community partnerships, teacher recruitment, the professional development of early childhood educators, or another related context, the “strategies and activities” proposed are to be “based on measurable objectives” and explainedrelative to “student academic achievement” (see e.g., subpart 4 (B) and (2)). The Act decidedly leans toward traditional, large-scale quantifiable methods, which means that, without vigilance and activism, qualitative, postmodern, and other alternative methods and ways of knowing will henceforth be seen as even less credible and relevant. Appropriate qualitative research methodologies fall outside the expectations for governmentally funded research. With its tendency toward qualitative method that is often criticized as less rigorous, educational research as a whole has been accused of sliding down an already tilted slope away from traditional science inquiry (Erickson & Guttierez, 2002). The debate centerson two issues—favoring quantitative over qualitative research and maintaining rigor regardless of method. Because of the widely held perception of declining standards in research, various groups have beenprompted to re-establish criteria for judging the efficacy of research. Thompson (2002) is among those who have observed a spreading distaste for discussions of methodology at the AERA convention where the “standardless” paradigm of “proof by assertion” appears to have silenced more rigorous forms of data analysis. Though educational research groups have not been systematically included in discussions with the National Research Council, AERA has spoken out on the issue. This leading professional association supports the value of increased quality in educational research, but from a perspective that is inclusive of a variety of methods. The AERA Executive Council’s (2003) Resolution on the Essential Elements of Scientifically Based Research asserts “that there are multiple components of quality research, including well-specified theory, sound problem formulation, reliance on appropriate research designs and methods, and integrity in the conduct of research and the communication of research findings” (p. 1). The resolution further states that “a fundamental premise of scientific inquiry is that research questions should guide the selection of inquiry methods.” The AERA Council categorizes randomized trials as only one sound methodology for conducting research and expresses “dismay” that the U.S. Department of Education has jeopardized other scientific methods and their usefulness by focusing on this “one tool of science” (p. 1). Hence, the standards debate raises several critical questions briefly explored here. First, we ask, how are emergent scholars socialized into the educational research process? Second, we wonder, has the peer review process served emergent scholars and alternative voices in their quest to be heard? Finally, we express our concern that the government-led view of qualitative based research as inadequate for producing valuable scientific data could have a serious impact on the new generation of scholars within the academy and, without doubt, on funded research within public schools. Implications for Emergent Scholars An essential component of effectively conducting quality educational research is dissemination of the findings and implications to appropriate audiences. In the academy, this dissemination regularly involves publishing in scholarly journalsthat are “refereed.” The influence of prevailing notions regarding what constitutes high quality research can both shape and limit the scope of funding, as well as what is ultimately accepted for dissemination to educators. Thus, the longitudinal impact of political–bureaucratic influences on scholarly endeavors should not be underestimated. Following, we discuss certain patterns in publication as related to emergent scholars and critique the peer review process itself. The concern over excluded voices, a constant problem deserving ongoing vigilance within the social sciences, has been exacerbated with the new political–bureaucratic trends in educational research. Postmodernists reject the singular vision of reality perpetuated by metanarratives, including those handed down by influential decision makers. Postmodernists oppose the damage this does to traditionally disenfranchised groups, including the marginalization of their knowledge and histories by “official narratives” (Peters & Lankshear, 1996). Standardized research reports and perspectives on scientific inquiry negate storytelling accounts of local events and daily experiences, rendering “counternarratives” and “counterpractices” deviant. The voices in education that are either excluded altogether or marginalized as “unscientific” are often newcomers, persons of color, women, disabled persons, nonspeakers of English, and international citizens. The proliferation of diversity-focused movements (e.g., The Holmes Scholars), committees (e.g., UCEA’s and especially AERA’s committees and Special Interest Groups on social justice, gender equity, scholars of color, and international relations), and new journals (e.g., Journal of Latino-Latin American Studies) attests to the continuing effort to liberate “unheard” voices within powerful sociopolitical contexts. A strong commitment to diversitywithin and across educational communities inclusive of traditionally underrepresented populations depends on the support of different forms of inquiry. A deep, cultural shift in the publishing culture can enable social justice commitments and intellectual freedom agendas to thrive in higher education (Mullen, 2003). This epistemological view of reality breaks with “the underlying assumptions of modernity” and “rejects the idea of differentiation based on order and hierarchy” (English, 2003, p. 42). Indeed, as Larson and Ovando (2001) urge, by questioning and changing “the received logics of our time,” educators can reach beyond the borders of personal experience (p. 2). Barriers to Scholarly Publication Focusing on the unheard in educational research raises concerns of whether peer review can provide an opportunity for mentoring within a context that perpetuates the status quo. As a related topic, we wonder why writing for publication has generally excluded women and minorities as part of their socialization. Insufficient networking and relative newness to the academy and positions of leadershipare dynamics thatcertainly figure in their experiences (Kochan & Mullen, 1999). What about the role and process of mentoring itself? Engstrom’s (1999) mentoring study of 18 prolific female scholars from 13 academic institutions revealed few stories of mentoring assistance—the women mostly attributed their accomplishments to hard-earned knowledge through trial-and-error experiences. Not surprisingly, there was no mention of editors as mentors. Similarly, but international in scope, Dinham and Scott’s (2001) surveystudyconcerning publishing support for a large sample of doctoral holders, including women, reinforcedthe need for proactivementorship (e.g., scholarly guidance, networking, and publishing interventions). Although one would naturally expect that the graduate degree would result in dissemination of the research, this proved to be the exception. This area of scholarly development constitutes, at best, an afterthought in the supervisory relationship. As many more stories and studies suggest (e.g., Kochan & Mullen, 1999), new scholars often have mostly unsatisfying experiences with publishing guidance. Mentorship within professional contexts could steer new scholars toward publication, but this still appears to be a somewhat novel idea, at least in practice.The days when assistant professors could begin their publishing careers after being hired are largely gone. A collective responsibility is necessary for helping to facilitate the scholarly endeavors of graduate students and junior faculty. Higher education institutions will improve with this goal of enhancing the same for untenured faculty (e.g., Sorcinelli, 1994). Even more novel is the idea that professional associations and journals can perform fundamental and compensatory mentoring and networking functions. Publications by beginning scholars and on critical educational topics have gradually begun to emerge in various refereed journals. But with the new tide of scientific inquiry sweeping the nation, the degree to which new scholars andpractitioners will be encouraged to experiment outside the purview of the recently endorsed scientific perspectives is questionable. Beyond this, editorial boards can purposely include new scholars and practitioners, and from a range of qualitative and quantitative domains, even dedicating entire issues to their voices—a practice that challenges elitism in the academy, or at least normative cultural mores. Sponsorship of specific groups—racially diverse individuals who represent international ideologies and places—represents a promising development in the mainstream literature, as in journal exclusives dedicated to scholars of color (e.g., Kochan & Mullen, 1999). New trends in journal publishing include not only such special issues but also calls for publishers to support traditionally disenfranchised persons in leadership roles. These social justice commitments suggest that a new movement may be afoot in the academy, as reflected in the concerted effort of some editorial teams to diversify the decision-making structure and nourish the mentoring culture of their internal operations (AERA Council of Editors, 2004). For example, the editors of Educational Researcher have announced in their mission statement that their “commitment to inclusion and diversity further extends to those who are new to the field” (Foster & Hood, 2004, p. 3). The coeditors specify that senior scholars will support “rising scholars” with review practices extended to coauthoring opportunities; significantly, they also encourage broader participation in such endeavors. However, providing such assistance to new scholars also perpetuates a form of indoctrination in the academy, ironically sustaining the status quo, if left unidentified in the mentoring process. The status quo in the publishing world is a significant barrier not only to many new scholars but also to those with experience. Fullan’s (1999) insight is that scholars and practitioners must empower themselves to create their own meanings of change as they implement reforms. It will surely become increasingly difficult to transcend the narrow prescriptions Fullan describes with the robust agenda that has been formulated to infuse the academy with a much more restrictive idea of educational research. Transformation of the academy will require a collective realization of the need for change from within the editorial community. It is essential to acknowledge that inequities and privileges, as well as cliques and invisible rules, characterize and constrain our publishing culture, and hence many of our scholarly outlets. The importance of developing vigorousnetworking connections with experienced scholars cannot be overstated. New scholars, although talented, often require mentoring assistance to become published authors, especially where controversial or countercultural topics are involved. Perils of Peer Review The role of senior faculty as reviewers and editors dovetails with a collective responsibility for mentoring new scholars as well as protecting space for alternative voices in academe. Senior scholars often regulate what studies get funded or published through the peer review process at numerous levels. Thus, the role of senior scholars in directing research agendas has tremendous influence at both higher and K–12 educational levels (Becher, 1989). In addition to peer reviewing, these senior scholars sit on landmark decision-making committees, including influential grant-funding and national committees for the review of scientific research. Such processes result in the norms and expectations for scholarship in our fields, including educational leadership and administration. Peer review evolved in the academies of the 17th and 18th century as a “system for certifying knowledge” and has become a primary professional and “social mechanism through which a discipline’s ‘experts’ maintain control over new knowledge entering the field”(Berkenkotter, 1995, p. 245). Most peer reviews coordinated by journal editors in the social sciences are “blind,” giving reviewers freedom to be candid with impunity. But efforts to ensure high quality in the ongoing production of knowledge have simultaneously become a means of social control. Isolated pockets of reviewers can exert such force as to shape research agendas for entire fields of study (Becher, 1989). The peer review process also has a related purpose of guiding emerging scholars in research and publication (Arlington, 1995; Gebhardt, 1995). However, the potential for peer review, both blind and open, to achieve this second purpose is underdeveloped. Talking to almost any author of scholarly publications will elicit anecdotesof peer review gone bad that have failed, particularly in the teaching/mentoring dimension. One such tale involves a session at the 2003 University Council of Educational Administration (UCEA) conference entitled “Discussing the Undiscussables” wherein panel members grappled with the rites of passage that junior faculty must often endure. One “undiscussable” item identified the fear thatfledgling authors have with regard to sharing their manuscripts with senior faculty, particularly those in their home departments who evaluate their progress, including tenure and promotion. While the session focused on how one university contended with such deeply entrenched emotions through proactive mentoring practices, some senior professors in the audience expressed concern that their less established colleagues had not sought out their opinions on scholarly work in progress. Several junior faculty responded that, in their own institutions, open engagement of their work could prove risky, and so they preferred a “blind” peer review process for academic journals. In addition to problems around sharing work in its early stages, many faculty have concerns about the efficacy of the blind review process. Some also question the integrity of the “blindness” standard that is to be taken on good faith (Armstrong, 1996; Burd, 1992; Campbell, 1999). With a powerful story of a conference session on professional publishing, Coates (1995) draws attention to the pretense of objectivity in reviews: [T]he editor of a university press was asked to comment on the value of peer reviews. He replied that he relied heavily on external evaluations, so much so that he often sent a manuscript to four or five reviewers before he got the kind of review that he wanted. The editor made no effort to hide the fact that, if he was very keen on a particular manuscript, he would send it to reviewers whom he expected to be sympathetic, and, if they responded negatively or in a halfhearted manner, he might well seek out additional reviews until he had an appropriate set. … Senior scholars, familiar with the reality of academic publishing, chuckled; junior scholars, still believing in the mystic of the academy, were horrified. (A40) Experienced authors and reviewers were tacitly aware of the duplicitous nature of “blind” review, whilethose less experienced underestimated the potential for manipulation of the publication process. As a consequence, the anonymity, integrity, and protection sought by junior authors may be at best, sporadic, and, at worst, a myth. Further, the partiality of the editors and reviewers toward quantitative or qualitative methods, a particular political view, or certain institutional affiliations influences what is published and, more importantly, what research is valued. In addition to editorial manipulation of the process, difficulties can ensue in assuring blind review when the number of authors writing on a given topic is limited. Reviewers are allegedly selected for their expertise in the field, and as reviewers begin to recognize one another’s or particularauthors’ work, the review process can become biased. Favorable and unfavorable reviews depend partly on the attitude of the reviewer as collegial (receptive) or competitive (combative). Unfortunately, as Coates (1995)reports, stories abound of retaliation from authors of rejected manuscripts who learn the identity of their reviewers. Even when anonymity in blind review is preserved, the politics of social control predispose reviewers toward certain research topics and manuscripts. Research can thus become aligned to a political agenda in response to availability of funding for certain research interests over others (Becher, 1989). Federal grant opportunities, for example, can influence researchers to stray from their primary research focus in order to pursue funding, potentially hindering the development of meaningful and longitudinal research emphases. In addition, reviewers and editors often define the conventions in preferred foci and method in scientific investigation. Authors who embody these predetermined interests are more likely to be published and funded on grant proposals (Becher, 1989; Fauske, 2004). Such a process can easily politicize what on the surface appears to be objective, blind review (Campbell, 1999). Innovative or challenging voices and alternative views can be marginalized, with little accountability for reviewers. Moreover, the selection of blind reviewers is problematic at several levels. Matching the appropriate reviewer to the content of the manuscript is challenging. Unlike some disciplines in the hard sciences, fields such as educational leadership and administration are not a single linguistic community but rather a collection of social science–based perspectives and approaches (Becher, 1989). Selection of critical readers for a study of teacher evaluation, for example, would require its own peculiar set of academic reviewers and perhaps review processes. Just as the academy’s gatekeeping function of the review process can limit the inclusion of new scholars, this potential mismatch between manuscript and reviewer can further restrict the new voices that are represented, with the added effect of inhibiting mentoring opportunities. Even when reviewers are inclined to exercise the second function of reviewing—mentoring emerging scholars—they have few guidelines for engaging in the process (Gebhardt, 1995; Kochan & Mullen, 1999). Authors know thefrustration of receiving reviews that are out of sync or even directly contradictory. One reviewer might describe the work as “an excellent piece of scholarship,” suggesting that it be “published expeditiously,” while another may declare that “he or she would not want to be considered a part of a discourse community in which such obfuscatory language and jargon passed for intellectual dialogue” (Berkenkotter, 1995, p. 246). Such dissension can present sticky dilemmas. Editors and authors must decide which of the reviews is more cogent, andthe former must weigh the merits of publishing a promising work that a senior scholar has scathingly reviewed. Thus, the traditional review process does little to mentor or instruct emerging scholars. Call for Reform Many scholars have called for revision of how peer review is both envisioned and conducted in publishing and review of grant proposals along several interrelated lines: (1) Established scholars may reject “new ideas and can do serious harm to scientific progress” by not encouraging innovation (Armstrong, 1996, p. b3). (2) Reviewers spend less than 6 hours per review on average, partly because the reviews are anonymous and their reputations on not on the line (Armstrong, 1996; Burd, 1994). (3) Reviews may be biased against minorities and women or controversial research topics(Burd, 1994; St. Pierre, 2002). (4) The limited ability to ensure truly blind reviews and competitiveness among those with like research agendas may produce negative assessments or idea-stealing (Chilton, 1999; Coates, 1995). (5) The intentional selection of sympathetic or antagonistic reviewers by journal editors can decidedly guarantee or inhibit publication(Coates, 1995). (6) “Personal relationships between the peer review panelists” and the authors may positively but unfairly influence decisions (Burd, 1994, p. a21). Burd (e.g., 1992, 1994) reports challenges to the peer review process for approval of grant applications at four U.S. federal agencies: National Institutes of Health, National Endowment for the Humanities, National Endowment for the Arts, and National Science Foundation. Accusations of biases and political favoritism in awarding grants from these organizations have been widely publicized. As Campbell (1999) explains, the National Institutes of Health’s response has taken the form of a rubric that reflects openness to different kinds of research. The call for reform of peer review for federal agencies has been accompanied by a parallel call for peer review for scholarly publication. Some senior scholars who have been both the reviewer and the reviewed perpetuate the content and tone of reviews they themselves have received (Gebhardt, 1995). Those whose manuscripts have been harshly assessed may in turn provide acrimonious reviews for others, just as those receiving constructive reviews may replicate that tone. Not surprisingly, authorswho have manuscripts under review value longer, more constructive feedback with a positive, constructive tone (Chilton, 1999). In response, some editors have called for a more collaborative process in which the author, after having his or her work blind reviewed, receives assistance in strengthening the work (Kochan & Mullen, 1999). This type of non-blind interaction more closely resembles the “helpful, nonthreatening way” that our classrooms are intended to operate (Armstrong, 1995, p. 250). And this form of academic coaching promotes scholarly exchange (Coates, 1995). Such collaborative views and solutions can keep intact the quality and social control functions of the peer review process. But they also set higher expectations for the full participation of senior scholars who have little formal training and few incentives for conducting reviews (Chilton, 1999). Academics receive little recognition for scholarly review, rendering this aspect of work hidden and underappreciated. Although necessary and desirable, the expanded collaborative/mentoring process would require more time, seemingly without any extrinsic reward. Postscript The debate on quality in scientific investigation continues. It will likely become even more heated as critical questions of rigor, standards, and interpretations of quality in research remain unanswered. Given the history of debate and the process of peer review as a central means of safeguarding standards of rigor, where do we in educational research circles find ourselves? Position statements that are methodologically inclusive can serve to guide as well as intensify multiple voices in educational research, yet the relative impact of grassroots and organizationally led groups in comparison to such national leaders as the National Research Council is small. Moreover, the power of educators to influence funded, recognized, and ultimately published research may be made inconsequential. Current federal trends adversely affect academic review, rigor, and mentoring while promulgating the exclusion of certain voices and the privileging of others. References AERA Council of Editors: Scheurich, J., Mullen, C. A., et al. (2004). A call for more editors of color in all education journals by some members of the 2003 AERA Council of Editors meeting. Educational Researcher, 33(1), 40. Armstrong, J. (1996, October 25). We need to rethink the editorial role of peer reviewers. The Chronicle of Higher Education,40, A18. Becher, T. (1989). Academic tribes and territories: Intellectual enquiry and the cultures of disciplines. Buckingham, UK: Open University Press. Berliner, D. (2002). Educational research: The hardest science of all. Educational Researcher, 31(8), 18-20. Burd, S. (1992, May 20). Role of NEA, NEH peer-review panels questioned.The Chronicle of Higher Education,40, A21. Burd, S. (1994, August 3). Report to Congress criticizes peer-review process at three federal agencies.The Chronicle of Higher Education,42, A26. Campbell, P. (1999, November 19). Plan to revamp NIH peer review process draws mixed assessment. The Chronicle of Higher Education,45, A40. Cherryholmes, C. (1988). Power and criticism: Poststructuralinvestigations in education. New York: Teachers College Press. Chilton, S. (1999). The good reviewer. Academe, 85(6), 54-55. Coates, K. (1995, June 23). It is time to create an open system of peer review. The Chronicle of Higher Education,41, A40. Dinham, S., & Scott, C. (2001). The experience of disseminating the results of doctoral research. Journal of Further and Higher Education, 25(1), 45-55. Eisner, E. (1997). The promise and perils of alternative forms of data representation. Educational Researcher, 26(6), 4-10. Eisner, E. (1999). Rejoinder: A response to Tom Knapp. Educational Researcher, 28(1), 19-20. English, F. W. (2003). The postmodern challenge to the theory and practice of educational administration. Springfield, IL: Charles C. Thomas Publisher. English, F. W. (2004, April). “Scientific research in education”: The institutionalization of “correct science” and the triumph of verification over discovery: Some implications for the study of educational leadership.” Paper presented at the meeting of the American Educational Research Association, San Diego, CA. Engstrom, C. M. (1999). Promoting the scholarly writing of female doctoral students in higher education and student affairs program, NASPA Journal, 36(4), 264-277. Erickson, F., & Gutierrez, K. (2002). Culture, rigor, and science in educational research. Educational Researcher, 31(8), 21-24. Feuer, M., Towne, L., & Shavelson, R. (2002). Scientific culture and educational research. Educational Researcher, 31(8),4-14. Foster, M., & Hood, S. (2004). Editorial comments. Educational Researcher, 33(1), 3. Fullan, M. (1999). Change forces: The sequel. London: Falmer Press. Gardner, H. (1999). The disciplined mind: What all students should understand. New York: Simon Schuster. Kochan, F. K., & Mullen, C. A. (1999, Fall). Guest Editors of “Opening new doors: Research by Holmes Scholars.” [Special issue]. The Professional Educator, 22(1). Lakatos, I. (1999). The methodology of scientific research programmes (pp. 46-67). In I. Lakatos, J. Worrall, & G. Currie (Eds.), Mathematics, science, and epistemology. London: Cambridge University Press. Larson, C. L., & Ovando, C. J. (2001). The color of bureaucracy: The politics of equity in multicultural school communities. Belmont, CA: Wadsworth/Thomson Learning. Mullen, C. A. (2003). Shifting the odds in the casino of academic publishing through mentorship. In F. K. Kochan & J. T. Pascarelli (Eds.), Global perspectives on mentoring: Transforming contexts, communities, and cultures (pp. 335-357). (Series: Perspectives in mentoring). Greenwich, CT: Information Age Publishing. National Research Council. (2002). Scientific research in education. (R. Shavelson & L. Towne, Eds.). Washington, DC: National Academy Press. Peters, M., & Lankshear, C. (1996). Postmodern counternarratives. In H. A. Giroux, C. Lankshear, P. McLaren, & M. Peters. Counternarratives: Cultural studies and critical pedagogies in postmodern spaces (pp. 1-39). New York: Routledge. Sorcinelli, M. D. (1994). Effective approaches to new faculty development. Journal of Counseling & Development, 72,474-479. St. Pierre, E. (2002). “Science” rejects postmodernism. Educational Researcher, 31(8), 25-27. Thompson, B. (2002, November). Featured address: Scientific standards in educational policy research. Keynote address presented at the meeting of the Florida Educational Research Association Conference, Gainsville, FL. U.S. Department of Education. (2004, January 16). No Child Left Behind Act of 2001. Washington, DC: Office of Elementary and Secondary Education. [Online]. Retrieved from http://www.ed.gov/policy/elsec/leg/esea02/107-110.pdf.