The pursuit of a rigorous, scientific basis for education in reaction to perceived waste and mismanagement is not a new phenomenon. At the turn of the century, for example, John Franklin Bobbitt espoused a scientific management approach, and David Snedden and Werett W. Charters extended the idea further to propose a scientific basis for curriculum development (Kliebard, 1979). In the 1950s and 1960s, education became firmly rooted in the science of behaviorism which sought to remove subjectivity from education and focus instead on task analysis, schedules of reinforcement, and observable behaviors. “Teaching machines” made their way into the classroom and instruction became increasingly programmed (Hunt, 1993). It was thought, during these eras, that education was finally becoming a science in its own right- that it need no longer be considered an art. A similar shift has also taken place in the field of medicine in which training and practice have moved from an apprentice-based art to an evidence-based science.

Similar to the epistemological foundation of education, that of modern medicine has historically rested on authority, tradition, theory, consensus within the field, and the individual practitioner’s experience more than upon scientific research. This has not been due to a lack of research per se, but to a gap between research and practice in both medicine and education. However, in the early1990s, a systematic process of collecting and integrating scientific evidence for medical practice was developed at McMaster University in Canada and labeled “evidence-based medicine” or EBM. Stated simply, EBM is the integration of the best research evidence with clinical expertise and patient values” (Sackett et al., 2000, p. 3). And just as the practice of medicine is thought to have been transformed by science, there are those who believe that the practice of education should be “dragged, kicking and screaming, into the 20th century” (Slavin, 2002, p. 2). In fact, third pillar of the No Child Left Behind Act (NCLB) - identification and selective funding of scientifically proven educational methods (USDE, 2001) - has spawned the development of evidence-based education (EBE), based closely on the EBM. Thus, EBE has been defined as, “The integration of professional wisdom with the best available evidence in making decisions about how to deliver instruction” (Whitehurst, 2002). And just as the “gold standard” of rigorous scientific methodology in medical research is considered to be the randomized clinical trial, so too is randomization or true experiment epitomized as the sine qua non of evidence upon which the practice of education should be based (Beghetto, 2003; Reyna, 2002; Slavin, 2002; USDE, 2003; Whitehurst, 2002). However, at least two questions arise regarding the similarities between medical practice and educational policy: (1) Are randomized controlled trials the gold standard in educational research- and should they be? (2) Can effectiveness of educational practice and programming be understood in the same way the effectiveness of medical interventions and treatments are understood? Each of these questions will be addressed below.

Randomized Clinical Trials

The use of randomized clinical trials (RCTs) in medicine has grown substantially in the past 50 years, gradually replacing patient anecdotal reports as the primary basis for clinical decision-making (Fisher, 1991) and contributing to the knowledge base that informs the practice of EBM (Sackett, et. al, 2000). Evidence supporting a specific medical treatment is built and validated over time through dissemination of meta-analyses, a process through which results of independent experimental research studies investigating a similar or related research question are systematically analyzed. There are numerous examples of common medical treatments that, once subjected to randomized clinical trials, were found to be ineffective or even harmful to patients, such as hormone replacement therapy to reduce the risk of coronary heart disease in post-menopausal women and the use of eye patches in the treatment of corneal abrasions. These examples are often invoked as a rationale for similar a similar approach to the evaluation of educational practice (Rayna, 2002; Raudenbush, 2002; Whitehurst, 2002, 2003).

Despite the advantages, however, randomized trials are not without limitations (Morrison, 2002). While randomization provides a degree of control that other research designs do not, there are several assumptions that must be made. First randomization achieves only a statistical probability that the two groups are identical in every respect except the variable(s) of interest. Researchers must also make assumptions as to the fidelity of the treatment or intervention of interest- that it was effectively and faithfully implemented as intended and that participants in the control group were not exposed in some way to a similar treatment or intervention outside the scope of the research study. Systematic attrition from the study, particularly from participants in the treatment group, can also undermine the comparability of groups and compromise the efficacy of made by researchers. Furthermore, the ethics of subjecting humans to potentially harmful or ineffective treatments as well as withholding potentially beneficial treatments must be considered. While these assumptions are important to consider when undertaking and interpreting medical research, they are particularly important to keep in mind when interpreting educational research.

Evidence of Effectiveness: Overlooked Distinctions between Medicine and Education

The ubiquitous references to EBM in the demand for a similarly scientific approach to the practice of education (Beghetto, 2003; Reyna, 2002; Slavin, 2002; USDE, 2003; Whitehurst, 2002) reflect two contestable assumptions: that the use of evidence in medicine has been whole-heartedly embraced by its practitioners and has vastly improved patient outcomes, and that there are significant and relevant parallels between the practice of medicine and education. In fact, EBM has been criticized by many medical practitioners as being reductionistic in its approach to medical practice, paternalistic in its treatment of patients, restricting and elevating certain kinds of evidence, producing elusive or unstable “truths,” neglecting attention to the individual in favor of abstract populations, and being contemptuous of the wisdom and integrity of predecessors of EBM (Little, 2003). In addition, the medical examples commonly invoked as a rationale for similar approaches to the practice of education imply that the use of evidence in clinical decision-making is simple and straightforward. In fact, EBM involves much more than randomized clinical trials. It has evolved into a complex, well-structured sub-discipline with specific processes for analyzing and responding appropriately to questions of diagnosis, therapy, prognosis, harm, and cost-benefit analysis. Yet, despite the relatively well developed structure of EBM, many practitioners find it difficult and impractical to locate and evaluate relevant patient-oriented research evidence when it is needed for clinical decision-making. Others argue that the contradictory findings across independent studies when no systematic review exists only serves to complicate and confuse decisions about treatments for individual patients and, conversely, that meta-analyses often mask contradictory effects and outcomes and fail to consider differences in population samples, treatment fidelity, and measurement techniques (Abelson, 1997) .

These criticisms of EBM are easily applicable to educational practice as well. But there is more to be concerned about. Whereas EBM seeks to balance external evidence with practitioner judgment and patient values at the level of the individual, this is not necessarily the case in education, where “treatment” rarely involves an individual student and teacher. Rather, educational prescriptions are sought for entire classrooms, schools, districts, and even states. Educational practice based entirely on external evidence and rigorous science will thus necessarily neglect to consider efficacy of the method for at least some individuals within the population. Perhaps of even more concern, however, are the important distinctions between medicine and education. Whereas the practice of medicine has as its clear goal the maintenance of health and elimination of disease, education has no such clearly stated goal. The purpose of education has been identified variously as cultivation of the intellect, transmission of western cultural heritage, perpetuation of a democratic citizenry, preparation for future vocational and occupational work, development of caring and compassionate humans, empowerment of marginalized groups, and motivation for life-long learning (Ozmon & Craver, 2003; Schubert, 2002).

Thus, while evidence based on randomized clinical trials may inform a medical practitioner about the relative risk of hormone-replacement therapy for a patient given her specific medical history and relevant lifestyle factors, similar conclusions cannot be drawn from evidence obtained through randomized field trials about the efficacy of a prescribed educational method for individual or groups of children. Medical treatments typically have a relatively well-known physiological process by which they work to reduce or eliminate specific disease or symptoms, and outcomes are readily identified; heart disease or breast cancer subsequent to HRT can be clearly diagnosed using the appropriate tools. The same cannot be said of educational outcomes. For example, Whitehurst (2003) cites a randomized controlled trial of Even Start- a federal program designed to improve the literacy of disadvantage families. Results of the study indicated that there were no significant differences between children in the program and those in the control group on the Peabody Picture Vocabulary Test and the Pre-School Inventory. Thus, the researchers concluded that the program had no effect on improving the school readiness of the participating children at the 18-month follow-up. However, one wonders if the assessment measures used in the research study are the best – or even appropriate- indicators of literacy and of school readiness. To put it in EBM terms, what was the sensitivity and specificity of the diagnostic measure (Sackett, 2000)? There may have been improvements in literacy and school readiness not assessed by those measures such as increased interest in listening to stories and looking at picture books, improved self-efficacy with regard to succeeding in school, and improved interpersonal skills with other children and adults. One must also consider whether there were other hidden or unanticipated costs and benefits not taken into consideration (Posovac & Carey, 2003).

Furthermore, specific medical interventions often do not vary considerably from physician to physician. The placement of an eye patch or dosage of hormones can remain consistent across physicians. And while patient cooperation in treatment is essential, the treatment works primarily upon the patient’s physiological being. The same cannot be said of educational interventions, and more importantly, neither should it. Learning is an active, constructive process (Beilin,1992; Case, 1992; Illerich, 2003) that depends extensively on the interactions between teachers, learners, and the environment, it is mediated by myriad socio-historical and contextual factors, and teachers bring to the instructional process their own unique set of skills, experiences, and personalities. Education is not something that is “done to” students the same way a patient undergoes medical treatment. Instead, it is a process of becoming (Marshall & Zohar, 1997) that manifests itself in many subtle ways over the course of time.

Conclusion

It is important for educators to look critically at educational programs and practice in order to improve student learning and ensure effective utilization of available resources. However, we must remain cautious and recognize that to seek “proven” educational methods represents a misunderstanding of the appropriate integration of evidence from scientific research into educational policy and practice; cardinal principles of evidence would preclude such a statement. As Kerlinger (1986) succinctly points out:

Let us flatly assert that nothing can be “proved” scientifically. Proof is a deductive matter, and experimental methods of inquiry are not methods of proof. They are controlled methods of bringing evidence to bear on the probable truth or falsity of relational propositions. In short, no scientific investigation ever proves anything. Thus, the interpretation of the analysis of research data can never use the term proof (p. 145).

Furthermore, the idea that randomized controlled trials are or should be the gold standard in educational research fails to consider the range of methods of inquiry appropriate to education and the complex web of “evidence” that should be brought to bear in identifying effective educational methods and programs. Edmund Husserl (1965) asserts a caution that is particularly relevant to the discussion at hand:

The confusion here comes from the supposition that there must be absolute uniformity of method in sciences and that the method is that of the empirical sciences. True science, on the other hand, conforms itself to its objects, regardless of methodological prejudices based on the success of one method in other scientific endeavors (p. 11).

Just as the developers of EBM sought to develop a structure and method of science that was uniquely suited for and could be appropriately integrated into the art of medical practice, educators must look beyond a narrow definition of rigorous science and allow for multiple methods of inquiry that are uniquely suited to and can be readily integrated into the art that is education.

References

Abelson, R. P. (1997). A retrospective on the significance test ban on 1999: Ifthere were no significance tests, they would be invented. In L. L. Harlow, S. A. Mulaik, and J. H. Steiger, (Eds.), What if there were no significance tests? (pp. 117-141). Mahwah, NJ: Lawrence Erlbaum Associates.

Beghetto, R. (2003). Scientifically based research. Eugene, OR: ERIC Clearinghouse on Educational Management. (ERIC Document Reproduction Service No. ED747304)

Beilin, H. (1992). Piagetian theory. In R. Vasta (Ed.), Six theories of child development: revised formulations and current issues (pp. 85-132). London: Jessica Kingsley Publishers.

Case, R. (1992). The mind’s staircase. New Jersey: Lawrence Erlbaum Associates.

Fisher, B. (1991). The importance of clinical trials. News from the Commission on Cancer of the American College of Surgeons, 2(2). Retrieved on March 21, 2004 from http://www.asri.edu/bfisher/clinical_trials.html.

Hunt, M. (1993). The story of psychology. New York: Random House.

Husserl, E. (1965). Phenomenology and the crisis of philosophy (Q. Lauer, Trans.). New York, NY: Harper & Row Publishers. (Original work published 1911).

Illeris, K. (2003). Towards a contemporary and comprehensive theory of learning. International Journal of Lifelong Learning, 22(4), 398-406.

Kerlinger, F. N. (1986). Foundations of behavioral research. Orlando, FL: Harcourt Brace Jovanovich Publishers.

Kliebard, H. M (1979). The drive for curriculum change in the United States, 1890-1920. Part I – The ideological roots of curriculum as a field of specialization. In J. R. Gress (Ed.), Curriculum: Frameworks, criticism, theory. Richmond, CA: McCutchan Publishing.

Little, M. (2003). Better than numbers…a gentle critique of evidence-based medicine. ANZ Journal of Surgery, 73(4), 177-182.

Marshall, I. & Zohar, D. (1997). Who’s Afraid of Schrodinger’s Cat? All the New Science Ideas You Need to Keep Up with the New Thinking. New York: Morrow & Co.

Morrison, K. (2001). Randomised controlled trials for evidence-based education: Some problems in judging ‘what works.’ Evaluation and Research in Education, 15 (2), 69-83.

Ozmon, H. A., & Craver, S. M. (2003). Philosophical foundations of education. Upper Saddle River, NJ: Merrill Prentice Hall.

Posovac, E. J. and Carey, R. G. (2003). Program evaluation: Methods and case studies, 6th edition. Upper Saddle River, NJ: Prentice Hall.

Raudenbush, S. (2002). Research. Presentation to the Working Group Conference of the U. S. Department of Education on The Use of Scientifically Based Research in Education. Transcript retrieved on January 15, 2005 from http://www.ed.gov/nclb/methods/whatworks/research/page_pg3.html.

Rayna, V. (2002, February). What is scientifically-based evidence? What is logic? Presentation to the Working Group Conference of the U. S. Department of Education on The Use of Scientifically Based Research in Education. Transcript retrieved on January 15, 2005 from http://www.ed.gov/nclb/methods/whatworks/research/page_pg3.html.

Sackett, D. L., Straus, S. E., Richardson, W. S., Rosenberg, W., & Haynes, R. B. (2000). Evidence-based medicine: How to practise and teach EBM (2nd edition). Edinburgh: Churchill-Livingstone Press.

Slavin, R. (2002). Evidence-based education policies: Transforming educational practice and research. Educational Researcher, 31(7), 15-21.

Schubert, W. H. (2002). The question of worth as central to curricular empowerment. In J. R. Gress (Ed.), Curriculum: Frameworks, criticism, and theory. Richmond, CA: McCutchan Publishing.

U. S. Department of Education (2001). NCLB Overview. Retrieved on January 24, 2005 from www.ed.gov/nclb/overview/intro.

Whitehurst, G. J. (2002). Evidence-Based Education (EBE). Retrieved on January 24, 2005 from http://www.ed.gov/nclb/methods/whatworks/eb/edlite-index.html.

Footer

This work is licensed by Eileen Johnson under a Creative Commons Attribution License (CC-BY 2.0), and is an Open Educational Resource.

Last edited by National Council of Professors of Educational Administration on Jan 17, 2008 1:54 pm US/Central.