Presented in chronological order of their appearance in the literature, thirteen articles and studies examined here best represent the processes available to define and evaluate the quality of online education programs. Table 1 provides the frameworks for quality online education examined for this review.
Table 1. Existing Frameworks for Evaluating Quality of Online Education Programs

The 24 benchmarks for success in Internet-based distance education. In their report, Quality on the Line: Benchmarks for Success in Internet-Based Distance Education (2000), The Institute for Higher Education Policy (IHEP) identified 24 benchmarks for success in Internet-based distance education that were individual quality indicators chosen as absolutely essential by various respected online education leaders of higher education institutions from an original 45 indicators determined through a literature search. While the study called each indicator a benchmark, in reality, they are attributes of online education programs to indicate overall quality; they are not measureable against other institutional results. However, the study sought to prove that “distance learning can be quality learning” (Institute for Higher Education Policy, 2000, p. vii).
Considered foundational to quality distance learning, IHEP’s research categorized the 24 quality indicators into seven themes: (1) Institutional Support; (2) Course Development; (3) Teaching and Learning; (4) Course Structure; (5) Student Support; (6) Faculty Support; and (7) Evaluation and Assessment. Within the Institutional Support (1) theme, the first indicator prescribes “a documented technology plan [in place] that includes electronic security measures to ensure both quality standards and the integrity and validity of information” (IHEP, 2000, p. 2). This theme includes reliability of the technology infrastructure and assurance that support is maintained for continued growth.
The Course Development (2) theme determines if guidelines are in place for the development of quality online course materials. Online course materials should engage the learner, encourage critical thinking, and undergo periodic revision. The Teaching and Learning (3) theme stipulates that interaction must occur during the teaching and learning process between student-instructor, and student-student. Additionally, timely and constructive feedback should be provided.
The Course Structure (4) theme addresses the quality of information provided to a student prior to enrollment in an online class, such as a student readiness indicator and course objectives. Included in this theme was a provision of library resources for online students, required by all regional accrediting bodies. The Student Support (5) theme considers the kind of information students receive about the program, admission requirements, proctoring requirements, and whether all services available to traditional students are also made available to online students. It is recommended that online programs have a repository of resource materials online so that students can be successful in the program.
The Faculty Support (6) theme identifies resources provided to faculty for developing and teaching an online course. Faculty also need clear policies, a support structure, training, and mentoring. The final theme, Evaluation and Assessment (7) examines if or how the online education program is being evaluated and what policies and procedures are in place for supporting an evaluation process. This theme recommends that data on enrollment, costs, and successful/innovative uses of technology should be reviewed to evaluate program effectiveness. Learning outcomes should be assessed and evaluated for clarity and appropriateness to support continued improvement.
ACTIONS model of quality. To evaluate instructional technologies in education, Tony Bates (2000) coined the acronym ACTIONS: Access and flexibility, Costs, Teaching and learning, Interactivity and user friendliness, Organizational issues, Novelty, and Speed. Although the ACTIONS model was designed to help with the selection of instructional technologies, the model may be used to evaluate distance learning programs as each of these themes can be applied to online education programs. Bates’ ACTIONS model was one of the first to address cost factors, which affect both the institution and the student.
Best practices for electronically offered degree and certificate programs. One of the first attempts to identify and assess quality in online education was developed by the Western Cooperative for Educational Telecommunications (WCET) in 1997. A second report, developed in 2001 along with the regional accrediting bodies, Best Practices for Electronically Offered Degree and Certificate Programs, expanded the prior report into five categories instead of three: (1) Institutional Context and Commitment, (2) Curriculum and Instruction, (3) Faculty Support, (4) Student Support, and (5) Evaluation and Assessment (WCET, 2001). In the prior report, faculty support and student support were considered subsets of the institutional context and commitment category. The WCET standards developed in 2001 were not created as an evaluation instrument; the standards demonstrate how basic principles of institutional quality already in place according to accreditation boards would apply to distance learning programs (WCET, 2001). The 2001 report is still frequently cited as a guide for indicating quality within online education programs.
Eight dimensions of e-learning framework. Badrul Khan (2001) examined the critical dimensions necessary for quality learning online and found eight primary categories: institutional, management, technological, pedagogical, ethical, interface design, resource support, and evaluation. Each dimension, presented in Table 2, is integral to a systems approach for evaluating quality. According to Khan, this comprehensive model may also be used for strategic planning and program improvement.
Table 2. Khan’s Eight Dimensions of E-Learning Framework

Reprinted from Managing e-learning: Design, delivery, implementation, and evaluation
(p. 15) by B. Kahn, 2005, Hershey, PA: IGI-Global. Copyright 2005 by IGI-Global. Reprinted by permission of the publisher.
Quality standards in e-learning. Frydenberg (2002) summarized existing quality standards for online education in the United States and found the following themes most common in the literature: institutional and executive commitment; technological infrastructure; student services; instructional design and course development; instruction and instructors; program delivery; financial health; legal and regulatory compliance; and program evaluation. She observed the institutional and executive commitment theme to be one of the most common in the literature, and evaluation of a program to be the least written about, since at that time, “few fully developed programs have arrived at a stage where summative evaluation is possible” (p. 13).
Five pillars of quality. The Sloan Consortium, an organization dedicated to improving the quality of online education, identified Five Pillars of Quality Online Education (Lorenzo & Moore, 2002) as building blocks for quality online learning: Learning Effectiveness; Student Satisfaction; Faculty Satisfaction; Scale; and Access.
The Learning Effectiveness pillar requires a commitment to providing students with high quality education at least equivalent to that of traditional students which includes interactivity, pedagogy, instructional design, and learning outcomes. In fact, the Learning Effectiveness pillar places a priority on learning activities because of student interactivity with the instructor, and creating a learning environment of inquiry (Lorenzo & Moore, 2002). The Student Satisfaction pillar focuses on the experience of the student by providing necessary support services such as advising and counseling and opportunities for peer interaction. It also examines student satisfaction with what and how they learned in either the online course or overall program.
The Faculty Satisfaction pillar examines the support and resources needed for faculty to have a positive experience while teaching online. According to the Moore (2002), “Faculty satisfaction is enhanced when the institution supports faculty members with a robust and well-maintained technical infrastructure, training in online instructional skills, and ongoing technical and administrative assistance” (p. 58).
The Scale pillar, originally entitled Cost Effectiveness, focuses on the cost effectiveness and capacity of programs so that quality learning is offered as an educational value to students. They believe an institution should monitor costs to keep tuition as low as possible while providing a quality educational experience for both students and faculty. Strategies for quality improvement were also addressed in the Scale pillar.
The Access pillar assures that students have full access to the learning materials and services they need throughout their online degree program, including support for disabilities and online readiness assessment. This pillar also examines barriers that may be in the way of online students having access to all resources necessary to achieve success.
Quality assurance strategy. Lee and Dziuban (2002) believed there were five primary components for evaluating quality within online education: (1) administrative leadership and support, (2) ongoing program concerns, (3) web course development, (4) student concerns, and (5) faculty support. Structured around the University of Central Florida’s online programs (Lee & Dziuban, 2002), their Quality Assurance Strategy (QAS) maintained the importance of administrative support and leadership for resources, training, and evaluation. They recommended that online programs be extensively planned through discussion, evaluation, and analysis, which is crucial to the overall success of the program.
Assessment model. Lockhart and Lacy (2002) worked with faculty and administrators at several national conference meetings to develop a model that consisted of seven components needed to evaluate online education: (1) institutional readiness/administration (budgets, priority and management); (2) faculty services (support, outcome measurement, and training effectiveness); (3) instructional design/course usability (technology must be user friendly and accessible); (4) student readiness (assessment for student readiness and preparation); (5) student services (effectiveness of provided services); (6) learning outcomes (measurement of learning outcomes); and (7) retention (comparing rates to face-to-face delivery and enrollment monitoring). Focusing on data collection and analysis, they suggested further surveys were needed in the areas of faculty support, training, and student support. They also recommended that student grades and retention rates be examined as well as results of online learning outcomes. Finally, they challenged higher education to understand how important it is for institutions to “plan, evaluate, and then revise programs based upon assessment results rather than just being another institution to deliver classes at a distance” (p. 104).
Accreditation and quality assurance. The Council for Higher Education Accreditation (CHEA) (2002) examined 17 institutional accreditors recognized by the United States Department of Education (USDE) or the CHEA. Each reviewed distance learning programs within their constituency which resulted in what they believed to be the seven key areas for assuring the quality of distance learning programs:
- Institutional Mission: Does offering distance learning make sense in this institution?
- Institutional Organizational Structure: Is the institution suitably structured to offer quality distance learning?
- Institutional Resources: Does the institution sustain adequate financing to offer quality distance learning?
- Curriculum and Instruction: Does the institution have appropriate curricula and design of instruction to offer quality distance learning?
- Faculty Support: Are faculty competently engaged in offering distance learning and do they have adequate resources, facilities, and equipment?
- Student Support: Do students have needed counseling, advising, equipment, facilities, and instructional materials to pursue distance learning?
- Student Learning Outcomes: Does the institution routinely evaluate the quality of distance learning based on evidence of student achievement? (p. 7)
The CHEA report (2002) described three challenges that must be addressed for assuring the quality of online education programs: the alternative design of instruction, the abundance of alternative providers of higher education, and an expanded focus on training.
Concentric support model. Osika (2004) developed a concentric model for supporting online education programs using seven themes: faculty support, student support, content support, course management system support, technology support, program support, and community support. She validated this model with a panel of experts that consisted of administrators and those with various roles in online education programs including faculty and staff members.
Assessment recommendations. Moore and Kearsley (2005) postulated that while everyone within the institution has a role to play in quality education, they believed senior administrators should be responsible for measurement and quality improvements. While they did not offer a prescriptive plan for evaluation, they suggested assessment of the following areas: the number and quality of applications and enrollments; student achievement; student satisfaction; faculty satisfaction; program or institutional reputation; and the quality of course materials.
Six–factor solution. Haroff and Valentine (2006) surveyed adult educators and found six dimensions they believed to be important to program quality: (1) quality of instruction, (2) quality of administrative recognition, (3) quality of advisement, (4) quality of technical support, (5) quality of advance information, and (6) quality of course evaluation. It is interesting to note their inclusion of student advisement, which has not been heavily identified as a quality indicator in previous literature. They suggested a qualitative study be used to triangulate their results.
The Quality Scorecard for the Administration of Online Education Programs. The most recent study for quality online education is The Quality Scorecard for the Administration of Online Education Programs developed by Shelton in 2010. A six round Delphi study was undertaken with 43 seasoned administrators of online education programs, with the majority of experts having more than nine years of experience in this field. Through a six month research process, they came to consensus on a list of 70 quality indicators that administrators should examine within their programs to evaluate quality. The original set of 24 benchmarks from the IHEP, Quality on the Line: Benchmarks for Success in Internet-Based Distance Education (2000) were used as a starting point and almost all were determined to still be valid in 2010, with modifications. The 70 quality indicators were categorized into nine categories: (1) Institutional Support, (2) Technology Support, (3) Course Development and Instructional Design, (4) Course Structure, (5) Teaching and Learning, (6) Social and Student Engagement, (7) Faculty Support, (8) Student Support, and (9) Evaluation and Assessment. The study’s expert panel suggested a method for quantifying the program evaluation by allowing up to three points for each indicator within each category (0 points - not observed, 1 point - insufficient, 2 points - moderate use, 3 points - completely meets criteria), depending on the level of existence each quality indicator can be demonstrated by the program administrator. A perfect score would equal 210 points. The study resulted in a quality scorecard for the administration of online education programs that administrators may use to identify areas within the online program that demonstrate excellence or are in need of improvement (see Table 3).
Table 3. Summary of The Quality Scorecard for the Administration of Online Education Programs

Because the Sloan Consortium was a gatekeeper in the research study, they offer the Quality Scorecard on their institutional website so that administrators of online education programs may freely use the scorecard tool to demonstrate program evaluation. In an ancillary handbook (published by the Sloan Consortium), each of the 70 quality indicators are defined with more depth and examples; and best practices are provided to better demonstrate the level of quality that may be reached with each indicator. It is important to note that all 70 of the quality indicators on the scorecard fall within the Sloan Consortium’s original Five Pillars of Quality (Learning Effectiveness; Student Satisfaction; Faculty Satisfaction; Scale; and Access).