Numerous studies have been conducted on various facets of online education focusing on e-learning and different methodologies used with online courses. However, the examination of the assessment of student’s work completed online has been limited and sporadic. With online education generating the fastest growth among student enrollment in K-12 and universities, research needs to be conducted regarding the best instructional practices.

As online education moves into the mainstream educational world, a key question needs to be answered, “How do I know what my online students have learned?” There are no easy answers, but with a little creativity and flexibility, it can be discovered that the online learning environment opens up a host of new educational assessment possibilities. Meyen, Aust, and Issacson (2000) reiterate while assessment in an e-classroom continues to develop, with a host of advantages and disadvantages, it must be explored to provide assistance to instructors so that students receive optimal feedback. Assessment is no longer the periodic formal process of exams and graded activities, which may or may not be discussed with the class; it is now in the context of a one-on-one relationship with the e-instructor and each student in an online course (Meyen, Aust, & Issacson, 2000).

A large amount of investigation and development is currently underway at the university level regarding the possibilities for effective and efficient online assessment. There are numerous reasons for online assessments being studied.

Many academies are seeking to diversify assessment tasks, broaden the range of skills assessed and provide students with more timely and informative feedback on their progress. Others are wishing to meet student expectations for more flexible delivery and to generate efficiencies in assessment that can ease academic staff workloads. All staff involved in such initiatives are discovering they face a large number of educational issues (Online assessment from the Centre for the Study of Higher Education, 2002).

In reviewing the literature, some studies have centered around the identification of key issues related to assessment of students’ performance in online education. In Dereshiwsky’s (2001) study, she stated that assessing student performance online is an admittedly challenging aspect of instruction. Often equal parts of art and science, it can cause anxiety for students and instructors alike. Are the assignments a valid reflection of the course curriculum? Is there an equitable and clearly understood evaluation and feedback system in place? Above all, it should be asked, is the assessment genuinely meaningful and useful to students in terms of their academic growth?

Research has shown that appropriately designed assessment helps to facilitate these positive learning opportunities and outcomes for students. Brown, Race, and Smith (1996) proposed that how we assess our students has a profound effect on what they learn, and on the ways in which they learn. If our choices of assessment provide systems under which students are goaded into activities that provide short-term memory, information recall and surface learning, we should not be surprised if the outcomes are exceedingly poor in terms of learning gains (Assessment for learning, 2010).

According to Hemby et al (2004), the online instructor must evaluate current assessment tools to identify the most appropriate assessment for the learner outcomes. The assessment must match the project so that e-students are aware of the key components that will be evaluated in the assignments. With a review of current assessment techniques comes the demand for taking the time to adapt assessment so that appropriate and timely feedback may be provided to the online students.

When converting traditional classroom activities to the online learning environment, instructors should remember that these activities require assessment tools to be developed and/or modified from traditional classroom assessments. Discussion postings, projects, papers, and student-led discussions are important in the engaged learning environment but assessing students’ participation and work product necessitates the development of discussion analysis tools, team assessment tools, and reflective self-assessments (Conrad & Donaldson, 2004)

Palloff and Pratt (2003) developed the following principles for student assessment in an online learning environment:

The core principles of assessment are outlined in the University of New South Wales (UNSW) Assessment Policy. The policy stated: Online assessment has the potential to increase the diversity and flexibility of assessment for staff and for students and to provide students with prompt and individually targeted feedback. It can also serve as a particularly valuable form of self assessment as there is software, that is readily available, that enables students to monitor their progress by accessing randomly allocated quizzes at a time that is convenient to them. Care needs to be taken, however, to ensure that online assessment is closely related to course aims and learning outcomes, and that it does not encourage students to focus on low-level cognitive skills (Core principles of assessment, 2009).

Dereshiwsky (2001) stated the online environment has fostered increasingly creative applications of multiple assessment procedures and tools. Thus, the use of creative assessment tools, such as, the N/A/R Rubric must be recognized. Huba and Freed (2000) illustrated how rubrics can be used to judge thinking processes and the affective components of learning. A rubric that was developed to address critical thinking of university students illustrates how this tool can be used to guide and evaluate higher-order thinking skills. This seven-dimension critical thinking rubric was developed at Washington State University through the Critical Thinking Project. It was found that 92% of student writers within a Writing Portfolio course demonstrated writing proficiency but ‘surprisingly low critical thinking abilities’. Dramatic improvements were found as a result of the introduction of the Critical Thinking Rubric whereby students’ critical thinking scores “increased three and a half times as much in a course that overtly integrated the rubric into instructional expectations, compared with performances in a course that did not.” The Critical Thinking Rubric allowed the faculty to “make a shift in our academic culture” and “has proven useful as a diagnostic tool for faculty in evaluating their own practices and testing the outcomes of different approaches objectively.” (Critical Thinking Project, 2003)

Rubrics should communicate the instructor’s expectations to meet the standards of the course’s assignments. Furthermore, a rubric may be used to define the expected performance levels for online discussions. The N/A/R rubric differs from one used to measure performance in the traditional classroom. While the creation of rubrics can be time consuming, students and teachers are better able to understand expectations for an assignment when evaluation criteria are provided at the time a task is assigned. Conrad and Doanldson (2004) emphasized a rubric clearly specifies the expectations for the activity and the effort required by the student to achieve a desired score.

The importance of feedback through online assessment must be part of this review. In examining the literature concerning online assessment, Dereshiwsky (2001) stated that online assessment is characterized by timely, efficient, and detailed individual feedback to students. Furthermore, Hatlie (1999) felt assessment has such a significant impact on e-learning that it has been described as the most powerful single moderator that enhances achievement. A study of the core principles of assessment indicate that to benefit student learning, assessment feedback needs to be:

Feedback follows these points, aiding students to think analytically regarding their work and to reflect on what they need to do to improve it. It can encourage students to see their learning in new ways and to gain increased performance and satisfaction.

In a general context, whether in the online or traditional classroom environment, assessment drives learning. What is needed is continued exploration of online education as a means of facilitating instruction, constructing knowledge and skills, and assessing learning in order to improve the online classroom experience (Henry et al, 2004).

Participants in the survey study were students in educational administration (EDAD) online courses at a small private university in the Midwest. These participants were graduate level students who had taken the EDAD courses as part of their completion of the Masters of Education degree program or the course fulfillment for their principal’s license.

Ninety-one students were sent the online survey. Thirty-nine students completed the survey for a return percentage of 43%.

The study was designed to examine the EDAD student’s perceptions regarding the Narrative/Analysis/Research (NAR) Rubric as an instructional tool in online courses. The following questions guided the research:

The purpose of these questions was to provide insight and answers concerning the effectiveness of the NAR Rubric. The data should provide confirmation of its beneficial utilization and suggestions for future use.

The survey instrument used in this study was developed by the author after discussions with EDAD faculty, students currently enrolled in the EDAD program, and reflection and analysis by the author related to the teaching and learning occurring in the online classes.

This survey consisted of three sections. The first section contained 13 “forced response” questions in which 5 of these questions related directly to the students’ perceptions of the NAR Rubric. The “forced response” questions were set up on a five point Likert Scale ranging from strongly agree – agree – neutral – disagree – strongly disagree. The second section had 2 open-ended questions regarding the students’ views of the course. These open-ended questions were:

The final section was made up of questions asking demographical information on the survey’s respondents.

The survey was sent electronically to the students’ e-mail addresses. Included with the survey was a cover letter explaining the reason for the survey. The participants were assured of the confidentiality of their responses. Survey results were submitted through the Zoomrang program. Through this program, results were compiled and set up according to the questions.

The respondents of the study’s survey were diverse. Of the 39 EDAD students who responded to the survey, 59% were female and 41% were male. Most of these graduate students were teachers at 69% of the total sample population, while 26% were administrators, and 5% were in other job positions. The breakdown of the educational levels for the participants was 51% at the high school level, 8% at the middle school level, and 41% at the elementary level. Finally, the graduate students’ years of experience were quite varied with 18% with 1-3 years of experience, 51% with 4-10 years of experience, 18% with 11-20 years of experience, and 13% with 20 + years experience.

The purpose of the study was to retrieve the student’s perceptions and viewpoints concerning the NAR Rubric and the learning outcomes attributed to the Rubric’s criteria. Some of the questions asked pertained to the NAR Rubric as a beneficial teaching and learning tool and other questions were concentrated on the learning outcomes of the student. To understand the distribution of responses to the survey items, frequency tables were set up to organize and summarize data. Frequency distribution results in Table 1 show the students’ perceptions to survey questions focusing on the NAR Rubric effectiveness.

Table 1 reveals the student’s viewpoints were positive and accepting of the NAR Rubric. The students clearly indicated that the NAR Rubric provided direction in completing online assignments with 87% of the respondents agreeing that the rubric provided this direction.

Only 6% of the respondents felt the rubric did not provide direction and guidance. In survey item #9 found in Table 1, the students indicated that the assessment feedback based upon the NAR Rubric was very beneficial with 82% agreeing with statement. When the respondents were asked to rate what were the most meaningful and helpful learning activities in the online course, the NAR Rubric was the second most desired learning activity with 19 students. Only case studies ranked higher.

Another set of survey items in Table 2 showed some learning outcomes were generated in online courses because of the key criteria found in the NAR Rubric.

Table 2

The EDAD student’s perceptions of the learning outcomes generated from online assignments were clear and evident. The students believed that they conducted more research for the assignments in the online courses than in the face-to-face courses (survey item #6), and they did more analytical thinking and work for the assignments in the online courses than face-to-face courses (survey item #7). Seventy-two percent agreed with more research being conducted and 64% of the students agreed that additional analytical thinking was produced in online courses. Again, online instruction has the potential to provide opportunities to promote reflective thought and critical thinking through realistically integrating and applying principles learned. Online instruction, such as a simulation, thrusts learners into a learning experience, increasing involvement and providing activities that actively engage learners to analyze, synthesize, and evaluate information while constructing knowledge (Driscoll & Carliner, 2005). Two of the three key criteria found in the NAR Rubric are research and analytical thinking. Thus, a large number of survey respondents perceived that research and analytical thinking are completed in the online assignments by the utilization of the NAR Rubric.

In the 2 open-ended questions, respondents expressed what they liked least about the online course (Survey item #14) and what changes they would make in these courses (survey item #15). When responding to what they liked least or what changes they would make in the hybrid courses, no student expressed dissatisfaction with the NAR Rubric.

In summary, a large percentage of students perceived the NAR Rubric to be beneficial and meaningful when completing the online assignments. Also, they felt that the key criteria found in the NAR Rubric helped produce more research efforts and analytical thinking with the online assignments.

Helping faculty members design assessment tools and practices for online student assignments are paramount. Bauer (2002) identified the emergent problem with online learning as being that of assessing students with traditional assessment measures.

The results of this study demonstrate that students perceive the Narrative/Analysis/Research Rubric as a good assessment tool to guide students in completing online assignments. It is important to note that students thought the NAR Rubric significantly provided direction and feedback in completing online assignments. In fact, students indicate in the study’s survey that it is one of the most helpful items used in the online courses they had taken.

Research suggest that students may need additional guidance in an online course which does not meet in a traditional, face-to-face format. The NAR Rubric was specifically developed for online assignments in order to act as an instructional guide and also to provide constructive feedback to students. When students were asked “which learning activities were the most meaningful and helpful”, they identified the NAR Rubric above discussion boards, e-portfolios, videos, power-point presentation and several other learning activities.

Even though the NAR Rubric provides instructional guidance with online assignments, students perceive its greatest importance comes from the feedback provided by the instructor. This feedback comes in the form of fulfilling the three main components of the NAR Rubric: the narrative in addressing the assignment’s question(s), the inclusion of analytical reasoning, and the support of research and references with the assignment. Dereshiwsky (2001) agreed that properly designed web course assessment can produce more detailed feedback for students, more individual assessment of student’s work, greater student engagement, and a clearer direction in what is required with the assignments. Harasin, Hiltz,Teles, and Turoff (1996) in discussing online learning, stated “In keeping with a learner-centered approach, evaluation and assessment should be a part of the learning-teaching process, embedded in class activities and in the interactions between learners and between learners and teachers”.

The students perceive the NAR Rubric cultivates critical thinking and research with online assignments. Overall, the results indicate students conducted more research and more analytical work in the completion of assignments. The reason for this increase in these two learning outcomes is that the students seem to follow the NAR Rubric’s Analysis and Research criteria. Thus, a rubric can not only be a teacher/instructional tool, but a rubric is a learning tool (Designing rubrics to fit assignments, 2010). Students should see rubrics as “red flags” that highlight the important components of a paper. Rubrics remind students what to watch out for, what to revise if necessary, and what will be considered in the assignment of the paper (Smithson, 2001). It seems many of the students recognize and appreciate the need to follow the three criteria of the NAR Rubric in order to achieve the expected learning outcomes.

The NAR Rubric became a major instrumental guide to their learning outcomes in the areas of research and analytical thinking. In the open-ended questions, one student commented there was definitely an emphasis in the readings and research with the assignments.

As online education continues to grow at a rapidly increasing pace, sound teaching and learning practices in these courses need to be implemented. It is important for faculty members to understand the best practices of online education and be cultivating the higher order thinking skills in this type of instructional environment. The NAR Rubric described in this paper was investigated by the author related to students’ perceptions of its effectiveness in online courses. Overall, the data from this study indicate that the rubric is a beneficial tool for students and their learning.

The NAR Rubric provides direction in completing online assignments. With no face-to-face classroom contact (in online courses) to introduce and/or thoroughly explain the assignments, the rubric becomes the guiding tool to work through the assignment and emphasizes the key components in the paper. These key components are: a comprehensive narrative that answers the assignment’s directions/questions; analytical comments and thoughts interwoven and supporting the narrative; and evidence of research and references embedded in the narrative. Students in this study felt their learning was enriched and expanded concerning research work to complete an assignment and the generation of critical thinking incorporated into the assignment. Contrary to some thinking, current online college courses are not alienating, lock-step programs. They are a labor-intensive and intellectually challenging forum which elicits deeper thinking on the part of the students and which present more individual communication between instructor and student.

Future research is encouraged to explore:

Faculty members teaching online courses may benefit from a review of the NAR Rubric found in this study. With the increasing number of courses going online, there needs to be “best practices” studied in order to aid instructors in optimizing learning outcomes. The NAR Rubric would provide this help and guidance for faculty when introducing and assessing assignments in online courses.

Appendix

Rubric to Evaluate Assignments