RSS

My Teacher is a Zombie – Marking by Rubric on Moodle

20 Nov

bczI have just finished marking a whole bunch of flash animations as part of a grade 8 computer skills examination, and the topic of the animation task just happened to involve a zombie. After assessing about a hundred of these things, I felt pretty zombie-like too! But the point I wanted to make is actually about rubrics. When I was a kid, teachers never used rubrics, or not that I was aware of anyway! The mark you got seemed fairly arbitrary for it appeared at the bottom of your essay with a circle around it and a disembodied comment such as “Good” or “Poor”. After a glass of wine, we speculated, the comment might have become more expansive, but also more illegible! Perhaps this is an unfair assessment of my teachers. There were, after all, helpful annotations in the form of underlined spelling mistakes, and red lines through phrases felt to be inappropriate or colloquial. I have to say though that I seldom understood why I had been given a particular mark, or how to go about improving my performance.

These days, the emphasis is on using rubrics to try to help students understand the criteria by which they have been assessed, and there is no doubt that a well-designed rubric can lay bare where marks were gained and lost. There is, though, still something awfully mechanical and routine about the whole assessment process. Anyone who has ever had a sizeable number of scripts to mark will know that catatonic, zombiesque state that marking induces. The petty nit-picking, or the cavalier acceptance of partially correct responses, the moments of self-doubt and angst over whether to deduct marks for spelling or not! Even intelligent human beings can be reduced to mind-numbing pedantry when faced with the challenge of assessing a pile of scripts that need to be finished before Monday 8am!

One hears stories about teachers who deliberately lose scripts rather than mark them, or the legendary stair method – throw the scripts down the stairs. the ones at the top get an A, the next step a B, and so on! Go into any staff-room during exam time and listen to the hysteria build after days of being forced to sit in front of piles of marking, armed only with a red pen and the promise of caffeine and nicotine at predetermined moments of the day, rewards for each batch of twenty, or every half-hour crossed off the boredom of the day! Some teachers mark a whole script at a time, while others tackle questions or batches of questions in sequence. If it gets too much you can count the scripts remaining. Some mark in solitary isolation, others in groups calling out particularly juicy answers to each other as they draw a red line through the page!

I’ve drawn a pretty gloomy picture about what is probably every teacher’s least favourite part of the job – the part that is least rewarding, and perhaps the least affirming both for student and teacher. Even loving, caring individuals become like zombies when marking!

rubricOne aspect of marking online is the magnificent affordance offered by rubrics. The screenshot shows my rubric for assessing the zombie flash animations which have haunted the last few hours of my life! The rubric module on Moodle allows you to set up a rubric, which you can then use for delivering feedback and assessment. After opening the file to be assessed, you simply click on the relevant box in the rubric, and attach relevant comments for each question, and a comment at the end. You can attach a feedback file if you wish. The one assessed here was perfect, except for one error, which has been noted. At the end I attached a positive comment and the software automatically adds up the marks and appends them to the grade-book which can be downloaded as a spreadsheet at the end!

Using a rubric in this way minimises a great deal of the pain, and possibility of error associated with adding up manually, or transferring to a grade-book, leaving more time for helpful comments! Rubrics can be saved as templates, and re-used, edited, or tweaked over the years. As soon as you have marked an assignment the feedback, rubric and mark becomes available to the student on their Moodle page together with any memo or exemplar you upload. I often make a screen-cast video of myself doing the exam, talking through sticking points and why something has been assessed in the way that it has. I post this on the Moodle page so that students can check their work against the exam questions. I find this works very well, and makes the task of handing back exam papers less fraught!

I do worry though that using the rubric module has made the process so slick, that I am running the risk of just going through the motions. Using an electronic rubric frees up the time to prepare a memo video, and to write out longer comments, but it is in many ways as zombiesque a process! Electronic or otherwise, … tick … tick … tick! Click … click … click!

 

 
8 Comments

Posted by on November 20, 2014 in Assessment, Graphic Software, MOOCs, Moodle

 

8 responses to “My Teacher is a Zombie – Marking by Rubric on Moodle

  1. sjc

    March 6, 2017 at 4:22 pm

    My first encounter with Rubric, was as a grad student @ SNHU studying Communications. At first I couldn’t understand why my A and B papers were getting sliced and diced down to C+. I read the Professor’s feedback, “Your narrative is articulate, and insightful. However, you incorrectly used a word on page 7, only communicated with 1 classmate, and not 2 on discussion board, mechanics were good, but could use improvement, and submitted assignment late @ 12:01AM.

    This cunt took an A paper and shredded it to a C-, because of nit picking bull shit. This is why I am going back to brick and mortar schools, because these on-line diploma mills create a race to the bottom with tools such as rubrics, that were designed for morons that can’t stay focused, and write objectively.

    Like

     
    • Steve Covello (@idmodule)

      May 8, 2017 at 2:41 pm

      SJC: What you encountered was unique to the instructor. I train faculty to teach online and also teach online myself. One of the parts of our training is how to interact online with students and how to use grading rubrics without driving yourself nuts.

      There is also the possibility that your paper wasn’t as good as you think it is despite instructor feedback. Maybe she/he was being overly picky, but then I have encountered many students in my classes who have filled out self-assessments of their papers (which is required in my class) who clearly do not understand what college-level writing is, are not able to produce it, yet naively feel they are meeting the standards. I’ll give you the benefit of the doubt, but it is possible.

      If you were in my F2F class and handed in a late paper without having completed the assigned tasks, then you wouldn’t get an A either. Just sayin’…

      Like

       
  2. tjhunt

    November 24, 2014 at 9:34 pm

    “After a glass of wine, we speculated, the comment might have become more expansive, but also more illegible! Perhaps this is an unfair assessment of my teachers.”

    Actually, probably not unfair, even if your teachers were completely sober. People have done research studies looking at consistency of marking, and it is surprising how much variation there can be between two different people marking the same scripts, or even the same person marking ad different times.

    I don’t have any specific references for grading essays, or extended pieces of work, but even for short-answer questions like “Why does oil float on water?” there can be only 90-95% consistency between different human markers. See, for example, Table 2 in http://oro.open.ac.uk/20868/1/Butcher_%26_Jordan.pdf

    Like

     
    • Dorian Love

      November 25, 2014 at 12:49 pm

      That’s pretty amazing! My experience is though that while teachers might disagree about marks per se, they ten to agree on rank order!

      Like

       
  3. Steve Covello

    November 20, 2014 at 4:50 pm

    Dorian – I can’t argue much differently with what you are describing since much of what we do here in our online courses in higher education is rubric-centered.

    One nuance I can add, however, is that the rubric-based grading happens to be just one of several ways students are assessed on activities throughout the course. If I were to deconstruct the total assessment package in a typical (good) online course, it would include a combination of rubric-based assessment, individual instructor feedback, and peer commentary.

    Let’s also mention that “assessment” could be interpreted as strictly grading, or it could be a combination of grading and feedback from various sources.

    If there is a case to be made in defense of rubric grading, it does at least streamline that portion of the overall assessment package. Having it semi-automated makes it all that less time consuming to do.

    And it may, more than anything, serve as an indicator of how good the instructional design and teaching is more so than the quality of student work.

    Like

     
    • Dorian Love

      November 21, 2014 at 9:10 am

      Agreed – Steve. I was looking at the chore aspect of marking! Moodle allows wonderful opportunities for peer assessment and feedback as well. This was also an exam situation!

      Like

       

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.