Marking policy

Assessment is part of learning. Doing the coursework and revising for the exam are important modes of learning that complement other modes (classes, readings, etc.). Therefore, the primary goal of the assessments is to create these active learning opportunities for you. It is the activity that helps you learn, even if you don’t get all the answers correct.

The marking policy is designed to reward students who engage with these learning activities: in other words, students who try.

Coursework

Structured marking schemes are used, in which marks are awarded in specific categories (e.g., theory, experiments). You may also be required to follow a specified structure for your report, which aligns with the marking scheme.

Marking is positive: there is no penalty for attempting something but getting it wrong, compared to no attempt.

What exactly does “positive marking” mean for the coursework?

It means that attempting to provide an explanation, or attempting to perform an experiment, can only increase your mark, compared to not attempting it. In the worst case, if your explanation is entirely wrong, then you’ll get no marks for it. This is no worse than not attempting it. Therefore, the correct strategy is to attempt all parts of the assignment, and to write something for each section on the structured marking scheme.

For comparison, under a negative marking scheme you might receive a lower mark for writing something incorrect, compared to not writing anything at all. This is not the policy in Speech Processing.

Online Tests: Multiple Choice Questions

You might think that marking a multiple choice exam is easy. Yes, marking is easy, but converting the raw point score (e.g., out of 50) into a student’s percentage grade is another matter.

Definitions:

• Raw point score: the number of questions with correct answers (e.g., 37 out of 50)
• Final mark: the percentage grade awarded to the student for the exam

Marking is positive: there is no penalty for attempting a question but getting it wrong, compared to not attempting it.

What exactly does “positive marking” mean for the test?

Simply that you will get more marks (on average) by attempting a question, than by leaving it blank. Blank answers always get 0 points. Answering randomly, without even reading the question, will get you an average of 0.25 points per question, if there are 4 options. Reading the question, and narrowing your guess down to a few of the available answers, then picking one of those at random, will get you more points.

If that’s not clear, consider this example:

Question: What are the two axes on a spectrogram?

• A) amplitude and frequency
• B) frequencies of the first two formants
• C) time and frequency
• D) time and amplitude

Now, we know that spectrograms don’t only show formants, so B cannot be right. We’ve narrowed it down to 3 options, and just guessing at this point will get us (on average) 0.33 points. We remember that time is one of the axes, but can’t remember the other axis. Now we’ve narrowed it down to C or D. Guessing will now score us (on average) 0.5 points.

This is what I mean by positive marking: attempting a question will never decrease your mark, and the key message is that partially-correct answers will get you more points than incorrect answers. For multiple choice, “partially correct” means narrowing down the options to a smaller set, which includes the correct answer, before guessing.

Why not just convert the raw point score into a percentage in the most obvious way: by dividing by the total number of questions, then multiplying by 100?

Multiple choice exams are quite different to exams with written answers, because there is a closed set of answers. This means it is possible to appear get some questions correct simply by guessing. The multiple choice format cannot detect the difference between guessing the correct answer and actually knowing the correct answer. The conversion from the raw point score into a final percentage mark needs to take this into account.

Consider the case when questions have 2 options (A or B). Answering randomly will (on average) get half of the questions correct. Simple conversion would turn that into a final mark of 50%. According to the University’s Common Marking Scheme that corresponds to “Performance at a level showing the potential to achieve at least a lower second class honours degree” for undergraduates, and “A good performance, satisfactory for a masters degree” for postgraduates.

Do you think someone who simply answers randomly should be awarded a 2.ii or even a Masters degree? No, I thought not!

Let’s go further. The number you will get right by answering randomly will vary with the number of options per question. With two options, you’d get 50%, with three options, 33%, with four options 25% and so on. This is not the kind of property we want from an assessment. As the person setting the exam, I could manipulate that number by varying the number of options. For example, if I cared more about student feedback than actually helping you learn something, I could set an exam with two options per question, and all students would pass, even those who learned nothing.

So how exactly are raw point scores converted to final marks?

We need a method that is invariant to the number of options per question. The method used in Speech Processing is to correct for chance. This was suggested by a previous external examiner of this course.

Raw point scores are mapped to percentages using a very simple function that has the following properties:

1. Corrects for chance: answering all questions randomly, without reading the questions, should result in a final mark of 0%
2. Does not impose an artificial ceiling: answering all questions correctly should result in a final mark of 100%
3. Proportional: between the above two extremes, there should be a simple relationship between the raw point score and the final mark.

First, the raw point score (e.g., 37) is converted to a raw percentage, by dividing by the number of questions (e.g., 50) and multiplying by 100 to get a raw percentage (e.g., 74%). This is then corrected for chance by subtracting the raw percentage that could be obtained by guessing. For an exam with 4 options per question, that is 25%.

final mark = (raw percentage – 25 ) / 0.75

Finally, rescaling using the value 0.75 ensures that the maximum obtainable mark is 100% (satisfying the no artificial ceiling property) and is a simple linear rescaling (satisfying the proportional property)

Any final mark below 0% is increased to 0% (to avoid negative marks).

Here are some example conversions performed by this formula:

Achieving an appropriate final course mark distribution

It is rather difficult to calibrate a multiple-choice exam: in other words, to judge its difficulty in advance. If an exam turns out to be particularly easy or difficult, the overall distribution of final course marks may be inappropriate. To achieve an appropriate final course mark distribution (which for Speech Processing will normally be a class average somewhere in the mid-60s and a standard deviation of 10-20%), the lecturer may need to employ one or more of the following mechanisms:

• Moderation of the exam by a global shifting and/or scaling of all final exam marks, after applying the above formula
• Moderation of the second assignment marks that takes the final course mark distribution into account

Understanding what your mark means

Work is marked according to the relevant Common Marking Scheme, to which each School adds a detailed description of each 10 point band across the full scale. You should read these carefully to understand what your mark means:

• • undergraduates should refer to pages 10-12 of the PPLS undergraduate student handbook.
  • postgraduates should refer to pages 14-15 of the MSc SLP handbook (there is no School-level postgraduate handbook).