How exams work

When it comes to exams, who decides what needs to be tested and how? Stephen Diston is the head of the Science Subject Team at OCR - a UK exam board - and he talked Kat Arney through the process...

Stephen - Well the exam paper will be written by a group  people who've clearly got expertise in that subject area and they'll also have experience of teaching students in that age range. But they're constrained as to the area they can test on the examination by the content that's laid out in the specification and that document is set to a certain degree by the government who will indicate the content that they expect to see in each of the individual subjects, and for some subjects the entire content is laid out by the government. In others you might get 50-60% of the content is mandated and then the exam board and the examiners have the freedom to select which additional content they'd like to put in there themselves.

Kat - So what does make a good assessment for kids? A lot is made of exams but there's things like coursework, practical exams as well - what's the best way of figuring out how to test a child's knowledge and ability?

Stephen - Well, with examinations you're always weighing up some competing requirements. One of the big worries with the high stakes exams, such as GCSEs and A levels, is that there is always the possibility that people might be looking to cheat or to game the system in some way, so a high degree of security around the examination is part of what we're looking for and that straight away tends to put some limitations on an examination. It's far easier to control the security of that examination than something like coursework where it has to be carried out under much less secure conditions and, of course, it may well be that the school needs to arrange equipment and resources weeks, even months in advance of the exam taking place.

That being said there's limits to what you can actually assess through an examination and so, if you look at practical science skills, the only way to really know if a student can put together the equipment and carry out a titration is for someone, generally the teacher, to actually watch them carry out the experiment and then assess how well they've done that. So there's always been a balance to be achieved between how much of the examination or how much of the weighting of the assessment is given to the practical activities, and how much is determined by exams.

But from a security point of view, as I said, if you can examine it, you should examine it is the current approach that's taken.

Kat - Well that's from a security perspective that I may be showing my age a little bit here but I was one of the earlier years to do GCSEs and a whole part of the GCSEs was it's this coursework element and different exams have different proportions of coursework, and I understand now things like A levels and AS levels here in the UK have more elements of coursework. Is coursework a good idea in terms of assessing pupils and is that changing, swinging back towards - no exams, exams are best?

Stephen - Coursework is a very different way of assessing pupils because, if nothing else, you can allow a much longer period of time for students to actually carry out the course work. That lets you tackle much more substantial pieces of work than you can do in an examination where it needs to be completed within a few hours and is unseen material so it widens the scope for the skills that you can actually assess in coursework.

Kat - In my view it's more a life beneficial skill. When I do my job as as a writer or a broadcaster, I'm not turning up and sitting a two hour exam every week; I'm getting sources, I'm pulling documents together, I'm writing things. Is coursework a better way of testing those kind of skills that kids actually need?

Stephen - I think it's an important way of testing those skills - it's not necessarily the only way you can do it. But when you look at the range of qualification out there, as you look at qualifications which have a more vocational design and they're more closely targeted on particular career pathways, you see that the weighting that's given to coursework increases quite significantly. And there's plenty of qualifications out there which might be as much as 75% coursework and only a quarter of it based upon an examination.

Kat - But more broadly in the sort of academic subjects, exams are still seen as the best?

Stephen - They're seen as the most secure and reliable way of assessing skills in students. One of the changes that's coming through into A level science at the moment is the decision being made to actually separate out the practical skills from the examined results, so you'll get a grade in the future that's based purely on the exam, you'll get a separate result that actually says how you did on the practical skills element.

Kat - And I guess when we're training the scientists of the future those are two equally important skills to be measured?

Stephen - They are indeed. And for universities, who arguably are the main consumers of A level results, it really does make a difference to know whether the students that you're looking at, they may have a very strong theoretical ability within the subject but actually is it safe to let them loose in the laboratory when they arrive?

Kat - A terrifying thought. I've got one more question which is a little bit of sting in the tail is: every year we hear the exam results and more and more people are getting better and better grades and things like that and people go  - Oh, they're easier now than they used to be in my day. Is that really true or are they just different?

Stephen - They are indeed very different from how they were years ago and, I mean, how long have we got to talk about this? We operate a particular system within the United Kingdom where it's not strictly what's called "a norm reference system," so you have a fixed proportion of students who get the grades each time but there is an element of comparing performance on students, year on year, against previous exam papers. So whilst you do have a lot of statistical analysis that goes into the results, you actually have experienced examiners, human beings, sat down looking at scripts that students wrote last year, looking at the scripts that students wrote this year and making value judgements about the relative difficulty of the exams that are being sat and looking at performance from the students that they feel matches the level of performance that was seen for an A, or a grade C, or whatever it might be, in previous years.

Kat - So I think that young folk, even if they don't have good knowledge of 80s and 90s pop music, it's not that there exams are a lot easier.

Stephen - No it's not!