If you're going to do this damn silly thing, don't do it in this damn silly way

In Yes, Minister (a vastly less sweary antecedent of The Thick of It) the senior civil servant Sir Humphrey tells the Minister: “If you're going to do this damn silly thing, don't do it in this damn silly way”.

This is relevant because of the news that the assessment resources that are due to be released around Easter won’t just be sent to schools, or made available only via secure websites, but will be published openly for everyone to access.  Everyone.  That’s teachers, students, parents, tutors.  Everyone.

Here I explain why I think this is a bad idea.

Exam papers have never been wholly secure.  There has always been the chance that an exam cupboard will be broken into, that a parcel will be mis-delivered, that a paper will be photographed before it should have been opened. One year a maths exam was replaced at short notice by an alternative version.  Very occasionally a primary school head teacher has been found guilty of malpractice for instructing pupils to ‘look again’ at wrong answers.  These situations were thankfully few and far between.

Back when the TES forum was the main social media used by teachers, every summer teachers would post reminders to others not to discuss the content of exam papers on the day of release, even after the exam had finished in this country, because those same papers might be sat at different times around the world. 

This summer the exam boards have been instructed to produce assessment materials, including questions, mark schemes, model answers and grade boundaries.  It is not clear to me (at the time of writing) whether it is just the questions that will be published, or whether the other materials will be available to students as well.

According to TES: An Ofqual spokesperson said: "We asked in our joint consultation with the DfE: “To what extent do you agree or disagree that exam boards should publish all of their papers shortly before the assessments in order to manage the risk of some students being advantaged through papers being leaked?.  66 per cent of respondents either agreed or strongly agreed with this proposal”

This question is flawed.  In my response to the consultation (one out of 100,000 that they received), I complained that the question was unclear.  If ‘publish’ meant ‘made available to schools’ and if the emphasis was on ‘shortly’, then this was sensible.  If ‘publish’ meant the damn silly thing they are now proposing to do (making it available to the whole world), then it wasn’t sensible.  How can Ofqual tell in which way the other 99,999 respondents interpreted the question?

The problems with releasing the questions to Uncle Tom Cobley and all:

·         Model answers will appear on The Student Room, older students, parents and others will record perfect versions of the questions and these will be available on YouTube.

·         Students who have a private tutor, or an older sibling who is studying that subject at a higher level, or whose parents are able to help will receive disproportionate levels of support.

·         Students will focus on learning these questions to the exclusion of learning anything related to their subjects.

But surely it is better to have transparency rather than to know that the first few students to sit the questions will then tell their friends in other schools and that it will advantage them?  Well no it won’t.  Earlier I mentioned the appeal not to reveal questions that was posted on the TES forum each year.  This year, students could be asked not to share questions, education-related websites could remove posts that refer to exam questions, etc.  The big thing is that it would be much harder for students to learn answers, because their friends would have sat only a small subset of the questions, it would be difficult to collate a complete set of questions, and each school is likely to use a different set of questions (albeit with overlap between them).  If they are all published to everyone and are all fully available then it will be much easier to learn the answers.

And the worst thing?  It will be in the interests of the school to use these flawed materials.

I am so pleased not to be a Head of Department and not to be in the situation of having to make a decision about which assessment questions to use.  If I were to choose to use questions I have selected, taken from different past papers, from different exam boards, over a number of years, then I could be fairly sure the questions will be unfamiliar to all of the students. 

If I wanted my students to get the grades they need for university, or to achieve extremely highly at GCSE, I would use these flawed papers, knowing that they would give me inflated scores that the officially-provided grade boundaries would translate into high grades.  I would then have evidence to demonstrate that the students deserve the highly suspect grades they would be receiving.

Where would this leave me and my personal integrity?  The second course of action isn’t cheating, isn’t bending the rules, but rather is using the materials that have specifically been provided by the exam boards, on the instructions of Ofqual.  Were I _not_ to use these I would disadvantage my students, would create more work for me and my department, all the while knowing that in other schools and colleges the students are using these freely-available papers.  This leads to another ‘what if’.  If I were a parent or student then I would be keen to know why my friends from other schools are being assessed using questions that are in the public domain, while my school had decided not to do this.

Surely I’m overselling this?  Surely it won’t happen?  Anecdotes suggest it will.  I have seen Yr 11s submit fantastic past papers, only to discover after a disastrous mock exam that the student had been assisted on a regular basis by their brother.  A memorable A-level student completed the past paper he had been set for homework by copying out the mark scheme, including the instructions to the examiners such as the symbols to say that answers could be rounded off, and giving alternative methods and answers.  I know of schools where the students do several papers as part of their mock exams and where they now give papers from different years because of the occasion where students used Google after paper 1 to find out the year of the paper and then downloaded and learned the answers to paper 2 from the same year, leading to a massive increase in the results for paper 2.  These are all genuine examples of the lengths pupils have gone to in practice papers and mock exams.  Ofqual will be publishing the _actual questions_ that students will be assessed on.  Clearly many students will learn/practise them.

And, even if you aren’t convinced by this, the DfE acknowledges that it will happen.  According to Schoolsweek:

Rebekah Edgar, the DfE’s deputy director for 2021 Qualifications, said the materials would also be published openly after Easter so “students and others can access them”.

She added that the publication was being delayed until after Easter “to try and avoid students sort of cramming with them over the holidays which we didn’t think was a healthy thing we wanted to encourage”.

I suggest they will still be cramming, but will need to pace themselves and catch up on sleep over Easter to allow them to do when the papers are published.

I accept that this system is going to be used in the summer: it’s too late to come up with anything different now. Maybe it’s not too late to change this decision about who the papers will be released to.  So I say to Ofqual, the DfE, and anyone else who will listen: “If you're going to do this damn silly thing, don't do it in this damn silly way”.

Notes:

https://www.tes.com/news/gcses-2021-car-crash-plan-share-test-materials

https://schoolsweek.co.uk/2021-assessment-materials-to-be-published-online-after-easter-dfe-confirms/

https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/964599/6747-2_response_analysis_-_GQ_consultation_on_awarding_grades_in_2021.pdf  See Q26.