I’d rather lost track of where things had got to with the Office for Students (OfS) requirements in relation to the retention of student work. There have been real concerns in universities and colleges in England for some time about the implementation of the part of the OfS B4 Condition which deals with this. It started out as a requirement that all assessed work for all students needed to be retained for five years after graduation – a hugely bureaucratic and costly exercise – but has since been moderated in guidance published in 2024 to say ‘appropriate’ records, which might be just a sample, rather than everything.

So, are we getting somewhere a bit more sensible with this then?

The heart of the matter

I wrote at the beginning of 2024 for Wonkhe on the problems and costs associated with the B4 condition which includes the following specification:

335U. As part of its approach to assessing compliance with this condition, the OfS is likely to need access to students’ assessed work, including for students who are no longer registered on a course. A provider is therefore expected to retain appropriate records of students’ assessed work for such regulatory purposes for a period of five years after the end date of a course. 

This is further amplified by a subsequent note in the same document about the consequences of absence of such records:

335W. The absence of records of students’ assessed work may lead the OfS to make negative inferences about a provider’s compliance and/or may result in the OfS taking targeted regulatory action to address the risk that it is unable to monitor compliance and regulate effectively.

The gargantuan exercise in records organisation and retention which this would entail is to allow for the possibility, however remote, that an inspector may, at some point, want to assess the judgements made by academics in marking assessments relating to a particular course at an institution. In other words, the English sector would be required to retain millions of pieces of assessed work, at massive expense, on the off-chance that the OfS had a concern about standards on a particular course. My estimate last year was that this approach, if followed as originally anticipated, would cost the English sector significantly more than £500m in set up and running costs over the next five years. This would be, by any definition, a disproportionate burden for a financially challenged sector to bear.

Responding to feedback

Whilst the B4 Condition has not changed from that set out in the OfS Regulatory Framework as updated and published in November 2022, there has since been some supplementary guidance issued.

Published in March 2024 this guidance followed some feedback from the sector (broadly of the form ‘you cannot be serious!’) about the Condition and a subsequent working group involving sector representatives including individuals from Universities UK, the Association of Heads of University Administration, Jisc and the Academic Registrars’ Council. I was one of those involved and was pleased to be able to contribute some thoughts about the challenges involved in the requirements of the B4 Condition. I do have to commend the openness of those from the OfS who participated too – there was undoubtedly a seriousness about the engagement and a willingness to listen.

The press statement accompanying the supplementary guidance stresses both institutional autonomy and sampling:

The supplementary guidance emphasises the latitude institutions have to make their own judgements about the retention of work and provides further guidance about the types of record that are sufficient for the OfS’s regulatory purposes. It also provides information about the factors institutions should consider when determining and sampling appropriate records of assessments for retention in their particular context.

Jean Arnold, Deputy Director of Quality for the Office for Students (OfS), said:

‘We have listened to the sector’s concerns about our existing guidance for conditions B4 and B5 and worked closely with a group of sector representatives to develop the new guidance published today. I’m grateful for the input from this group. Informed by their feedback, this new guidance establishes that a representative sample of work may be appropriate, rather than retaining all work. Individual universities and colleges can also decide what assessed work it is appropriate for them to keep.’

The OfS is at pains though to stress the status of the new document as unequivocally being in the form of ‘guidance.’ It is up to institutions to interpret it and implement the guidance as they see fit. In other words, universities and colleges have to decide for themselves 

what may constitute ‘appropriate records’ for retention and should not understand this to mean ‘all records for all students in all contexts’. The use of ‘appropriate’ in the guidance deliberately gives a provider space to make reasonable judgements about what to store and what not to store.

Critically though, in making this judgement, institutions do need to record the basis of their decisions in relation to appropriateness and retention.

Take away the sculptures

In explaining why the B4 Condition specifies the retention of student work, the OfS offers the following rationale:

Providers registered with the Office for Students must, among other requirements:

• assess students effectively using assessments that are valid and reliable
• grant students with credible awards
• make sure that they follow standards recognised by the higher education sector when they do so.

We have set out these requirements in our conditions of registration B4 and B5. 

When we consider whether a provider is complying with these conditions, students’ assessed work is likely to be relevant evidence.

The OfS needs access to primary evidence – in this circumstance the assessments themselves – to reach a judgement about a provider’s compliance with the requirements in conditions B4 and B5, including those listed above. 

The view in relation to certain forms of primary evidence though has changed and the other good news for institutions which emerged from the discussions in the OfS working group with sector representatives is that all of those complicated physical assessed pieces of work, ie all the things which aren’t essays or exams or the like, do not need to be stored. For example, in relation to “Sculptures, textiles, fashion and artwork” records might instead include marks and feedback, a photo or video and the assessment brief

So, no need for institutions to lease additional warehouse space for all of those sculptures and art installations. Phew.

Sample this

Providing representative samples of student work is very much the approach which has been followed in higher education in more recent years, essentially as a pragmatic response to a mass system. In the days of a much smaller learner population external examiners often inspected all student work associated with a course (some perhaps more diligently than others) and, in principle, this remains an option although the sheer scale of some programmes will make this simply impractical. Sampling therefore is a more manageable approach whereby examples are taken from the range of essays or exam scripts to show the full range of marks awarded, from top to bottom and ranges in between.

Similar arrangements obtained under the Teaching Quality Assessment regime where substantial samples would be provided for assessors including the full range of work for a sample of individual students to give a rounded picture.

So, what is the OfS looking for in its samples?

A provider may decide that retaining a sample of assessed work would be an appropriate record.

A sample of retained work should be sufficiently large and comprehensive for the OfS to draw reliable conclusions about the provider’s provision. 

However, a large and comprehensive sample would still represent a major logistical issue for universities and colleges and the scale of the effort required to retain even a smaller appropriate sample of assessed work will be a non-trivial exercise.

The whole approach to retaining student work for an extended period, regardless of sample size, remains, in my view, fundamentally flawed. It assumes that by looking at a range of assessed work that judgements about the standards set by academics and achieved by students can be reached. But on this basis they cannot. You can’t assess standards or unexplained grade inflation from this evidence base alone – inspectors will be trying to second guess academic judgment or replicate it and that simply is not possible without the full context of every other dimension of the assessment process. The retained material itself therefore is never going to allow the expected judgements to be made – it essentially amounts to re-marking by external inspectors and is simply not a viable strategy for assessing academic standards. Looking back more than 30 years ago it is worth noting that CNAA, when it was still regulating quality and standards in part of the UK sector, made no such demands in relation to archives of retained student assessed work.

Not heard it all B4

The B4 Condition relating to the retention of assessed work has been a very concerning issue and one that has risked imposing huge additional costs on institutions at a time when they could ill afford such expenditure. The OfS does deserve credit for engaging with the sector on this matter, responding to these concerns and making the shift from the requirement to retain all assessed work to sampling. It has also successfully downplayed the significance of all of this over the past year or so, following the issuing of the guidance (indeed there is no mention of it in the October 24 OfS Insight Brief on the findings from a range of recent quality assessments) and the regulator does therefore seem to have backed away from taking a hard line on retention of assessed work.

But prespecifying even a sample  for the entire sector in England would still be hugely costly and also disproportionate in that over 99% of the material retained would never, ever be inspected. Therefore the other element of the OfS guidance has to take precedence here – institutions will have to make their own judgments. It will fall to each university and college to demonstrate how they are assuring and maintaining academic standards with one of the means they may choose to do this being to provide a sample of retained work. They will then have to document and justify their decisions.

So, everyone will make their own decisions on this. And it is good to see that the OfS really does seem to have moved on from what was a really unhelpful, inappropriate and massively costly demand.