One step in the right direction, conclude Peter McOwan and Charlotte Thorley
November 2013 will see a short collective sigh of relief across UK universities, as the submissions for the 2014 Research Excellence Framework (REF) are made. It will be around a year later that the results are available, determining how research funding from government will be distributed to UK tertiary education institutions.
As in the past, a large proportion will depend on the quality of research papers published, and how panels of experts assess the standing of these pieces of work. This time, however, something new is on the table: the REF impact statements.
Generally it’s fair to say that academics are still unsure of REF and Impact. After all, it’s a new set of largely untested rules. However, impact has the potential to be useful for those trying to establish and reward a culture of public engagement (PE) activity. It provides both a carrot (potential funding) and a stick (provision of robust evidence, in other words more data to collect and forms to fill in).
Scope of impact
The REF guidelines  state: Assessment criteria: impact: ‘The main panel welcomes case studies describing impacts that have provided benefits to one or more areas of culture, the economy, the environment , health, public policy and services, quality of life, or society, whether locally, regionally, nationally or internationally.’
This definition clearly includes PE but critically, only PE based on specific and citable research papers from the individuals involved.
General PE activity, such as promoting and undertaking dialogue on the subject area, is not counted. This is worrying, especially if it leads universities to reframe their PE activity to the metrics instead of the interests of teachers and students.
The problem is that the rules for assessment are not entirely clear, and as it is a new system there is no precedent to indicate exactly what the assessors are looking for.
Though a number of exemplar case studies were undertaken, reviewed and rated, and the results made public, many institutions will choose to play it safe, reducing their risk. They will put in case studies where direct impact can be clearly demonstrated, for example in industry, where research paper A leads to the production of widget B.
In medicine and science this may be a safer bet. However, it acts as a disincentive to wider PE activity, as it encourages researchers to focus on doing PE around their own research papers rather than more generally on the subjects they teach.
In the humanities the needs and delivery models of impact are frequently very different, and many feel we have a long way to go in understanding the best ways of reporting, documenting, evaluating and sharing practice from current PE work .
Just doing more activity isn’t necessarily a good thing right now. Quality is the key. Evidence collection and activities that are thoughtful, have the audience in mind and consider wider economic and social benefits should be what makes a good PE REF case study, and perhaps as importantly a more transparent and engaged university system.
However if we encourage staff to do more PE, and they don’t make it into impact statements, we run the risk of discouraging them.
Regardless of REF, institutions need to ensure that they value PE work more generally in promotions criteria, workload allocation and appraisal, and by finding ways to support and develop practitioners, and sharing best practice.
This time we learn; next time we’ll be more ready and know what’s what. Unless of course, the next REF includes as big a change as this one – then we’ll have to get our crystal balls out again to make sure we’re ahead of the game.
Note: This article reflects the authors’ personal views as experienced PE practitioners.