The University of Liverpool is a signatory to the San Francisco Declaration on Research Assessment (DORA) which outlines several recommendations on how to improve research assessment.
As an institution, we have committed to:
- being explicit about the criteria used to evaluate academic activity;
- recognising the value of all relevant research outputs (for example publications, datasets, software and arts practice outputs), as well as other types of contributions, such as for example training early-career researchers, influencing policy and practice, and research impacts;
- avoiding the use of journal-based metrics as individual or surrogate measures of the quality of research in decision-making for funding, appointing and promoting staff, and instead judging outputs based on their intrinsic merit.
Reporting of Research Publications Metrics
The following text should be used to accompany any report documents that provide bibliometric analysis of the citation impact of our researchers’ publications:
"Bibliometric indicators of the citation impact of researchers’ publications should support, not supplant, expert qualitative evaluation of research performance. The limitations of quantitative indicators should be understood, in particular, no measures specifically related to venue of publication, such as Impact Factor or Citescore, should be used to inform judgements. Assessors should always be aware that data from different sources (e.g. Web of Science, Scopus, Google Scholar, Dimensions) will always provide different information."
This ensures that those reading the report documents are aware of the limitations of bibliometric analyses and the context associated with the use of quantitative indicators of citation impact in research assessment. The statement reflects the principles of the institutional policy on the responsible use of metrics.
How did we get here? Institutional Policy on the Responsible Use of Metrics
Development of an institutional policy on the responsible use of metrics and an associated implementation plan was undertaken in consultation with our research community over Spring to Autumn 2020. The Policy was approved by the University governance structure and formally approved by Senate in March 2021.
You can access the Policy for the Responsible Use of Metrics in Research Assessment via the University Policy Centre.
The Policy sets out the University’s approach to the use of metrics in research assessment, and is high-level, spanning the whole institution. This policy complements the Code of Practice for the Evaluation and Selection of Research Outputs.
Equality Impact Assessment
An Equality Impact Assessment was carried out in February 2021 following consultation. This happened in advance of the policy being formally approved. You can view the EIA screening for RM policy here.
The Policy directs staff involved in recruitment and assessment activities only to use metrics in conjunction with peer review and never in isolation, and to avoid the use of metrics related to venue of publication (i.e. journal-based metrics) or individual metrics that are influenced by length of career. In this way the policy should support the objective of making improvements in the recruitment of some underrepresented groups by helping to avoid indirect discrimination of researchers: who are more likely to have had career breaks (e.g. female researchers, disabled researchers); whose careers may have largely been outside academia; near the start of their academic career; whose disciplines do not rely heavily on journal publication; or who are from countries with traditionally low rates of publishing in certain journals.
Implementation
The policy is fully implemented in the University’s working practices. We recognise that the research landscape differs between disciplines and that practical guidelines might be tailored to suit each Faculty/School/Department. Actions will always be needed to facilitate the responsible use of metrics by staff at the University of Liverpool. These include procedural monitoring where required and new or improved resources to aid responsible research assessment going forward. The most important actions, however, consist of training for all relevant staff to guide them through the approaches, aid understanding and facilitate successful implementation. Please contact the Research Publications Metrics Officer if you need any assistance.
The actions span four main categories to support unified adoption of the principles:
- Leadership
- Recruitment
- Internal staff assessment (incl. Personal Development Reviews and promotion)
- Training
The plan was completed by July 2021, published internally, and widely communicated. The implementation started in August 2021 and occurred in two phases:
- The first phase focused on raising staff awareness and understanding of the Policy. All relevant staff will receive training on how to implement the Policy principles in hiring, career progression, promotion and funding decision.
- Once the new processes were established internally, all remaining external-facing processes were introduced (e.g. related to job advertisements). This is to avoid a conflict between the expectations of external candidates and University staff practices.
Since 2022, monitoring and training continued.
Project Governance
The project is sponsored by the Associate Pro Vice Chancellor for the Research Environment and Postgraduate Research, and led by staff in the Open Research Team.
Back to: Open Research