Making a case for impact

In the UK, a new system for assessing the quality of research in higher education institutions is being implemented for 2014. The outcomes of  the Research Excellence Framework (REF), as it is known, will be used by the four UK higher education funding bodies to help inform their allocation of research funding to higher education institutions. The assessment also provides accountability of public investment in research. As you can imagine, this has made this assessment a very high priority for universities, particularly with a November deadline for submission fast approaching.

For the 2014 assessment, a new element has been incorporated in which universities must prepare and submit case studies that illustrate how research is having an impact beyond the academic community. The REF guidance gives some very specific definitions of what is considered impact in a variety of contexts including policy, environment, economic, and health and welfare. This means everyone is talking about impact, though admittedly with various levels of enthusiasm.

For many researchers it is hard to say the word impact without a mandatory eye roll or grinding of teeth. This frustration bordering on disgust with this six letter word seems to be rooted in several issues.

1. Impact doesn’t mean the same thing to all people

Impact does not mean the same to all people. For example, a researcher, who has landed a paper in the journal Nature, which received world-wide media coverage, and has subsequently been cited 1,000 times as it essentially revolutionised the way scientists think about a particular discipline, would be pretty happy and feeling like they had had a significant impact. However, they would be wrong. This is not considered impact in terms of REF unless you can prove that the world-wide media coverage somehow informed public debate or improved public understanding. Or if the scientists that used your revolutionary new approaches then went on to have some REF-defined impact. It’s not surprising that a researcher’s definition of impact is different when they have been raised in a “publish or perish” environment. Publications are what secure funding and faculty positions not necessarily impact. So its definition and, quite frankly, the priority placed on it, will differ from individual to individual.

2. It means asking people for things

The impact case studies need to be well evidenced. Fair enough. This is a very competitive game and those assessing them need to know that the impacts being claimed by the university can be substantiated. However, often this means making some big asks of relationships that researchers have fostered over a very long time. In some cases, this is easy; researchers work closely with government or with industry and they have a strong and mutually beneficial relationship. So, asking for some information isn’t stretching the relationship too far. However, for some case studies this information can be sensitive (e.g. pharmaceutical drug development or annual revenues from sales of a product) and it is more challenging to get the information at all, let alone without involving corporate lawyers and breaching non-disclosure agreements. For other case studies the research was conducted ten or more years ago and contacts with industry have moved on and relationships have dissolved making it very difficult to articulate an impact let alone evidence it.

3. It takes a lot of time

Putting together a case study takes a lot of time. It can involve drafting letters and then chasing them up with industry contacts, interviewing people, dredging through ten years of annual reports, following up on clinical trials in drug development, and reading stacks of government reports. This is just the research end of things. Then, it all has to be written down in a way that the layperson can understand. Basically, the end product needs to be a summary of the key findings of a body of research from the last 15 years, the impact it has had in the world, and the evidence of that impact; all within four pages and all in plain english. It’s harder than it sounds and this is where I have been able to help.

Writing impact case studies

I first started writing impact case studies for the Natural Environment Research Council (NERC) as they were looking for examples of how their research funding is having an impact (told you EVERYONE is talking about it). So, I developed a number of case studies from the University of Bristol that covered everything from economic reform in Vietnam to using nano-iron for environmental remediation. These case studies will be used by NERC to rationalise their own funding allocations and to showcase the research that they fund and they have been highlighted on the University of Bristol’s website also (see below).

I stumbled across some great discoveries whilst working on the case studies, including the fact that some research conducted by Prof Richard Evershed FRS became the international standard for maize oil adulteration. I secured quotes from Prof Sir John Beddington, then the Government’s Chief Science Advisor, for the contributions that Bristol volcanologists made during the Icelandic volcano eruptions in 2010. I loved the research and I learned a lot.

The success of these case studies have now led to contracts with multiple departments across numerous universities to help develop case studies for the REF2014. This is the mainstay of my work at the moment and it is diverse, interesting and challenging. However, as I eluded to in my last post, it is also work that is difficult to explain over a cocktail and it isn’t in the public domain. Due to the sensitive nature of the work, I have had to sign non-disclosure agreements with the universities, particularly as I am working for multiple institutions. So, for now, a lot of my writing remains unseen, but I don’t think this makes it any less important.

From volcanoes to Vietnam: sample case studies

I have written news pieces for the University of Bristol that showcase the NERC case studies that I prepared. As these are the only ones in the public domain, I thought I would share them. In terms of impact, the case studies reflect a whole range from developing impact to mature impact.

• Research in rare Sorbus reproduction informs conservation of the Avon Gorge
• Monitoring the world’s forgotten volcanoes from space
• Early season treatment of sheep could reduce the incidence of blowfly strike as climate warms
• From maize oil to murder: the diverse applications of sophisticated chemical analyses
• Cretaceous Sediment Investigations (CSI) of the Deep Biosphere
• Tracing the evolution of genes sheds light on the origins of life, including humans
• Research that marries molecular biology and palaeontology helps solve evolutionary puzzles
• Vietnamese reform influences design of government interventions
• Making sense of uncertainty in complex systems
• Model that shakes entire cities improves earthquake risk assessment
• Predictive model used to assess quality of carbonate reservoirs
• New approach to nano-iron composite materials improves environmental remediation
• Volcanological input reduces uncertainty surrounding volcanic ash forecasts
• Conker Tree Science: how small investments can reach thousands
• Predictive model serves as blueprint for the flood risk management industry