International Journal of Nursing Studies 52 (2015) 1–3
Contents lists available at ScienceDirect
International Journal of Nursing Studies journal homepage: www.elsevier.com/ijns
Guest Editorial
Research assessment: The impact of impact
We grow no food on campus, so like every poet, priest or potter. . ., we must explain why we have faith in the usefulness of what we do provide (Gray, 2012, p. 33) For almost thirty years publically funded research in the UK has been assessed for quality in a series of research assessment exercises. The most recent one is called the Research Excellence Framework (HEFCE, 2014). Such exercises are carried out in many countries across the world. These include Finland, Norway, Sweden, Denmark, Holland, Italy, New Zealand, Australia, Romania, Hong Kong, Germany and most recently the Czech Republic. The results are mainly used to inform the allocation of research funding and provide accountability for tax payers’ money. Every four or five years universities in these countries make submissions to their government research funding bodies. Such submissions are mainly composed of publications and other outputs of the research process. In addition, some include information pertaining to the research environment including research strategy, infrastructure, funding, PhD completions, staffing, esteem, and collaborations. For the first time the UK’s Research Excellence Framework (REF) has assessed the impact of research. This is not surprising since politicians and others are asking universities to outline what has been achieved from publicly funded research. It is also not surprising that other countries are watching the results of this with great interest and many are already planning to assess impact as part of their future research assessment exercises (e.g. Sweden and Hong Kong). Increasingly, these governments believe it is not unreasonable to ask those whose research work is undertaken at public expense to account for and provide some evidence of their activities and outcomes. Incidentally, the pilot for Excellence in Research for Australia (ERA) exercise attempted to assess impact but this did not find its way into the final ERA process (McKenna et al., 2012). REF and research impact To meet the REF criteria, the impact must have occurred outside of academia and influence areas such as, the http://dx.doi.org/10.1016/j.ijnurstu.2014.11.012 0020-7489/ß 2014 Published by Elsevier Ltd.
economy, society, quality of life, culture and health care. To illustrate these impacts, universities have had to produce impact case studies; they also have to show how they support the development of impact from their institutions’ research activities. Research impact accounts for 20% of the total REF profile. When you consider that the results of the exercise enable the allocation of over £2 Billion each year (for five years) to UK universities, this is not an inconsiderable percentage. It would seem that this will be increased to 25% or more for the next REF, probably in 2020. Impacts are assessed for their reach and significance; here reach does not necessarily refer to geographic reach. Another important thing to remember is the quality of the underpinning research. Obviously, research impact that is built of poor quality research would be questionable and unlikely to have significant reach and significance. Furthermore, while impact can indeed occur contemporaneously with the underpinning research, it can often take many years for research to have benefit. REF guidelines allowed the research to have been undertaken as far back as 1993 as long as the impact of that work occurred between Jan 2008 and July 2013. Nursing and research impact For nursing research, impact could relate to improved health or welfare outcomes, enhanced professional standards, ethics, guidelines or training, improved quality, accessibility or efficiency of a public service, or changes in professional practice or the development of a new incontinence product. It is not unreasonable to think that identifying research impact for the nursing profession should be easy; after all we are an applied discipline and it should be possible to trace how the findings of research studies have changed practice or policy. But consider the following questions; is writing a best-selling nursing book impact? Is speaking about your research findings at a conference, on radio or on television impact? Is providing a continuing professional
2
Editorial / International Journal of Nursing Studies 52 (2015) 1–3
education (CPD) course to the staff of a nursing home company impact? Is showing that a particular procedure should no longer be undertaken impact? I will attempt to answer these questions but before I do readers should remember two key messages: dissemination is not impact; and to have impact something must change positively, there should be some benefit. Take the question about the best-selling book. OK, so the publisher’s profits may have been enhanced but did anything really change as a result of people buying or reading the book and if so, how do you evidence that? The same applies to the second question; how do you know if anything has changed as a result of you sharing your research findings through conferences or through the media? Similarly, how do you know if there has been any benefit from people attending a CPD course in your school? Providing evidence of impact is a crucial but difficult task. So, if you surveyed a sample of your readers, viewers or listeners and found that they had changed their practice as a result of your research, this is a good start to developing your evidence base. What if a large nursing home company changed their processes, policies or practices as a result of your research-led CPD course? This too would provide the basis for a significant impact case study. From time to time, research findings allow us to stop some long established practice and make care safer for patients. No reasonable person would dispute the benefit of this and therefore this is another example of research impact that could have outstanding reach and significance. For example, the reach could be all patients who have dementia and the significance could be evidence of reduced morbidity and improved quality and safety of care. Therefore, impact case studies should not simply reflect the widespread dissemination of your research results or extol how good you or your research team are. Rather they need to demonstrate that all important reach and significance. They should also show a clear link between the original research and the subsequent impact and provide clear verifiable evidence of the impact produced. In other words there should be a good evidence trail to strongly link the impact to the underpinning research and this should be confirmable by independent sources. The political need for universities to show impact In most countries universities rely on public funding for their teaching and research activities. This reliance is lessening with the advent of student fees, and a greater emphasis on philanthropy, industrial funding of research and other income diversification measures. Nonetheless, universities have to prove their value to society and the economy. This is the philosophical basis for the assessment of research impact. It is also indicative of the trend in recent years for vice chancellors and politicians to emphasise the importance of universities for the economic and social regeneration specifically or for wealth creation generally. A commonly held view is that ‘‘Institutions should encourage the seamless transfer of ideas from the research lab and lecture hall to the marketplace because we recognise that research drives
innovation and innovation feeds productivity, which fuels the creation of wealth and economic growth.’’ (Hamdullahpur, 2012, p. 30). However, is the pursuit of such impact really the reason why universities exist or why students go to university? Are not research and teaching the core businesses of universities? Also, for much curiosity driven research is it possible to predict the impact at the outset of the project? What about the importance of universities for character forming and developing individual potential? Collini (2012) asserted that society does not educate the next generation in order for them to contribute to the economy. It educates them in order that they should extend and deepen their understanding of themselves and the world, acquiring in the course of this form of growing up, kinds of knowledge and skill which will be useful in their eventual employment, but which will no more be the sum of their education than that employment will be the sum of their lives. It is about extending human understanding, through open ended enquiry (p.239). It is my view that nurses undertake an academic career and nursing students undertake a particular course to give optimum care to patients and further their knowledge and skills of nursing, not to contribute to the social, cultural or economic regeneration of the country. This may occur tangentially as a result of better public health or the recipients of care having shorter lengths of time away from employment. However, the research impact agenda is with us and is not going to go away. In fact it will play a greater role in further research assessment exercises and in research grant applications. Furthermore, I predict that it will spread internationally. For nursing, it is laudable that the REF definition of impact captures non-economic elements such as benefits to society, culture, health care, quality of life. Be prepared There are a number of things that nurses should do to prepare for future assessments of research impact. These include: knowing the definition of impact and the types of evidence required to show it; educate end users such as clinicians, companies and health services on impact, why we need their involvement, what that involvement entails; involve patients, carers, and health service managers in research teams so that they can advise on how the study can achieve impact; develop mechanisms for supporting research impact at institutional/unit level; and identify impact case studies early and have them critically assessed against criteria by research users and lay readers. It is also a good idea to plan a pathway to impact when you are writing your research proposal. It is an example of how widespread the impact agenda is that many research funders require you to do this. Also, it is a good idea to collect evidence for impact on an ongoing basis. Doing this retrospectively is difficult and time consuming. There has been much criticism around the assessment of impact. Nonetheless, it has raised people’s conscience about its importance. While most nurse researchers would wish that their research would change things, many carried out a research study, published and presented the results and moved on to their next funded study. The research report
Editorial / International Journal of Nursing Studies 52 (2015) 1–3
3
gathering dust on a library shelf is a truism. It is almost as if they thought a person on a white horse would ride up and implement the findings, but that was of little interest to them. Now, not only do they think about impact before the study is funded, they think of it during the study and after it is completed. Increasingly, impact is moving away from part of a government research assessment exercise to being an integral part of being a researcher and doing research. Soon there will be no reasons left why researchers do not consider research impact – only excuses!
McKenna, H.P., Daly, J., Davidson, P., Duffield, C., Jackson, D., 2012. RAE and ERA – spot the difference. Int J Nurs Stud 49 (4), 375–377.
References
1 Editor’s note: Professor McKenna is Chair of UK Government’s Research Excellence Framework Panel for Nursing, Allied Health Professions, Pharmacy and Dentistry.
Collini, S., 2012. What are Universities For? Penguin Books, London. Gray, N., 2012. Letters. The Times Higher Education No. 2059,, p. 33. Hamdullahpur, F., 2012. The Times Higher Education No. 2057,, p. 30.
Hugh P. McKenna CBE, PhD, B.Sc(Hons), RMN, RGN, RNT, DipN(Lond), AdvDipEd, FFN RCSI, FEANS, FRCN, FAAN1 Pro-Vice-Chancellor, Research and Innovation, Ulster University, UK
Contents lists available at ScienceDirect
International Journal of Nursing Studies journal homepage: www.elsevier.com/ijns
Guest Editorial
Research assessment: The impact of impact
We grow no food on campus, so like every poet, priest or potter. . ., we must explain why we have faith in the usefulness of what we do provide (Gray, 2012, p. 33) For almost thirty years publically funded research in the UK has been assessed for quality in a series of research assessment exercises. The most recent one is called the Research Excellence Framework (HEFCE, 2014). Such exercises are carried out in many countries across the world. These include Finland, Norway, Sweden, Denmark, Holland, Italy, New Zealand, Australia, Romania, Hong Kong, Germany and most recently the Czech Republic. The results are mainly used to inform the allocation of research funding and provide accountability for tax payers’ money. Every four or five years universities in these countries make submissions to their government research funding bodies. Such submissions are mainly composed of publications and other outputs of the research process. In addition, some include information pertaining to the research environment including research strategy, infrastructure, funding, PhD completions, staffing, esteem, and collaborations. For the first time the UK’s Research Excellence Framework (REF) has assessed the impact of research. This is not surprising since politicians and others are asking universities to outline what has been achieved from publicly funded research. It is also not surprising that other countries are watching the results of this with great interest and many are already planning to assess impact as part of their future research assessment exercises (e.g. Sweden and Hong Kong). Increasingly, these governments believe it is not unreasonable to ask those whose research work is undertaken at public expense to account for and provide some evidence of their activities and outcomes. Incidentally, the pilot for Excellence in Research for Australia (ERA) exercise attempted to assess impact but this did not find its way into the final ERA process (McKenna et al., 2012). REF and research impact To meet the REF criteria, the impact must have occurred outside of academia and influence areas such as, the http://dx.doi.org/10.1016/j.ijnurstu.2014.11.012 0020-7489/ß 2014 Published by Elsevier Ltd.
economy, society, quality of life, culture and health care. To illustrate these impacts, universities have had to produce impact case studies; they also have to show how they support the development of impact from their institutions’ research activities. Research impact accounts for 20% of the total REF profile. When you consider that the results of the exercise enable the allocation of over £2 Billion each year (for five years) to UK universities, this is not an inconsiderable percentage. It would seem that this will be increased to 25% or more for the next REF, probably in 2020. Impacts are assessed for their reach and significance; here reach does not necessarily refer to geographic reach. Another important thing to remember is the quality of the underpinning research. Obviously, research impact that is built of poor quality research would be questionable and unlikely to have significant reach and significance. Furthermore, while impact can indeed occur contemporaneously with the underpinning research, it can often take many years for research to have benefit. REF guidelines allowed the research to have been undertaken as far back as 1993 as long as the impact of that work occurred between Jan 2008 and July 2013. Nursing and research impact For nursing research, impact could relate to improved health or welfare outcomes, enhanced professional standards, ethics, guidelines or training, improved quality, accessibility or efficiency of a public service, or changes in professional practice or the development of a new incontinence product. It is not unreasonable to think that identifying research impact for the nursing profession should be easy; after all we are an applied discipline and it should be possible to trace how the findings of research studies have changed practice or policy. But consider the following questions; is writing a best-selling nursing book impact? Is speaking about your research findings at a conference, on radio or on television impact? Is providing a continuing professional
2
Editorial / International Journal of Nursing Studies 52 (2015) 1–3
education (CPD) course to the staff of a nursing home company impact? Is showing that a particular procedure should no longer be undertaken impact? I will attempt to answer these questions but before I do readers should remember two key messages: dissemination is not impact; and to have impact something must change positively, there should be some benefit. Take the question about the best-selling book. OK, so the publisher’s profits may have been enhanced but did anything really change as a result of people buying or reading the book and if so, how do you evidence that? The same applies to the second question; how do you know if anything has changed as a result of you sharing your research findings through conferences or through the media? Similarly, how do you know if there has been any benefit from people attending a CPD course in your school? Providing evidence of impact is a crucial but difficult task. So, if you surveyed a sample of your readers, viewers or listeners and found that they had changed their practice as a result of your research, this is a good start to developing your evidence base. What if a large nursing home company changed their processes, policies or practices as a result of your research-led CPD course? This too would provide the basis for a significant impact case study. From time to time, research findings allow us to stop some long established practice and make care safer for patients. No reasonable person would dispute the benefit of this and therefore this is another example of research impact that could have outstanding reach and significance. For example, the reach could be all patients who have dementia and the significance could be evidence of reduced morbidity and improved quality and safety of care. Therefore, impact case studies should not simply reflect the widespread dissemination of your research results or extol how good you or your research team are. Rather they need to demonstrate that all important reach and significance. They should also show a clear link between the original research and the subsequent impact and provide clear verifiable evidence of the impact produced. In other words there should be a good evidence trail to strongly link the impact to the underpinning research and this should be confirmable by independent sources. The political need for universities to show impact In most countries universities rely on public funding for their teaching and research activities. This reliance is lessening with the advent of student fees, and a greater emphasis on philanthropy, industrial funding of research and other income diversification measures. Nonetheless, universities have to prove their value to society and the economy. This is the philosophical basis for the assessment of research impact. It is also indicative of the trend in recent years for vice chancellors and politicians to emphasise the importance of universities for the economic and social regeneration specifically or for wealth creation generally. A commonly held view is that ‘‘Institutions should encourage the seamless transfer of ideas from the research lab and lecture hall to the marketplace because we recognise that research drives
innovation and innovation feeds productivity, which fuels the creation of wealth and economic growth.’’ (Hamdullahpur, 2012, p. 30). However, is the pursuit of such impact really the reason why universities exist or why students go to university? Are not research and teaching the core businesses of universities? Also, for much curiosity driven research is it possible to predict the impact at the outset of the project? What about the importance of universities for character forming and developing individual potential? Collini (2012) asserted that society does not educate the next generation in order for them to contribute to the economy. It educates them in order that they should extend and deepen their understanding of themselves and the world, acquiring in the course of this form of growing up, kinds of knowledge and skill which will be useful in their eventual employment, but which will no more be the sum of their education than that employment will be the sum of their lives. It is about extending human understanding, through open ended enquiry (p.239). It is my view that nurses undertake an academic career and nursing students undertake a particular course to give optimum care to patients and further their knowledge and skills of nursing, not to contribute to the social, cultural or economic regeneration of the country. This may occur tangentially as a result of better public health or the recipients of care having shorter lengths of time away from employment. However, the research impact agenda is with us and is not going to go away. In fact it will play a greater role in further research assessment exercises and in research grant applications. Furthermore, I predict that it will spread internationally. For nursing, it is laudable that the REF definition of impact captures non-economic elements such as benefits to society, culture, health care, quality of life. Be prepared There are a number of things that nurses should do to prepare for future assessments of research impact. These include: knowing the definition of impact and the types of evidence required to show it; educate end users such as clinicians, companies and health services on impact, why we need their involvement, what that involvement entails; involve patients, carers, and health service managers in research teams so that they can advise on how the study can achieve impact; develop mechanisms for supporting research impact at institutional/unit level; and identify impact case studies early and have them critically assessed against criteria by research users and lay readers. It is also a good idea to plan a pathway to impact when you are writing your research proposal. It is an example of how widespread the impact agenda is that many research funders require you to do this. Also, it is a good idea to collect evidence for impact on an ongoing basis. Doing this retrospectively is difficult and time consuming. There has been much criticism around the assessment of impact. Nonetheless, it has raised people’s conscience about its importance. While most nurse researchers would wish that their research would change things, many carried out a research study, published and presented the results and moved on to their next funded study. The research report
Editorial / International Journal of Nursing Studies 52 (2015) 1–3
3
gathering dust on a library shelf is a truism. It is almost as if they thought a person on a white horse would ride up and implement the findings, but that was of little interest to them. Now, not only do they think about impact before the study is funded, they think of it during the study and after it is completed. Increasingly, impact is moving away from part of a government research assessment exercise to being an integral part of being a researcher and doing research. Soon there will be no reasons left why researchers do not consider research impact – only excuses!
McKenna, H.P., Daly, J., Davidson, P., Duffield, C., Jackson, D., 2012. RAE and ERA – spot the difference. Int J Nurs Stud 49 (4), 375–377.
References
1 Editor’s note: Professor McKenna is Chair of UK Government’s Research Excellence Framework Panel for Nursing, Allied Health Professions, Pharmacy and Dentistry.
Collini, S., 2012. What are Universities For? Penguin Books, London. Gray, N., 2012. Letters. The Times Higher Education No. 2059,, p. 33. Hamdullahpur, F., 2012. The Times Higher Education No. 2057,, p. 30.
Hugh P. McKenna CBE, PhD, B.Sc(Hons), RMN, RGN, RNT, DipN(Lond), AdvDipEd, FFN RCSI, FEANS, FRCN, FAAN1 Pro-Vice-Chancellor, Research and Innovation, Ulster University, UK
