Measuring societal impact can be a bit too much
The Future of Research and Development: A Prime Minister’s Address on Government Research and Innovation in the English-French Parliamentary Report on Hunt’s Autumn Statement
The UK finance minister, Jeremy Hunt, has said that the economic crisis will not harm ambitious plans for research spending, much to the relief of researchers.
But during today’s address on government spending — called the Autumn Statement — Hunt, the chancellor of the Exchequer, told parliament that he would protect the entire UK research budget because cutting it would be a “profound mistake”. He added that the government will invest £20 billion (US$24 billion) per year in science by the 2024–25 financial year — a commitment made by the government of former prime minister Boris Johnson that was left hanging in the balance when he resigned in July. Truss remained tight-lipped on science spending during her 44-day leadership.
Stephanie Smith, head of policy at the Russell Group, which represents 24 leading UK research universities, tweeted: “Big sigh of relief” at the news the science budget would be protected.
The decision was welcomed by Anne Johnson, president of the Academy of Medical Sciences. She warned that there could still be problems with the research. Budgets are being squeezed by inflation in real terms and we must protect collaborative research between UK researchers and their international partners.
Paul Nurse is a geneticist who runs the Francis Crick Institute in London.
Sunak was the prime minister when Johnson was chancellor. This change of leadership was not known in what direction the future of science spending would take.
Hunt has said that he will spend 2.5% of the country’s GDP on research and development by 2027. The government has already met this target, according to the UK Office for National Statistics. This has something to do with the new way that research and development spending is calculated.
The website for the government department that oversees science spending, the Department for Business, Energy and Industrial Strategy (BEIS), lists both Freeman and Ghani as “minister of state”, with Ghani also listed as “minister for science and investment security”. A spokesperson for BEIS could not say who has overall responsibility for science, stating that the ministerial portfolios “are not formally confirmed”.
Science has always been a tool in international relations. But hard-nosed political initiatives must not be allowed to crowd out the kind of ‘no strings’ cooperation that leads to discovery and invention, and that is needed to solve global challenges such as biodiversity loss or pandemic prevention. There is a danger of science getting caught up in political projects because of the current tension between China and the United States. Researchers need to be vigilant, recognize the risks and stand firm when necessary.
Overall, the past year has been a mixed bag for cooperation towards common global goals. The deal reached at COP15 was a good one but there were problems to be worked out. There was some movement on the important issue of loss and damage from finance transfers from high- to low-income countries, although there was little progress on decarbonization.
The COVID-19 Pandemic gave textbook examples of nations working in their own interests. The governments of a relatively small number of wealthy countries had already bought and hoarded vaccines from pharmaceutical companies in Europe and the United States (Nature 607, 211–212; 2022). Nature played a small part in opposing the international campaign urging the sharing of vaccines, therapies and intellectual property. Had they heeded the words of World Health Organization chief Tedros Adhanom Ghebreyesus, who repeatedly stressed that “no one is safe until everyone is safe”, more people in low- and lower-middle-income countries could have been vaccinated more quickly, and fewer lives lost.
The invasion of Ukraine, meanwhile, has halted all official research cooperation between Russia and Europe and the United States. The chairmanship of the intergovernmental forum for cooperation in the region was held by Russia this year but some of its work has been suspended because other countries are not taking part. Europe and the US have become interested in supporting Ukraine’s research community at the same time.
Next year is likely to see further reductions in US–China scientific collaboration after two decades of growth in science and technology collaborations. There have been tensions on both sides for a while. In 2018, the administration of former US president Donald Trump launched its ‘China Initiative’, a poorly thought-through surveillance programme to counter what the government regarded as intellectual-property theft and economic espionage. This led to the investigation of many researchers from China or with Chinese heritage and resulted in completely innocent people being arrested and brought to trial. The initiative ended in February, and the damage had already been done.
Rivalry between the two countries is also being played out in trade and technology, with the era in which powerful countries encouraged open markets looking to be at a turning point. The United States is restricting sales by US companies (and non-US companies that use US technology) to China of the types of microchip that are used in artificial intelligence and supercomputing. It has placed restrictions on US citizens and residents working for Chinese technology companies. African countries are being encouraged to become an alternative base for technology cooperation due to its desire to partner with itself instead of China. China retaliated against the US move by lodging a case with the WTO, arguing that the US move is in contradiction to free- trade agreements between the countries.
The world is clearly in what economist Pedro Conceição calls “a new uncertainty complex”, with an ongoing pandemic, war, climate risks and associated economic shocks. We’re likely to see more instances of countries raising trade barriers and using technology to protect their economies, and more instances of nations using science and technology towards foreign-policy objectives.
Governments have responsibilities to ensure the integrity of international cooperation in science-based policymaking. In the case of climate change, that responsibility now falls to the United Arab Emirates, which will take over the presidency of the next climate summit, COP28.
There is a big gap between the intention to measure broad impacts and actually creating a system to do so. Paul Nightingale said that the evaluation culture hasn’t hit science to the extent that it might. The government has a good idea of what it will get from its investment when it invests in roads, education or welfare. Science is the only area of significant public funding where there is no proper impact evaluation.
In 2016, the Netherlands Association of Universities of Applied Sciences (NAUAS) took a broad view of scientific contributions in a document called Onderzoek met Impact (Research with Impact) (see go.nature.com/3wtq3). According to the report, the ten areas that university research should affect are health and vibrancy, education and talent development, societal resilience, sustainable transportation, sustainable agriculture and responsible business. The NAUAS wanted to prove the worth of research, but no methodology for actually tracking impact was included in the document.
The impact of research on governments, medical societies or other institutions would probably be over-simplified by such a score. They compared the policies of the universities with the scores in the impact database. Their analysis suggested that institutions with the most policy citations tended to have been given higher scores by peer reviewers in the REF, but the correlation wasn’t especially strong. Szomszor says that a score derived from a policy database is likely to fall short. Including some form of peer review “that would allow you to judge the worth of policy influence” would be preferable.
Innovations are essential for science and society — and it’s becoming increasingly important to identify the most consequential projects and researchers, Wunsch-Vincent says. But the simplest metrics for measuring innovation might not be the most illuminating. “We’re doing particularly well at measuring intermediate innovation outputs, such as patents or citations,” he says. There are serious drawbacks, but they are not bad. Just as researchers often cite themselves in their publications to boost their own impact factor, inventors often cite their own work in patent applications. He says that it is common for inventors in certain countries to cite a single patent from that particular country to avoid international copyright disputes.
Alternative metrics, or ‘altmetrics’, attempt to track the reach of research through social media, news sites and other outlets, rather than academic citations. Bornmann says they have fallen short of their goal when it comes to telling the full story of research impact. He thinks that by counting mentions on social media, we don’t reveal much about the significance of the paper, since anyone can share it with no stock in the results. “If you take a look at tweets on papers, it’s clear that in most cases there is no deep understanding or deep reading of the paper,” he says. One can question the use of the data for researcher evaluation.
Bornmann says that the use of alternative metrics is not a good situation if they don’t prove to be valid. “There needs to be a longer period of research on these metrics, and then they can be used in evaluation,” he says.
A break-up of the BEIS could mean that science gets more attention from government, according to a University of Manchester researcher.
Michelle Donelan, a former universities minister, has been made Secretary of State of the new department, while George Freeman retains his science portfolio, with a promotion up the ministerial ranks to minister of state.
Science without Go-it-alone: The value of reforming the R&D system and access to European research programmes in the 21st century
Rathbone highlighted reform of the R&D tax relief system and access to European research programmes as two big issues the new department needs to tackle.
There is value in giving science more visibility and a seat at the cabinet table, but I don’t believe it has anything to do with the fundamental challenges of the real economy and the research economy.
The change in the machinery of government when the country is going to have a general election is odd, says James Wilsdon, who studies science policy atUniversity College London. “It does have a bit of a ‘deckchairs on the Titanic’ feel,” he says.
Donelan was elevated to a cabinet-level department, and science was broadly welcomed by science leaders. But there has been discontent about the Go-it-alone rhetoric.
But building research for any purpose (whether for discovery science, the science needed in environmental protection, or science for industrial development) is not like heating an oven-ready microwave meal. Results are not a matter of going to the store and grabbing something, then pressing a button to get an outcome.
In his address to the UK Parliament last year, Robin Roberts said that collaborative relationships that lead to advances in research have been happening for decades. If these are severed, new ones can’t be formed instantly.
Research in the United Kingdom is already suffering from the loss of European relationships. The holders of the prestigious European Research Council grants had to move to an EU country to keep their grants. A collapse in the number of students from the EU studying in the United Kingdom may affect the future of European researchers there. Many of these students would have stayed to pursue further studies and research, making the country their home in the process.
The European Union insists that the United Kingdom can participate in the EU’s bloc but only if a dispute over trade with Ireland is resolved.
It was a rare occurrence last week when politicians and experts from the UK’s major political parties met with business leaders to discuss how to move forward with their EU partners following the departure of the EU. This, too, is welcome, because it suggests that there’s a real desire for a national consensus on new beginnings with the EU.
