Data retractions could be a powerful tool for cleaning up science

Nature’s News reveals institutional retraction rates around the world in a metric-obsessed world: a case study in China, Saudi Arabia and Ethiopia

Nature’s news team used figures supplied by three private research-integrity and analytics firms to analyse institutional retraction rates around the world over the past decade. Jining First People’s Hospital tops the charts, with more than 5% of its total output from 2014 to 2024 retracted — more than 100 papers (see ‘Highest retraction rates’). It is 50 times the global average, and it is an order of magnitude higher than China. The hospital can be the world’s highest retraction rate, depending upon how one counts.

An awareness of indicators such as retraction rates and retraction volumes could prompt institutions to examine and change their incentives. Instead of counting only articles and citations, institutions could start considering retraction rates too, a kind of counter-metric in a metric-obsessed world. These metrics could also be considered in rankings, or by funders.

A different way to view the data is to pick out institutions with the largest overall numbers of retractions. Most are Chinese universities, but King Saud University in Riyadh, Saudi Arabia, makes the top three. The institutions with the most retracted papers tend to have more retractions, since they are larger and publish a lot. The Chinese universities that included hospitals on their lists of highest- overall retracteds were Jilin University, for instance.

Some sleuths suspect that some African authors may have been added to paper-mill products from elsewhere, to take advantage of the free open-access fees for scholars from low- ormiddle-income countries. A spokesperson for Wiley (the owner of Hindawi) did not give details, but commented: “We are aware of authorship for sale and waiver manipulation schemes deployed by paper mills,” adding that the publisher has strengthened its internal checks.

Many of the retractions were linked to more than one author at the institution. This suggests that the research institutions may have perverse incentives to encourage that kind of behavior, even if individual researchers are solely responsible for their actions.

But this is surpassed by retraction rates in Ethiopia and Saudi Arabia, and exceeded or rivalled (depending on the data set) by those in Iraq and Pakistan. Because many Russian retractions do not have global databases, the firms excluded them from their analyses, even though Russia is in the Retraction Watch data. Meanwhile, countries such as the United States and the United Kingdom have rates of around 0.04%, much lower than the global average of 0.1%, and many countries have even lower rates (see ‘Retraction rates by country’).

A public database of retractions and their affiliations can be misleading: The Indian universities and government institutes of science publish more often than private universities and colleges

There are both data errors and different approaches in the underlying databases, which makes analyzing institutions even more difficult. Open Alex uses a public Research Organization Registry whereas Dimensions uses a private Global Research Identifier Database to map institutional affiliations. It is possible that affiliations can be missing or wrongly attributed, or the database curators might have made different choices about how to assign an affiliation. Accordingly, the firms’ analyses vary.

Researchers in public universities and government institutes in India face fewer pressures to publish than do those in private universities and colleges, says Agrawal. Private institutions, he says, push students and researchers to publish many articles, and in some cases pay bonuses for papers published.

Retraction Watch staff have also manually filled in the reasons — as far as they can determine them — for each recorded retraction, indicating that the majority are due to misconduct. Those data aren’t available for the records that the firms have added.

Their tools aim to alert users to potential ‘red flags’ in research articles or submitted manuscripts, such as authors who might have high numbers of misconduct-associated retractions. To build them, the firms have created internal data sets of retracted papers and their affiliations. The database launched by Retraction Watch was acquired by crossref, a US non-profit organization that indexes publishing data. This made it easier for others to use and analyse the information.

It is said that Scitility will make its figures public later this year. “We think scientific publishing will be helped if we create transparency around this,” says the firm’s co-founder Jan-Erik de Boer, a former chief information officer at Springer Nature who is based in Roosendaal, the Netherlands.

“It’s tempting to think about whether differences are related to varying incentives for researchers in different institutions,” says Ivan Oransky, co-founder of the website Retraction Watch, which maintains a public database of retractions on which the companies contacted by Nature partly depend.

The problem was that some young physicians at hospitals had purchased fake manuscripts from paper mills: companies that churn out fraudulent scientific reports to order. The doctors were under pressure to publish papers if they wanted jobs or promotions, according to integrity sleuth Elisabeth Bik. Sleuths started spotting signs of the problem, identifying duplicated pictures in large numbers of papers. They publicized the issue and a wave of retractions followed.

In general, there is little consistency in how publishers record and communicate retractions — although last year, publishers did agree, under the auspices of the US National Information Standards Organization, on a unified technical standard for doing this, which might help to improve matters.

Online records of work that was done are messy. In Cross Ref, a website that records data on published articles, one paper in The Lancet is currently categorized as retracted when it has only been corrected. This was caused by an error by the paper’s publisher, Elsevier.

The mountain of retractions data is important and should not be ignored. Retractions data are a reminder that quality, as well as quantity, count in science.