Can we trust scientific papers?
Robyn Williams: The Science Show on Radio National. Over in America, universities are facing massive cuts, the loss of scientific staff and misinformation in the press, as well as on the dreaded social media. So what can we make of some of that? Here's Len Fisher who writes in the Blue Mountains and does research in Bristol.
Len Fisher: How do you know when to trust a scientist? For that matter, how does a scientist know when to trust a scientist? These questions are stimulated by an article that Robyn has brought to my attention. James Heathers from the Linnaeus University in Sweden claims that up to one scientific paper in seven may be at least partly fake and not to be trusted. So what is going wrong? Is Heathers paper itself partly fake? If not, what can we do to protect both science and the wider community from such fakery?
Let's start at the beginning. The trust of scientists in published science relies, or at least it used to rely, on peer review. Careful and critical examination of the work by one or more experienced fellow scientists who may award a pass, fail, or go back and try harder.
One problem with this process is the possibility of favouritism or even delaying the publication of a paper so that the reviewer, who will sometimes be working along similar lines, can get there first. I've seen it happen, although it does seem to be rare.
A more likely reason for delay is that reviewers are not paid for their work, and with many pressures on their time, reviewing tends to be put on the back burner. This gets to be real problem in a rapidly moving field, where younger workers are often involved and where priority can be important both for self-esteem and for actual promotion or the awarding of a degree. To get round the problem, authors these days tend to resort to online publication prior to peer review and conventional journal publication.
In fact, Heather's paper itself was published in this way, in what is called the Open Science Framework. It has not yet been peer reviewed. Does this matter for trust? At least Heather's paper is out there in the open, subject to critical commentary.
Some of that commentary has been pretty devastating. One fellow scientist, an expert in the sort of meta-analysis that Heathers uses, claims that Heathers' falsely labels studies with some problem as definitely being fake and incorrectly lumps together different studies measuring different phenomena. To his credit, Heathers accepts these criticisms in essence and even admits that his work is wildly non-systematic averaging as it does the results of 12 different studies of fakery in different fields and using different criteria. But he says that this is the best that he can do with the data available.
So far, so what? This is a squabble between scientists, and let's leave them to it. But the real problems start when the work impinges on the wider world. Scientific misconduct undermines public trust in science. It is rightly a matter of public concern.
So studies like those of Heathers find their way into the media, but with no mention of the essential caveats that, in Heathers' case, could well have meant that it would never have passed peer review. Even with quite careful journalistic treatment of the caveats, the majority of people who come across the story are likely to notice only the headline. 'One in seven science papers are fake'. Not even one in seven science papers could be fake.
Such headlines provide a convenient reason to reject scientific findings for those whose beliefs are challenged by these findings. A prime current example is the political dismissal of even very rigorous studies on the role of fossil fuels in global warming.
Let us return to the underlying problem, which is the current erosion of trust in science. What can be done to restore and maintain trust, both of scientists with each other and of the wider public the science?
The first thing to note is that most published papers are essentially trivial, small studies that receive at most one or two citations and whose results are of little concern except to the authors, whose quota of publications has increased, and their small circle of specialists. When a paper does address an important issue or reveal an important new finding that is of wider interest, then other scientists are likely either to repeat or to use the results. If there is fakery or sloppy science involved in the original publication, then hopefully the truth will out.
Unfortunately, this process can take some time, especially if the original work was convincing and fitted scientific preconceptions, as happened with the attribution of Alzheimer's disease to the presence of protein plaques in the spaces between brain cells. Also, replication can be difficult, or even impossible when specialist techniques sometimes possessed only by the originator are involved, or when conclusions are based on large-scale surveys that may have had inadequate controls. There is also the very human problem that scientists are likelier to accept the results of their fellows, especially the senior ones, without going to the bother of replication. This can be a problem of money or resources. Whatever the reason, it is a systemic problem in some disciplines witness the current well-publicised crisis of reproducibility in the psychology literature.
In general, however, replication, or even just its possibility, remains a powerful tool for the evocation of trust among scientists. The wonderful online retraction watch reports such exposures on a routine basis. Another welcome development is that of online pre-publication facilities such as arXiv which has the very peculiar spelling of small a small r capital X small i small v, which filters papers on the basis of the established reputations of the authors or recommendation by an established scientist. In fact, as I sit writing this talk, my collaborator on another publication, the Swedish polymath Anders Sandberg, is sitting in the corner of the same room submitting a paper of ours to Archive Physics but they won't accept it until it receives the approval of another known expert in the field, and even then it will be subject to moderation.
It is a pity that this same process can't be applied to books, especially those that make exaggerated claims based on little or no evidence. A recent New Scientist article has pointed out how few of such books are fact-checked. This may be just as well for the profits of the publishers, who often rely on the sales of such books.
The New Scientist itself employs two levels of fact-checking. But facts are dull, and sadly they are often trumped by simplistic exaggerations, especially when these appeal to pre-established prejudices. Maybe the answer is for scientists to learn to share more fully the very real excitement that comes from the pursuit and capture of facts, and their sharing with other scientists in an established atmosphere of trust. Only then will public trust in science already high in most countries, grow to overcome misleading stories about its very human vicissitudes.
Robyn Williams: Len Fisher is a Fellow of the Royal Society of New South Wales.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
ABC News
an hour ago
- ABC News
Mahmoud Khalil vows to resume pro-Palestinian activism after release
Palestinian rights activist Mahmoud Khalil says he will continue to protest against what he calls US government-funded genocide in Gaza. Mr Khalil said the Trump administration and Colombia University, where he protested, were complicit in Israel's actions in Gaza. "Not only if they threaten me with detention, even if they would kill me, I would still speak up for Palestine again," Mr Khalil said. "I just want to go back and just continue the work that I was already doing, advocating for Palestinian rights, speech that should actually be celebrated rather than punished." Mr Khalil, 30, was reunited with his wife Noor Abdalla, a US citizen, at Newark Liberty International Airport in New Jersey on Saturday afternoon. He was met by friends and supporters, including US Democratic congresswoman Alexandria Ocasio-Cortez. The US government still wants to deport Mr Khalil, a permanent US resident, arguing his activism is detrimental to American foreign policy interests. Mr Khalil, who recently graduated from Columbia University in Manhattan, was a prominent figure in the pro-Palestinian and anti-Israel student protest movement that swept campuses last year. Federal immigration agents arrested him in the lobby of his Columbia apartment building on March 8, making him the first target of Mr Trump's effort to deport international students with pro-Palestinian or anti-Israel views. Ms Ocasio-Cortez, speaking alongside Mr Khalil at the airport, condemned the Trump administration for what she called "persecution based on political speech". "Being taken is wrong. It is illegal," she said. "It is an affront to every American." Mr Khalil was born and raised in a Palestinian refugee camp in Syria and lawfully became a permanent US resident last year. Nonetheless, citing an obscure part of federal immigration law that has not been invoked in more than 20 years, US Secretary of State Marco Rubio said he had determined that Mr Khalil and several other foreign pro-Palestinian students at US schools must be deported because their presence could harm the government's foreign policy interests. Protesters, including some Jewish groups, said the government wrongly conflated their criticism of the Israeli government, one of the United States' closest allies, with antisemitism. This month, US District Judge Michael Farbiarz in New Jersey ruled that the government could not detain or deport Mr Khalil based on Mr Rubio's determination, finding the Trump administration was violating Mr Khalil's constitutional right to free speech. On Friday, he ordered the Trump administration to release Mr Khalil on bail while he continues to fight the government's deportation efforts and his lawsuit accusing the government of wrongful detention. A spokesperson for Mr Trump said in a statement after the ruling that Mr Khalil should be deported for "conduct detrimental to American foreign policy interests" and for omitting or incorrectly describing his employment history on his application form to become a permanent resident. Mr Khalil has said his application form was correct and the allegations of omission were spurious. Also on Friday, an immigration court in Louisiana ruled that Mr Khalil must be deported. He will now challenge the decision in the immigration court, which is run by the Department of Justice rather than the government's judicial branch, through the Board of Immigration Appeals. The Trump administration appealed Judge Farbiarz's rulings on Friday evening to the US Court of Appeals. ABC/wires
ABC News
an hour ago
- ABC News
US moves B-2 bombers as Israel-Iran conflict continues
Several US B-2 stealth bombers are heading across the Pacific as Donald Trump weighs up whether to join Israel in attacking Iran.
SBS Australia
2 hours ago
- SBS Australia
Morning News Bulletin 22 June 2025
Listen to Australian and world news, and follow trending topics with SBS News Podcasts . TRANSCRIPT Reports the US is relocating B-2 bomber aircraft as Donald Trump considers a potential airstrike in Iran The grandfather of an Indigenous man who died in police custody writes a letter to the Prime Minister In cricket, Sam Konstas replaces Marnus Labuschagne ahead of Australia's opening Test match against the West Indies The United States is reportedly moving B-2 bomber aircraft to the Pacific island of Guam - as President Donald Trump weighs whether the United States should take part in Israel's strikes against Iran. According to US media reports and tracking data, multiple B-2 bomber aircraft have left their base in Missouri, headed for Guam. It is unclear whether the bomber deployment is tied to tensions between Israel and Iran. The B-2 is capable of striking Iran's deeply buried nuclear facilities. The Pentagon has not responded to a request for comment. Israel and Iran have exchanged more strikes, as the conflict enters its second week. Israel is claiming it has killed three of Iran's commanders - Saeed Izadi, Behnam Shahryari and Aminpour Judaki. Iran has not confirmed the deaths. This man in Iran says he wants to see the fighting stop. "I'm not in that situation to decide which one has the winning point at the moment. I just think about my family, my people, and that's it. No matter for me the names - Iran, Israel, I don't know, United States. We are thinking about peace and we hope it happens. The only thing that is important is people. People should be safe." The grandfather of a 24-year-old Indigenous man who died in police custody has drafted an open letter to the prime minister calling for the federal government to step in and address problems in the Northern Territory's justice system. The senior Warlpiri leader's grandson Kumanjayi White died after being forcibly restrained by two plain clothes officers inside a supermarket in Alice Springs in May. In the letter, Ned Jampijinpa Hargraves wrote that the justice system in the Territory was "in crisis", telling Prime Minister the federal government has total power over the NT - and saying "this madness must stop." The letter addressed to Prime Minister Anthony Albanese came amid protests in Sydney and Alice Springs demanding justice for Mr White. Pope Leo has recognised the work of South American journalists who uncovered sexual abuse by leaders of a dissolved Catholic group. The Sodalitium Christianae Vitae group, founded by Catholic laity and priests in 1971, was investigated by the Vatican, which concluded in 2017 that its leader, layman Luis Fernando Figari, abused along with three other former members, 19 minors and 10 adults between the 1970s and 2000s. In Lima, Monsignor Jordi Bertomeu read the leader written by the Pope, ahead of the premiere of a theatrical production that exploring the abuse scandal. "I want to thank those who have persevered in this cause, even when they were ignored, disqualified and even legally persecuted. In this time of deep social tensions, defending free and ethical journalism is not only an act of justice, but a duty of all those who long for a solid and participatory democracy." In cricket, Sam Konstas says he is ready to go again after being picked to play for Australia in their opening Test match against the West Indies on Wednesday. The 19-year-old has replaced Marnus Labuschagne in the squad following the latter's poor form in the recent World Test Championship final. Konstas made a stormy entry to the Tests against India at last year's Boxing Day Test match and scored a half century that contained some audacious shots against the world's best fast bowler Jasprit Bumrah. The opening batter says he still reflects on the moment but has also been working to further improve his game. "To be honest I've been watching it quite a bit. Even some of my mates still send it to me but yeah I still can't believe it. I don't know what was going through my mind at that time but no I thought it was the right time in the moment and yeah it was good fun. It was my first few Test matches... the emotion probably got to me with the crowd and obviously it was good reflecting having a bit of time off, trying to strengthen my game up and yeah hopefully we get to win the start in the next Test match." The three-Test series against the West Indies starts this Wednesday in Bridgetown, Barbados.
