Boosting Bad Science
The science is already bad
Don’t blame AI for making a bad problem even worse. But AI is going to make the problem worse:
The rise of generative AI further complicates matters. If AI models are trained on tainted literature, their outputs — used to generate new research or aid diagnostics — could be compromised from the start. Then there’s the issue of AI-generated science papers. In 2024, a peer-reviewed science journal published a study with an obviously AI-generated diagram showing a cartoon rat with a gigantic penis. Only after the picture made the rounds on social media was the paper retracted.
“We have no clue what’s going to end up in the literature, what’s going to be regarded as scientific fact and what’s going to be used to train future AI models,” Richardson warned.
Some academic publishers, like Springer Nature and Frontiers Media, are starting to issue large-scale retractions — Frontiers recently pulled 122 studies after finding evidence of citation manipulation and undisclosed peer-review collusion. But even these efforts only scratch the surface. Richardson estimates that only 15–25% of fake papers will ever be retracted.
The problem is that science has already been completely corrupted. It started with the need to rescue the theory of evolution from the discovery of Mendelian genetics, hit its stride with all the intrinsically non-scientific “theoretical physics” which is less scientific than the average science fiction novel, and is now the vast majority of all the published, peer-reviewed “science” papers being published.
AI will help produce more science fraud, but it’s important to keep in mind that it is the scientists who are the actual problem, not the AI. The massive fraud ring will certainly make use of AI, but its existence, not its tools, is what needs to be addressed.
Garbage in, garbage out. So it was and so it is. Science 2025.
I wonder what extent of a solid knowledge base it would take for an AI to start developing useful, reality-based innovations? If you could feed it what we know is almost certainly true in physics, engineering, practical biology, essentially any science rooted in solid, observed information and not just speculation; what kind of results could be expected?
It would truly be revolutionary if you could just type in something like, "I have an idea to design a product that does *useful thing*. Is it possible? Given current resources and technologies? Unit cost, output waste created, practical applications..."
Currently, it seems like it would tell you an affirmative answer, then cobble together a bunch of information which may or may not have any basis in reality, or which may do the thing but also create terrible consequences.