Is the paper your AI cited even real? The checklist I use
Four checks for catching fake citations, non-existent trial results and wrong gene names in AI answers, with research on how often citations are fabricated and a free checklist.
Relevant roles: Regulatory affairsClinical developmentMedical affairsBD
I use AI tools such as ChatGPT, Claude, Perplexity and Consensus when I study and when I prepare the newsletter. They're genuinely useful for getting my bearings quickly. But one thing I've learned while studying is that even when an AI gives you a source, you still have to check the source is real.
The numbers
- One study checked the 115 references in 30 short medical papers written by ChatGPT (GPT-3.5): 47% did not exist, and only 7% were accurate [1].
- Another found that 55% of GPT-3.5's citations and 18% of GPT-4's were fabricated. Newer models do better, but the problem doesn't go away [2].
In biotech and pharma roles, this kind of mistake is costly. A non-existent paper or a wrong number in a regulatory document, a clinical report or a BD assessment can undo trust in one go.
The four things I check
- Paper citations: I open the DOI at
https://doi.org/, or search PubMed if there's no DOI, then match the title, authors, year and journal. - Clinical trials: I search the NCT number on ClinicalTrials.gov and check that the primary outcome and comparator match what the AI said.
- Gene names: I look up the official symbol on HGNC (genenames.org). Some old symbols were changed because of Excel, such as SEPT to SEPTIN [3].
- Numbers and dates: I find them again in a primary source, such as the company's press release or the regulator's announcement.
The detailed steps, research sources and a practice exercise are in the free Checking AI Answers checklist in the resource library.
What I've learned
I don't think the answer is to stop using AI. Keeping to the order "AI drafts, I verify" lets me work quickly and still produce something I can stand behind. I also looked at how to weigh the strength of evidence in BIO:ON Insight Issue #16.
If an interviewer asks "How do you use AI tools?", explaining how you verify will probably be far more convincing than listing which tools you use.