The Illusion of Intelligence: Why Drug Developers Cannot Rely on AI for High Value Decisions 

May 18, 2026

Whilst drug development scientists invent ever more amazing ways to improve the outcomes of patients, these developments are more expensive and riskier than the development of traditional small molecules. And so, the value of fast, accurate decision making has never been more important. 

Preclinical decisions now carry multi-billion-dollar implications. Missing a critical competitor asset can mean the difference between success and failure.  

The need for speed has seen scientists embrace AI. There have been some notable success stories especially where AI is used as a workflow tool, like AlphaFold, or in clinical trial optimization and supply chain management. More recently scientists have begun to embrace AI for researching competitors or markets, prompting a debate about how much trust scientist can place in AI to provide accurate, decision-grade data. 

The Results Speak For Themselves

The leading LLM was able to identify just 14% of all programs and the accuracy of data was highly flawed with only, at best, 13% of data points being accurate. It took Beacon researchers over 20 minutes to re-research each drug program suggested by the LLMs to eliminate hallucinations. So, not only were the results disappointing but the process was time consuming too.

LLM’s Performance vs. Expert Curated Data: No. of B7-H3 Drug Programs

To prove that this wasn’t an outlier. We also looked at some other high–profile targets, namely TROP-2, Claudin 18.2 and FAP.  We found a similar story. Beacon’s expert curated data considerably outperformed all the AI-assisted alternatives. 

AI Performance vs. Expert Curated Data: No. of Drug Programs 

To find out more about how LLMs understand science which has yet to mature fully, we undertook a test. We asked the LLMs to identify as many preclinical Antisense Oligonucleotides as possible. Again, we asked for developer name; drug name; target gene; target organ; and delivery system. The results were very poor. LLMs failed to identify ASOs (antisense oligonucleotides) and when they did find them, they were unable to accurately understand concepts like delivery systems (such as lipid nanoparticles). 

AI vs. Expert Curated Data: Preclinical Antisense Oligonucleotides

LLMs are evolving at a breathtaking rate. But are they getting better for drug development scientists? Here at Beacon we have been tracing some typical user questions against LLMs as they develop – for instance tracking the questions on ChatGPT from v3 to its current v5.5. The results show little or no improvement over time. The volumes and accuracy from v3 to v5.5 have not improved. The volumes of data are not increasing anywhere near the levels required. Advanced prompting techniques can only get you so far. In fact, at Beacon, we have seen drug developers largely abandon LLMs as a research tool – rather it is now seen as a useful interface layer – where they can plug Beacon data, via an API, into a “walled” LLM along with their own data, and with this gold standard input they can lean into the LLMs ability to analyze and synthesize at speed. 

Even if drug developers restrict input data to avoid “garbage in, garbage out”, they still must contend with model drift, where the answer to a question changes from one day to the next which has led to a lot of users taking screenshots of results to “cement” the answer. There is also the issue of overconfidence to deal with. LLMs are designed to please their users and hence come up with answers – hallucinations. They want to make you happy. Of course, users are increasingly leaning into expert prompting to force LLMs to provide confidence scores and eliminate hallucinations. 

It is clear that LLMs are incredibly helpful assistants and can cope with large tasks quickly, and with the right guardrails can be great workflow tools. For instance, tools like AlphaFold have revolutionized parts of the biopharma sector. But in terms of being a reliable partner for decision-grade trusted data there is a long way to go. In fact, the underlying probabilistic nature of the LLMs and their lack of ability to cope with very large and emerging ontologies, means that they are unlikely to ever achieve the kind of accuracy levels required to inform multi-million and billion dollar deals. It is accepted that human experts are required, still, to make decisions, to resolve disputed data and claims; and be curious (and skeptical) about the new. In short, LLMs can be great assistants if fed great data and great instruction they need.