New AI model enhances clinical trial approval predictions by quantifying uncertainty

Researchers from Stanford University and Rensselaer Polytechnic Institute have developed an advanced AI model that improves the prediction accuracy of clinical trial approvals. The study, published in the journal Health Data Science, introduces a novel approach that quantifies uncertainty and enhances interpretability, significantly outperforming existing methods.

Accurately predicting the success of clinical trials is crucial for optimizing resources and improving the development process of new therapies.

The new model, based on selective classification (SC) and integrated with the state-of-the-art Hierarchical Interaction Network (HINT), offers a substantial advancement in clinical trial management by identifying trials with low confidence and withholding predictions when necessary. This selective approach enables a more reliable forecast of trial outcomes, especially for early-stage trials where prediction uncertainty is greatest.

“Our AI model can predict clinical trial approval rates accurately and help optimize the management of clinical trials,” said Tianfan Fu, Assistant Professor at Rensselaer Polytechnic Institute. “The next step is to mimic clinical trials in a more fine-grained manner, with the ultimate goal that AI can fully simulate clinical trials.”

Clinical trials are a crucial yet costly and time-consuming phase in the development of new drugs and therapies. Currently, many trials fail due to issues such as drug inefficacy, safety concerns, or flawed trial design.

The Hierarchical Interaction Network (HINT) has previously been a leading model in clinical trial approval predictions, but its lack of uncertainty quantification limited its effectiveness in real-world applications. By enhancing HINT with selective classification, the new model addresses these limitations.

Led by Tianfan Fu and his team, the researchers developed a method that combines selective classification with the HINT model to quantify uncertainty in predictions. This novel approach enables the model to provide predictions only when it is confident, thereby improving the accuracy of clinical trial success forecasts.

In extensive experiments, the new AI model demonstrated a 32.37% relative improvement in the area under the precision-recall curve (PR-AUC) for Phase I clinical trials compared to the base HINT model. For Phase II and III trials, the model achieved 21.43% and 13.27% improvements, respectively. Notably, the model achieved a PR-AUC score of 0.9022 for Phase III trials, marking a significant enhancement over existing prediction models.

The research team aims to continue refining their model to simulate clinical trials more accurately and explore new applications in drug development. “By fully leveraging AI to simulate clinical trials, we hope to revolutionize the way new therapies are developed and approved,” added Fu.

Provided by
Health Data Science