Quantitative methods had a field day predicting the US presidential election November 6, making political pundits look silly. The election “quants” outperformed the pundits using both Bayesian predictive probabilities and frequentist methods.
Those of us in clinical development should take a lesson. Elections and pharma development are all about decisions—voters deciding how to cast their ballot or program managers deciding when and how to move to the next stage of development. The FDA guidance on adaptive design notes that Bayesian predictive probabilities “…may aid in deciding which adaptation should be selected, while the study design is still able to maintain statistical control of the Type I error rate in the frequentist design.”
A recent development program provides a good example of how the same tools that help predict an election can predict clinical progress and outcomes. Although those tools don’t shorten the process of electing officials, they can help dramatically shorten development timelines and reduce risk. The latter is particularly important because of adaptive’s key benefit of identifying problems early, allowing adjustments or discontinuation.
For a novel biologic, we worked with the client to combine PoC and dosing studies into a single study based on ability to continuously track outcomes. One of the other key features was continuously projecting where the study would end up, based on data to date. The design provided for stopping the study when an adequate signal was defined. This contrasts with the rigid fixed-sample traditional approach that may leave the sponsor with too much or too little data.
Less than halfway through the planned timelines, the predictive probability of success exceeded 94% with reasonably tight credible intervals. (See decision point in figure below.)
This information would have made it reasonable to terminate the study and progress to the next step. But this program was particularly interesting because, for unrelated reasons, the sponsor decided to continue the study for an additional year. At the end of that year, the halfway predictions held true. In retrospect, this study could have saved half the cost and time involved.
Sometimes conditions do limit the ability to predict the future for drug developers, just as they do for political commentators. For example, projections often assume uniformity of data, i.e., that future data will be the same as the initial data. While this generally holds, a variety of factors add nuances and exceptions. Variability is one example. That said, we can generally gauge most such factors during the course of the study, and we also have the option for adjusting expected future data.
Predictive probabilities and combined-phase studies are powerful tools in their own right but they become even more powerful when combined with other adaptive components that can effectively control risk in later stages
Overall, use of predictive probabilities, combined-phase studies and other adaptive tools brings us closer to the ideal of a continuous development model. While progressing faster, we have greater and greater assurance that we are on the right path. We can add resources with confidence when necessary. That’s the heart—and the extraordinary power—of adaptive.
PS: Don’t forget that adaptive requires more than a sound design – you also need the right tools for information flow and decision-making. Many programs fail for want of such tools.