European Biopharmaceutical Review
Spring 2007 - Adaptive research, the ability to alter clinical studies in progress based on analysis of partial results, is among the most important developments in the biopharmaceutical industry for several decades. For smaller and emerging biopharmaceutical companies, adaptive techniques present an unusual, perhaps unique, opportunity to improve development efficiencies and gain competitive advantage over much larger companies.
Adaptive research, the ability to alter clinical studies in progress based on analysis of partial results, is among the most important developments in the biopharmaceutical industry for several decades. For smaller and emerging biopharmaceutical companies, adaptive techniques present an unusual, perhaps unique, opportunity to improve development efficiencies and gain competitive advantage over much larger companies.
Adaptive techniques are the latest and perhaps most significant stage in a transition from big pharma’s long domination of drug discovery and development, to a new and more competitive era in which big pharma is just another player. Advances in technology have lowered barriers to entry, transforming drug development from a capital-intensive, smokestack industry into a technology-leveraged, knowledge-based enterprise. Smaller companies are now on an equal footing with industry giants. Indeed, in many cases, quicker decision-making processes have enabled small companies to lead the way in adopting innovative ideas and methods.
Innovations in drug discovery include combinatorial chemistry, molecular design, RNAi, production of biologics in vivo by recombinant DNA, high-throughput screening, and the use of DNA microarrays. Such innovations have enabled small, ambitious competitors to enter drug discovery, and have yielded vast numbers of candidate drugs and treatments. The leading role played by smaller companies in many of these advances would have been unthinkable just a decade ago.
While innovations in discovery have expanded the number of potential drug candidates, clinical trials remain a chokepoint. They also contribute substantially to development costs estimated at more than US$800 million per drug (1). Even such astronomical spending levels have brought surprisingly few new drugs to market. The US pharma industry’s annual R&D spending reached an all-time high of $40 billion in 2006 but resulted in the approval of only 18 truly new drugs, the lowest number in recent years (2).
THE ADAPTIVE APPROACH
The fundamental difference between adaptive capabilities and more traditional development is reliance on a ‘refine as you go’ rather than a ‘black box’ approach. An adaptive design permits adjustments along the way based on experience. This common sense approach stands in stark contrast to the familiar routine of waiting until all the data are available before analysing it and determining success – an approach best summed up as ‘making a guess and keeping your fingers crossed’. Because adaptive requires continuous refinement along the way, these techniques translate to shorter timelines and lower costs. Indeed, continuous adaptive adjustments guarantee success at meeting informational goals at minimum cost in time and money.
Figure 1 illustrates how adaptive approaches continuously acquire knowledge by examining both data and the operational aspects of a study while it is being conducted. In contrast to the usual knowledge gap between the time at which data or study performance metrics (which are most often not available at all, except in the crudest sense, during a study) are available, clean, and in an understandable form, adaptive designs and processes focus on immediately transforming a stream of raw data into meaningful information that can quickly be transformed to knowledge. This capability demands more effective data collection and management than is commonly used.
Adaptive research refers to a broad range of capabilities, all of which relate to what has occurred most recently in a clinical trial. The intellectual groundwork was first laid in 1763 by Thomas Bayes, who gave formal mathematical expression to what many would consider the essence of common sense: continuously updating or revising beliefs in the light of the latest evidence.
The following two examples show the instinctual appeal of this approach, and begin to demonstrate the profound consequences for development timelines and costs. The first example is the simple and well-accepted notion of reassessing the sample size during the course of a study. Sample size is determined by acceptable levels of possible error (risk of both false-positive and false-negative results), the level of difference in outcome measures between the drug being evaluated and a comparator, and other elements such as variance of data. In practice, assessments of such elements at the beginning of a trial are little more than guesses – and they almost always turn out to be wrong.
If fortune smiles, estimates may be only slightly off, but they will still be off. The consequence of underestimating these factors is negative results for a drug when positive results are expected. To prevent wrong estimates from having such drastic consequences, studies are generally overbuilt, using the most conservative assumptions and/or deliberately intending to be overpowered.
Sample size reassessment means that during the course of a study (generally about halfway), the parameters that determine study size can be re-estimated using the actual levels encountered so far. This procedure can also be repeated later in the study to provide even more refined calculations. This results in arriving at precisely the desired informational endpoint without risking the futility of undershooting or wasting resources in overshooting.
A second example of adaptive techniques is pruning dosing arms during a study. Safety as well as efficacy outcomes are continually tracked, and at given points, the less promising arms can be dropped. Doing so is ethically preferable, since it eliminates arms that experience has shown to be less safe and/or efficacious. The adaptive approach also affords the luxury of a greater number of dosing arms than would be possible under the financial constraints of a conventional trial.
REQUIREMENTS OF AN ADAPTIVE PROGRAMME
Adaptive research can be considered a customisation for the clinical research of modern techniques of tight management. Modern management relies on reacting to a continuous stream of information about critical business processes to make decisions that optimise future results, such as reducing manufacturing runs when real-time data shows disappointing sales. A decision to reduce the number of dosing arms in a Phase II study involves complex medical issues, but the underlying principle is the same: based on the latest data, make decisions that minimise waste and optimise results.
An adaptive development programme requires a different approach to study design, data capture, job functions, business processes and planning. Reaping the full benefits of adaptive techniques requires thinking in terms of entire programmes, from first entry in man through completion of clinical trials and submission of regulatory applications. Thinking in these terms provides an invaluable opportunity for companies to look farther ahead and consider a range of scenarios before launching a trial.
Looking over the horizon is essential because adaptive protocols require each course of action for each possible scenario to be spelled out ahead of time as a means of ensuring a study’s scientific integrity (and reassuring regulatory authorities). Learning and decision-making no longer wait until the very end, with the sudden transformation from having no data and knowledge to having a deluge of data and the need to extract knowledge all at once. In adaptive trials, progress must be assessed weekly, emerging trends continuously identified, and decisions taken when necessary to optimise the trial.
The adaptive approach requires tight management. Since tight management is impossible without current and accurate data and analysis, adaptive trials demand the use of efficient technology, starting with data capture. Without rapid, accurate data capture, all the goals and benefits of adaptive research are frustrated. Unfortunately, current web-based EDC systems fail to measure up to the needs of adaptive studies because of the unacceptably long lag between the time when data are generated and when they are entered into the system and made available in a form that can be used as the basis for decision-making. Systems exist that make data available within minutes of its collection and also enable tracking study metrics as well as data (3).
Realising adaptive benefits requires the production of a reliable stream of timely data to illuminate performance and empower practical decisions to optimise a study. Providing the required flow of data to support decisions presents challenges not only for the technology used in clinical trials but also for the typical work processes, which are often poorly defined (4). For example, no two project managers or site monitors do things exactly the same way – a clear indication that clinical research today is still operating with poorly designed processes more appropriate to a 19th century craft than an efficient modern business.
Despite the current variability in the conduct of clinical trials, almost all trials require repeated performance of the same essential processes. These processes can be defined much more clearly and precisely than is customary. In the course of defining a process precisely, we can also see ways to improve it. It is process improvement in combination with new technology that brings huge rewards. The gains from inserting expensive technology into an old, inefficient process are minimal.
STRATEGIC AND TACTICAL DECISION-MAKING
After the processes for a study are well defined, it becomes possible to design in the real-time data collection, analysis and performance metrics essential for supporting adaptive research. Experience with more than a hundred adaptive studies shows that the critical enabler for adaptive research is the combination of technology and processes that permit timely decisions on multiple levels. Optimisation of both tactical and strategic decisions is necessary. This requires the ability to collect, validate, manage and analyse data, and then report in real-time – within minutes of a patient having walked out the door.
With the necessary data flow assured, optimisations can take place at many levels. For example, at the site level, consider a specific patient visit. Suppose immediately after the visit, the input device used by the clinician to record data on the case report form is placed in a cradle that transmits data from the visit to a central study web site (5). Receipt of the data triggers a pre-planned process that immediately checks the data against requirements and automatically generates and transmits any necessary query to the site. This represents radical improvement in the query process, resolving them in minutes instead of days or weeks.
Performance metrics can contribute to tactical decisions that substantially improve the efficiency of an entire study. For example, experience in numerous studies has shown that a long interval between a site’s receipt of queries and the site’s required response predicts poor future data quality. Tracking this metric on all sites permits monitors to address the issues causing slow query response early in the trial, preventing a backlog that will later delay site close-out and database lock.
At the strategic level, decisions focus on such issues as the dosing arms to be used for the balance of the study. A typical Phase II study has several dosing arms and a goal of finding a single or perhaps two doses for use in a large and expensive Phase III study. Rather than waiting to examine data on dosing arms until the Phase II study ends, an adaptive study would likely determine part of the way through whether some dosing arms are clearly ineffective. Enrolment in futile arms could stop immediately, resulting in substantial savings.
EXAMPLES OF ADAPTIVE APPLICATIONS
Figure 2 shows a simplified diagram of how pruning occurs, and Figure 3 shows how this approach changes the study itself. When unpromising arms are cut off, the enrolment pool is focused on a smaller number of arms, increasing that enrolment rate. In this case, the adaptive approach allows the desired informational goals of 80 patients to be achieved four months sooner than a traditional approach, at a saving of ˆ1.5 million.
The financial consequences of sample size reassessment, another adaptive technique, in a recent oncology study reduced time-to-market by nine months and saved $16 million in development costs. The extra time on the market allowed the sponsor to gain an additional $366 million in revenues. This was achieved by refining an initial guess of the magnitude of difference between the drug being evaluated and its comparator (δ), which was better than the initial guess and allowed the study to be completed with fewer patients.
A different set of adaptive tools – data collection tools and reporting capabilities that focus on tight study management, without any strategic components – also provided dramatic gains in the Phase III trial of an Alzheimer’s drug candidate. Measured against internal timelines of $100 million over five years, adaptive techniques saved 1.6 years and $32 million in direct costs. NPV and IRR calculations make clear that even single elements such as the pruning described above can produce quite striking benefits. Profound benefits also flow from other adaptive optimisations, including rolling from one study or phase straight into another, saving as much as a year between studies and phases.
THE IMPORTANCE OF ADAPTIVE-GRADE DATA CAPTURE
For biopharmaceutical companies attracted by the substantial rewards of adaptive research and tight management, it is important to note that improving data capture and handling alone offers much more modest benefits. This shortcoming has arisen because webbased EDC systems lack the ability that is critical for taking advantage of adaptive techniques – the ability to empower timely decisions on multiple levels. EDC systems have been designed to quickly collect data, but in practice they produce disappointing results because of frequent delays in data entry, lack of integration, lack of analytic capability to inform decisions, and thus an inability to manage studies tightly. Web-based EDC’s failure to measure up to the standards of adaptive-grade data capture is especially apparent in studies that are large and geographically diverse.
The typical data capture technology used in web-based EDC has also failed to keep pace with technological advances. One example of a more advanced and efficient input method is the smart pen, which directly enters data from the CRF and transmits it to a central database instantly when inserted in a cradle. This eliminates wasted steps and reduces the need for source verification because what is transmitted is identical to the source data – in fact, it is the source data.
THE TRANSITION TO ADAPTIVE TECHNIQUES
An adaptive approach requires more careful thought and planning than is traditional, including the exploration of multiple ‘what if ’ scenarios. This is probably a benefit, because it forces forward-thinking about situations further ahead, emphasising the need to focus on the development programme as a whole rather than individual studies. Protocols employing adaptive techniques need to specify precisely what will occur in different circumstances, a requisite element to ensure that scientific integrity is not compromised, whether unintentionally or knowingly. When unblinded assessments are required (such as sample size reassessments or in some cases of pruning), a firewall must be in place to ensure that those running the study are not unblinded. In practice, these and other pragmatic aspects of adaptive designs can be straightforwardly achieved and are accepted by regulatory authorities.
Adaptive techniques will also change the way companies make decisions during a study. Some elements, notably those that relate to tight study management, open new horizons for effective management. For example, closely monitoring enrolment should result in continuous improvement in how patients are recruited. This can easily be achieved with systems that enable continuous tracking and the ability to determine why certain sites do well and others do poorly. The lessons, both good and bad, should be shared immediately and widely. The transition to adaptive research can easily be implemented in baby steps, starting with a small study and a modest goal of shortening it by 20 per cent. The important thing is to begin the transition to far more efficient processes and methods rather than clinging to an outdated approach.
The potential of adaptive research to improve operations industry-wide, for companies large and small, is simply too great to ignore. The most compelling argument is simply that not utilising such techniques condemns programmes to suboptimal decision-making, but utilising them entails no risk of conducting less efficient studies – the worst that could happen in an adaptive trial is that no adaptive changes would be made, leaving study performance at the same level as before. On the other hand, aspects that are exercised can only be beneficial – any aspect reduces time and costs.
Adaptive research also offers biopharmaceutical companies an arena where entrepreneurial energy and sheer brainpower can be more than a match for the larger resources of slow-moving industry giants. The situation calls to mind the words of the selfmade American billionaire Ross Perot: “Brains beat money ten times out of ten”.
- Dimasi J, Hansen R and Grabowski H, The price of innovation: new estimates of drug development costs, Journal of Health Economics22: pp151-185, 2002
- Bloom J and Pettypiece S, Few new drugs OK’d in 2006, Reuters News Service, Washington, DC, 5th January, 2007
- Visit http://www.healthdec.com/technology/smart_pen/, http://findarticles.com/p/articles/mi_pwwi/is_200406/ai_mark3909333615, http://www.healthdec.com/services/data_management/
- Schoenberger C, An Alzheimer’s Drug Goes On Trial, Forbes Magazine, 20th March, 2000 http://www.healthdec.com/media/articles/AnAlzheimersDrug%20GoesonTrial.pdf.
- Visit http://www.healthdec.com/technology/smart_pen/
Dr Michael Rosenberg is the founder and CEO of Health Decisions, a global clinical research organisation (CRO) specialising in adaptive clinical research methodologies. He was awarded Ernst & Young’s Entrepreneur of the Year in Health Sciences in 2002, and has led Health Decisions to multiple inclusions in the Inc. 500 fastest growing private companies and the Technology 500 fastest growing technology-based companies in the US. The author of more than 150 scientific articles, Michael serves on advisory groups in business, technology, and medicine. He is Clinical Professor of Obstetrics and Gynecology for the School of Medicine and Adjunct Professor of Epidemiology for the School of Public Health at the University of North Carolina. He practiced emergency medicine for more than 20 years. Michael received his undergraduate and medical degrees from the University of California and his Masters degree in Epidemiology and Biostatistics from Harvard University. He is currently a fellow at the American College of Physicians, the American College of Preventive Medicine, and the American College of Epidemiology.