Time and money are increasingly critical issues for every drug development company. Choosing an adaptive research strategy over a conventional approach can cut down on any waste of resources and give your company a faster, more effi cient means of development.
Future Pharmaceuticals: What is adaptive research?
Michael Rosenberg Adaptive research essentially means that you can make mid-course corrections to a study while it’s in progress. This is a change from the traditional path where we plan a study, execute it, and then at the end we analyze it to see what’s happened.
Basically, there are two components to adaptive: design and operational. Most people think about the former, design, which includes things like sample size, re-estimation and seamless transitions. However, the operational aspect is perhaps more important for most companies, and that’s simply because it cuts across all aspects of how a study is conducted. Both of these components should be considered for virtually every project, because it’s rare that there’s not an opportunity to improve efficiency by using at least some elements of both.
It is also important to realize that both the design and operational aspects rest on a common technology platform that allows you to collect, analyze and disseminate information quickly. You need relevant, actionable information according to different roles in the studies. We’ve never found an instance where there wasn’t some element of adaptive that could improve the efficiency or speed of how a study was being done. Sometimes we get called in to speak about a study which is late in the design phase, and then we go back and point out things that could have been done better. A lot of people respond, “I didn’t know we could do that.” At that point, we either step backwards and redo the work or find that it’s too late to incorporate some of the elements that would be beneficial.
FP You mentioned design and operational components. What are the differences between the two?
MR Design issues can be thought of as protocol -level — how the study is designed. Design issues tend to have a statistical basis, but they also have operational implications. For example, if you’re doing a seamless transition between one phase and another, you need to have a lot of operational pieces lined up in exactly the right sequence, at exactly the right time.
Operational adaptations are changes to how the study is actually run, such as maximizing site performance, enrollment strategies and allocation of resources. The fundamental concept is the notion of continuous measurement and refinement of a study. Being able to track things like screening rates, screen failure rates and reasons for screen failure can provide important clues about what might need to be changed.
Let’s look at an example. We ran an STD study where we found that one of the sites was enrolling far higher than the other sites from the outset; so of course, we called the site and asked them what they were doing. They had a brilliant idea to advertise the study in the bathrooms at nightclubs. We shared that idea with the other sites, which were then able to pick up their enrollment as well. In the end, this improved the study timelines by about a month. There are a lot of other reasons why enrollment can be improved. We often find that slight changes to a study’s inclusion and exclusion criteria are enough to increase enrollment, but you need real-time information about why patients are screenfailing in order to make those judgments.
Operational adaptations also pertain to how sites are monitored. We worry that if we send our clinical research associates (CRA) out into the field on a fixed schedule, they might find there’s not really that much work to do — or that there is too much. So we track the number of unmonitored fields and time visits according to need. That can even be fine-tuned according to the CRA: whether you’ve got a good CRA that does a lot of fields in one day, or whether you’ve got a beginning CRA that doesn’t do as many. We also track a number of performance metrics that reflect CRA performance: query rates, turnaround time and rework effort can provide a clue to the CRA ahead of time about what they’re going to need to do when they go there.
We also find that sites themselves vary quite a bit in performance. Some are very good at solving problems and others aren’t, and by tracking what works well, we can share the lessons learned with others.
FP Can you describe why these are so important to the industry right now?
MR Everybody who works in this industry recognizes that we face a crisis of efficiency. Today, it costs about five times as much as it did a decade ago to get a new drug to market. Though trials have grown larger and more complex in that interval, cost increases have far outstripped the increase in complexity. We simply have to do better. We’re on an unsustainable trajectory, and the benefit of adaptive is that it represents a major leap in productivity.
FP How major can the impact of adaptive be for the industry?
MR When fully implemented, adaptive design and operational components can decrease costs and timelines by somewhere between 10 and 25 percent. We have even seen some exceptional cases where gains of 30 percent and more have been realized. However, with the implications of getting these kinds of drugs to market, even a 10 percent gain is an enormous benefit. For example, there is one major drug that we were able to get to market about a year earlier than planned through a combination of design and operational adaptations. Most of the adaptive changes based on this platform leverage the advances in technology and communications that have occurred over the last couple of decades. A key problem that we face as an industry is that most of the processes we use are based on the communications capabilities that existed more than a couple of decades ago.
FP How did you arrive at these numbers?
MR Let’s look at an example to illustrate. One aspect of drug development is dose-finding studies. Traditionally, you do a dose-finding study by selecting a couple of arms and a comparator, and you run that study for a certain period of time to get the information you need. At the end of the study, you go back, look at the dosing arms and select the best one. If that same study were run adaptively, you would follow the outcomes as the study is being run, so you could cut off some of those arms if it was apparent that they weren’t working very well. What that means is, for your enrollment pool, you’re enrolling the same number of patients into a smaller number of arms, so your enrollment in the remaining arms goes faster. So, you get to that finish line sooner.
Here’s a real-life example: We ran what was originally planned as a 16-month study, but we were able to cut four months off the timeline and 25 percent off the budget as well — so that’s 25 percent off both time and cost. This relationship is common: whatever magnitude of reduction there is in time, it tends to be about the same in cost. So we were able to save $1.5 million out of a $6 million budget. From the investment perspective, the ability to cut off even a modest amount of time makes a huge difference to the net present-value calculation for the drug. This example was of a fairly modest drug with a projected sale of $300 million per year, but it increased the net present value of that drug by about $80 million.
FP That seems to create concerns for statistical and study validity. How do you ensure that key design elements are maintained?
MR Validity is paramount, and that always has to be maintained. The way adaptive studies are run is pretty much the same way that any study is run: by guaranteeing that no bias can creep in. It does mean that sometimes there are some extra steps. With the dosing arm example, that would require that the people making the decisions are firewalled from the people who are actually running the study. When you understand that one of the goals of that study was to be able to roll seamlessly into a Phase III study, that would eliminate about six months. There’s an investment that you make, but the investment can be paid back handsomely when things take a shorter period of time to be done and cost less money.
FP Can you outline an example of how a design adaptation works in practice?
MR Let me give you two examples. The first is something that everybody wrestles with: How large should this study be? You have to make a guess; you never know the answer — which is of course the reason we do the study to begin with. We don’t know how much better our drug will do than a comparator, so we always end up being conservative and over-build the study.
When we plan studies and sample size, no matter how careful we are, there are basically three possibilities that can occur. The first is that we undershoot it. For one reason or another, there might be more dropouts or a different degree of efficacy than we’re looking for, to name a few issues. Essentially, we find out that the study was too small to meet its statistical goals, which means the study fails.
The second possibility is that we do the study, get to the end, and find out that we’ve actually got more information than is necessary. This is a common practice — people shrug and say, “That’s what we had to do to make sure we got past the goal posts.” However, this is a waste of time and resources.
The third possibility, which you’ll virtually never see, is to hit it pretty much right on. The reason that we virtually never have this occurrence is simply because there are a number of estimates that go into determining study size, and getting every one of them right is highly improbable.
In contrast to that, we would do an adaptive study by simply running the study, and then about halfway through looking to see exactly what parameters are being observed. We use those parameters to reassess the sample size. You can do that without taking a statistical hit, and you can do that more than once if you want. It allows you to reach the statistical goals with whatever margin of error you prefer, and without overshooting or undershooting dramatically. In a sense, sample size re-estimation is a degree of insurance to make sure that you achieve your statistical goals but you don’t waste resources.
The second example is a common issue: enrollment. Under-enrollment is a very widespread problem in this industry. About 80 percent of the studies that we do are slow to enroll and take longer than we initially plan. The protocol will tell us how many patients we need and will define inclusion and exclusion criteria, but we still have the major operational challenge of finding, recruiting and enrolling enough of the qualifying patients within study timelines. The reason that a lot of studies miss their enrollment target is because we tend to be optimistic when we’re planning these studies. The operational component of adaptive allows us to track enrollment from the very beginning, and when slow enrollment becomes a problem, we know immediately rather than having to wait a number of months. The typical response to slow enrollment is to add more sites, which costs more time and more money.
Instead, with adaptive enrollment, you say, “Let’s figure out why these patients aren’t getting in.” It might be a question of advertising — where you’re advertising, how frequently you’re advertising, your message, and so forth. However, a lot of times, what we find is that we’re screen-failing patients, and there needs to be some fine-tuning of the inclusion and exclusion criteria. Often, you can go back and adjust those things, which allows you to increase the enrollment rate.
So there are two basic benefits of adaptive. The first is that you get an answer much sooner, and sometimes you get information you may never have found during a conventional study. But you also get information that you need to actually manage the situation, which is absolutely lacking in conventional studies. When you have this information available, you can share what works well and what doesn’t with other sites. Then people begin to work more as a team, which works much better than the individual approach.
These are enormous leaps over the way we do things right now — what I refer to as having a GPS during the course of a study. Adaptive enables you to be able to look at different components whenever you want, from the very big design elements to the day-to-day operational issues, and to be able to continuously refine those things over a period of time.