Interim reviews in clinical trials have always been controversial in some circles because they require somebody to look at otherwise confidential data while a trial is in progress, raising the possibility of introducing bias. On the other hand, not letting anybody see interim data is a great way to have a trial with serious issues on day one run to completion with no hope of success, needlessly exposing patients to an experimental treatment and wasting every penny invested. It’s also a great way to ensure the allocation of research funds for future projects based on out-of-date information. The grand historical compromise is to allow interim reviews but limit their scope and ensure that they are done by someone with no interest in the outcome of the trial.
Matthew Herper’s recent piece in Forbes describes the important role that data oversight groups (Data Safety Monitoring Boards) play and examines the concerns of statisticians about certain decisions that DSMBs make based on partial data. The danger is that the totality of study data may look different from what’s available at the halfway point. In the case of a recent Zytiga trial, the DSMB recommended that all patients receive the test drug rather than placebo, because the drug appeared to work so well that it would be unethical to withhold it from the patients assigned to the placebo arm. The issue came down to weighing the value of stronger evidence as a basis for treatment of patients in the future vs. the value of doing what appears best for the patients in the current trial.
Regardless of how the Zytiga decision plays out, evaluation by independent DSMBs is a compelling topic that involves a number of important issues—too many to address in a single article. One issue that Herper’s thoughtful article did not to address is how DSMBs make decisions. The independent committee members aren’t improvising—they’re following decision rules defined in a charter.
I happen to find the decision rules more interesting than the decision makers. Most DSMBs operate under a charter that specifies how and when they are able to make decisions. To the extent that these issues can be specified ahead of time, this helps ensure a lack of bias. But things also change, sometimes in unanticipated ways, and DSMBs are generally assembled on the basis of their expertise in statistics, medicine, and other areas that enable them to collectively exercise judgment. Appropriate safeguards can be put in place that ensure that the DSMBs, who are generally aware of which treatment each subject received (e.g., not blinded), do not allow knowledge that might affect the conduct of the trial to leak out to those controlling that trial.
Even more interesting is the balancing of many interests in execution of clinical research, some of which conflict. One of these is the business perspective. Many in pharma feel that research should be isolated from the business context, but this ignores many of the practical realities of the world. If research were not such a financial risk, reflected in part by universal alarm at plummeting pharma productivity, maybe there wouldn’t be as many DSMBs to fret about because there wouldn’t be as many studies.
Plenty of people in the pharma industry can define decision rules that signal whether a second trial should be launched before the first is complete in order to accelerate market entry and patient access to a new treatment. The same or similar rules could determine whether a venture capital firm provides additional funding, whether a big pharma company exercises an option to acquire a biotech sponsor and so on. The presumption is, of course, that the risk of introducing bias increases as more people are in a position to infer results from the trial. But knowing that further investment is warranted according to a predetermined decision rule is different from having the ability to intervene in the trial and manipulate results.
By requiring ignorance virtually everywhere in the name of excluding bias, we are paying a high price in other areas, notably R&D productivity. We may well be wasting vast amounts of R&D funding and delaying the availability of medicines to people who need them by insisting on maximum ignorance about complex and expensive scientific experiments. Particularly when we insist on this ignorance for as long as possible and as many people as possible, including those who have to make important decisions about investing in specific research projects important to both pharma companies and medical progress.
In other industries, allocating research funds without current information on which novel products are generating the most promising experimental data would be idiotic. Can anyone envision requiring a semiconductor company to invest billions in a new fab without access to the most current experimental results on which of several alternative approaches to an 11 nm manufacturing process is working in the lab and which is not? Or expecting an aircraft company that must order materials to manufacture a next-generation fighter to choose a material without access to the latest experimental data on the critical properties of advanced alloys and composites?
Pharma executives have no choice. We handicap their decision making in the name of excluding bias from clinical trials. But in an age of sophisticated encryption, access control, information filtering and informatics for decision making, there has to be a better approach to excluding bias. DSMBs and their decision rules may point the way. The people making decisions about the execution of an ongoing trial based on safety and efficacy data aren’t the people who should be making investment decisions and vice versa. But I question whether excluding bias from trials requires excluding as much information from business decision makers as we do. This is not to say that there’s an obvious solution. An Independent Business Decision Making Committee is out of the question. But there might be a body that passes signals along to business decision makers based on rules that the same decision makers approved in advance.