Adapting to Adaptive Studies

International Clinical Trials

international_clinical_trialsBy Michael Rosenberg

Adaptive studies promise significantly lower costs and shorter timelines. Michael J Rosenberg of Health Decisions reviews the organisational foundations for conducting adaptive studies most productively and explains why web-based EDC systems are not up to the task

Dr Michael Rosenberg, MD, MPH is the founder and CEO of Health Decisions. He was awarded Ernst & Young’s Entrepreneur of the Year in Health Sciences in 2002, and has led Health Decisions to recognised success. The author of more than 150 scientific articles, Dr Rosenberg serves on advisory groups in business, technology and medicine. He is Clinical Professor of Obstetrics and Gynaecology for the School of Medicine and Adjunct Professor of Epidemiology for the School of Public Health at the University of North Carolina. He practiced emergency medicine for more than 20 years. His professional achievements have been recognised by fellowship in the American College of Physicians, American College of Preventive Medicine, and the American College of Epidemiology. Dr Rosenberg received his undergraduate and medical degrees from the University of California and his master’s degree in Epidemiology and Biostatistics from Harvard University.

Adaptive clinical research promises profound and dramatic benefits to the pharmaceutical industry. Adaptive programmes, which allow changes during a trial based on analysis of data collected up to the time of the change, provide major advances such as shortening development time, reducing the incidence of treating patients with ineffective or unsafe doses, and allowing regulatory agencies a more nuanced assessment of both efficacy and safety.

Several trends have converged to enable adaptive methods, including increased computational power, more efficient communications technology, and powerful supporting statistical methodologies. Agencies such as the US Food and Drug Administration and the EMEA have acknowledged the significance of adaptive methods in a variety of ways. Moreover, the direct experience of research groups that deal regularly with relevant design and reporting issues, including my own, confirms not only a growing receptiveness of regulators to adaptive methods, but also encouragement of well designed adaptive studies.

This article summarises the organisational and process issues in adaptive research. After briefly discussing the principles of adaptive research and the types of optimisations an adaptive trial can enable, the focus for the balance of the article will shift to what is required to make the powerful engine of adaptive research deliver maximum benefits.

What is Adaptive Research?

The term ‘adaptive research’ broadly denotes changes that can occur during the course of a study or development program without compromising validity or integrity. An adaptive trial design can correct mistaken assumptions or seize an opportunity to optimise the trial. Often the result is shorter timelines, lower costs and superior information for sponsors, regulators and future prescribing physicians.

Adaptive methods may be considered exotic, but in contrast to traditional and prevailing methods of drug research, they are the soul of common sense. The underlying principle is not to wait until the end of a study to find out what went wrong, but to learn from the beginning and use the latest knowledge to fine-tune studies and programmes that are in progress. Examples of adaptive fine-tuning include:

  • Changing sample size to assure that study goals are met at minimal cost
  • Changing how patients are allocated to study arms
  • Starting clinical testing with a greater than usual selection of dosing ranges
  • Pruning the number of dosing arms during dose finding
  • Rolling a Phase II study into Phase III, shortening Phase III time and cost and eliminating the gap between completion of Phase II and initiation of Phase III
  • Focusing the study on a responsive sub-population

The benefit of such an approach is shown in Figure 1, which concerns a dose-finding study. If the information goals of a study can be achieved with 80 patients, a traditional approach would enroll 80 patients each in, say, four dosing arms, and continue the four arms for perhaps 16 months. Assuming an average patient cost of €15,000, the dose-ranging study would cost €2 million.

In contrast, an adaptive study would review data on the four arms as it is generated. It would likely be apparent early on that some doses are outliers and should be excluded, either on the grounds of less efficacy, safety, or both. Cutting off these unproductive arms reduces costs, exposes fewer patients to less safe and/or less efficacious treatments, and also allows the same enrolment pool to be concentrated in fewer arms. The consequence is that the same study can be done four months sooner, with a savings of €1.6 million, while meeting the informational goals of the study. Even greater benefits would flow if advance planning permitted rolling the dose-ranging study directly into a pivotal study. This would use relevant data from the Phase II effort, decreasing costs and saving time.

This example shows why making such changes is dependent on the timely collection and availability of accurate data, a thoughtful and continuous examination of the data, and a commitment to tight day-to-day study management throughout.

Reaping the benefits of the adaptive approach requires organisational and process changes. While such changes always require a degree of effort, the effort required to make the transition to adaptive studies represents an evolutionary, not revolutionary, progression. Adaptive studies do not require major, company-wide changes, but rather a realistic goal of, say, completing a given study 20 per cent faster than traditional means. A major implication of adaptive approaches then becomes immediately apparent: the unit of development becomes the entire programme rather than a single study. This is because adaptive provides the ability to look far further ahead during development and to reduce or eliminate (including contingency planning) choke points. For example, rolling from a Phase II into Phase II programme eliminates the usual six to 12 month gap between completion of Phase II and initiation of Phase III. This step also shortens the Phase III evaluation by utilising the relevant portions of data generated during dose finding as part of the Phase III evaluation. In the long term, these organisational changes enable optimisation of entire programmes.

The commitment to optimise a study or program based on analysis of data as it is collected is not just a matter of adding more interim analyses to existing processes. There must be a continuous flow of up-to-date, clean data. To the extent that data are not available, whether because they have not been entered, validated, or made accessible in a useful form, means study managers are playing with an incomplete deck, increasing the possibility of an erroneous decision or none at all. In our experience, providing a continuous stream of clean, up-to-date, accurate data is perhaps the single most important factor in conducting a successful adaptive study. It is also the most difficult requirement to execute well because it represents a significant departure from a rather lackadaisical norm.

Current Web-Based EDC Systems Don’t Cut It

The cornerstone of any adaptive approach is ensuring the availability of as much useful data as possible as early as possible. The worst sin is to have generated data and yet fail to make it available as a basis for decision making. This applies on both the strategic level (for example, sample size reestimation) and the frequently overlooked day-to-day level (‘tight study management’ of study basics such as recruitment strategies and measures to lower query rates and accelerate query resolution). The requirement for efficient data collection and analysis to manage effectively at both the strategic and tactical levels will surely sound the death knell for pen-andpaper data collection – the lag of weeks or months between data generation and availability precludes making decisions quickly enough. With far better systems widely available, the industry should reject 1950s-era data collection methods that increasingly compromise the performance of clinical studies.

However, current web-based electronic data collection (EDC) systems are also not up to the task of effectively supporting adaptive research. Although these systems are faster than manual data entry, there is an unacceptable delay between the time data are generated at the site and when they are available to guide decision making. The reality of commercial web-based EDC is a lag of weeks or even months. This common delay, the dark secret of web-based EDC, is a consequence of giving sites the burden of data entry. Clinical personnel are loath to perform such tedious tasks, untrained in data entry, and – even apart from the time delays – generally represent the least efficient approach to data entry.

Optimal EDC, then, collects data electronically starting with the CRF and transmits data directly from the electronic input device without the need for re-entry at any stage. An example of such a capability is an optical pen (SmartPenTM), which records strokes as CRFs are filled out. The pen is then docked in a computer station and data is transmitted by the input device to a central database within seconds, from anywhere in the world. Such input systems clearly represent the future of data collection, as they are far faster, less expensive, and more accurate than either paper or web-based EDC systems. The pen configuration also makes data entry as simple and familiar as writing by hand, minimising the need for training.

Integration for Simplicity

Turning incoming data into information – allowing each study team member to see elements important to him – is another simple concept that is difficult to execute well. The current piecemeal approach requires that multiple systems work together effectively, a laudable goal that the computer world has repeatedly shown to be Sisyphean. Just as with desktop productivity suites, all elements must be tightly integrated to function efficiently. Difficulty with effectively integrating multiple computer components, ranging from data collection to randomisation to validation to data reporting, is common. This is especially true for smaller pharmaceutical firms. In practice, well-implemented components such as web-based EDC can easily cost millions each year in training and support. Many companies, especially smaller ones, greatly underestimate this task.

When it comes to conducting adaptive studies, integrated electronic clinical trials management systems may not be the only way, but they are definitely the easy way. Integration should extend to all essential functions, including study startup, monitoring, query resolution, data management and site closeout.

Efficient Communications and Active Management

Anyone who has managed a clinical study knows all too well how frequently communications among sites, monitors, managers and other team members can cause delays. Small delays for many individuals can add up to big delays and higher costs for a clinical study. For an adaptive study, they can also cause missed opportunities to optimise or delayed implementation of optimisations.

Many factors cause delays in communications during a clinical trial, but by far the most significant is reliance on a system and processes that require taking specific action to bring about even the most routine communications. Every time someone has to interrupt a busy schedule to compose an email, place a telephone call, or compile data needed to respond to a communication, there is another opportunity for delay. To the maximum extent possible, routine communications should be generated, distributed and tracked automatically.

The most effective method to automate routine communications today is a secure website designed for the needs and goals of a specific study. Such a custom website can become a convenient source of real-time information on study trends to everyone on the study team. The website reduces the need for individual handling of routine communications in many ways. For example, a common electronic repository for all study documents reduces the need to distribute copies or send replacements for papers lost at a study site. Data automatically downloaded and updated from the study website by ‘dashboards’ or ‘widgets’ (see Figure 2) eliminates the need for many individual messages. With a central website handling the burden of many routine communications and a common knowledge base shared by all team members, person-to-person communications can address important issues more productively.

Consider the example of a programme manager talking on the telephone while looking at a custom widget on the computer screen. The widget shows continuously updated performance metrics, including summaries of enrolment progress. A project manager at the CRO is on the telephone while looking at the same information. Quickly the two confer and agree on criteria for dropping non-performing sites and ways to make up the shortfall at more productive sites. Contrast this with the alternative: the project manager sends an email notifying the programme manager that enrolment is disappointing. The programme manager requests a report. Off goes the project manager to collect the necessary data. Next, the programme manager hastily sets up a spreadsheet to look for trends. Not until the following day does the project manager send a data table and spreadsheet as email attachments. Since the programme manager is unfamiliar with the format, the first 10 minutes of their next telephone call are spent explaining how the report is organised. Then the two managers begin an attempt to identify the worst performing sites. Unfortunately, the rushed report sorts the data by location, not performance. Time passes and tempers grow short.

Just as adaptive research requires up-to-date, accurate data, it also requires managers who are willing to act on the data. Greater efficiency in an adaptive study is the result of active managers making timely, informed decisions and directing specific changes. Such managers will relish the opportunity to make decisions that improve operations. For them, adaptive research is a godsend – an exhilarating liberation from the paradoxical combination of facing responsibility for a study but having very little control while awaiting the outcome. For personalities who prefer to wait and see what fate has in store for their projects, adaptive research may require a change in roles.

An Industry Perspective on Adaptive Research

Although the pharmaceutical industry is one of the most successful and advanced in human history, clinical trials today share characteristics with practices that other industries abandoned as obsolete decades ago. As noted above, traditional clinical trials are run in some respects like a craft. In other respects, typical trials resemble the outdated manufacturing operations that used to keep enormous batches of parts on hand until they were needed to make a car months later. Just-in-time inventories and modern supply – chain management have long since replaced these inefficient batch operations in manufacturing. However, most of today’s clinical trials are still hoarding batches of data for later input, validation or analysis. What is hidden away in these hoarded batches is the knowledge required to manage the study properly.

Rather than modelling historical practices, adaptive research tries to use the best available practices and tools to meet the objectives of clinical research in an optimal way. Work must be organised around efficient, precisely defined processes rather than the varied individual work styles of a 19th century craft. The notion of leaving data hidden away in batches is rejected out of hand.

In essence, adaptive research is about using continually updated information to support timely decisions that optimise a critical business process. That describes nothing more or less than the standard way of managing a modern business. Viewed in this light, the move to adaptive research is not a risky leap into an uncertain future. It is overdue modernisation of a critical process on which the pharmaceutical industry and the world’s healthcare systems depend. What is more, the transition to adaptive research will be easier than many believe. Other industries – and other parts of the pharmaceutical industry such as distribution and manufacturing – have long since developed and refined very similar processes. For clinical research, the time to modernise has come.