Rick Farris, COO, Health Decisions, talks about ensuring more efficient international trials
Winter 2009 - A biotech company’s high-stakes confirmatory study of a new biologic treatment of brain metastases involved 92 sites in eight countries. When the FDA rejected the company’s application for marketing approval, the company’s stock lost almost half its value. The CEO blamed a few sites in one country that had not administered radiation as promptly as required, got poor results, and skewed the entire study.[3-4] If the company had known about the issue during the study, it could potentially have corrected the problems that led to poor quality data. Instead the company had to change strategy. The foul-up at a few sites wasted several precious years and cost investors millions of dollars.
Familiar Horror Stories
This company’s travails are familiar: Virtually every study includes sites that perform poorly, whether measured by enrollment, data quality, or other measures. International studies, increasingly a fact of pharma life, are even more prone to problems of all descriptions.While some are detected, problems are typically detected late and response is limited. Even worse, many “small” issues often go unrecognized despite cumulative and substantial effects on enrollment, timelines and costs.
International studies suffer such disappointments so often that we may take them as a given. We think we have to take the bad with the good, to accept higher risks to get benefits such as lower costs and better access to patient populations. When managing disparate groups — across language, cultural barriers, regulatory differences and time zones — risks seem to grow exponentially, given the number of sites involved. Many companies today are finding that the initial appeal of quick enrollment and inexpensive labor fades when quality proves poor. As the individual in charge of R&D for a major multinational stated, “A bad subject in China is just as bad — and sometimes worse — than a bad subject in the U.S.”
Many companies have turned away from international studies, concluding that they can take longer and cost more when all is said and done. Taking studies international no longer seems a surefire way to increase the efficiency of clinical research. It’s hard enough to manage multiple sites that speak the same language, share the same culture, and face the same regulations. Even then, 20 sites can be as different as 20 breeds of cat, and herding cats is never easy. Undetected issues are the biggest problem in managing clinical studies. An Alzheimer’s study uses standardized assessments of memory and cognitive ability. Minimizing the variability of data is essential for controlling study statistical power and ensuring that sample size will be adequate. Reducing variance allows reducing sample size and costs. On the other hand, if you plan on variance of a certain percentage and set sample size accordingly, poor quality control can increase variance enough to undermine the power calculations — also increasing the likelihood of a failed trial even if the drug actually works.
A Study with a Happy Ending
Managers of one Alzheimer’s study addressed the issue of minimizing variability in subjective assessments through a system that provided a timely flow of data from all sites. Tracking of data affecting variability alerted managers to a sudden change in ADAS-cog outcome measures (measures of cognitive performance). The individual responses did not trigger a query because they fell within the prescribed range. Nevertheless, automated tracking detected the large and sudden change. Prompt detection allowed correcting a problem that might have increased variability and compromised the study’s ability to detect a difference between the investigative drug and comparator.
What made the difference between a devastating surprise in one study and a satisfactory outcome in the other? It was the trial management system. The nasty surprise was typical of what happens with trial management systems in wide use today. The satisfactory outcome was typical of what happens with a trial management system based on the approach that companies in other major industries use to manage complex operations at multiple locations. Efficient major industries don’t just parcel work out, they replicate the operation at each location. Each site uses the same standardized processes and collects the same data, and that data includes numerous performance measures. Managers and workers at each location receive identical training in performing the same roles and tasks with the same tools. Central managers have a timely flow of information from all sites. Thanks to standardization, the information coming in allows managers to compare status and performance metrics for the same tasks at each site. The lesson is that effective management of any complex distributed project requires a flow of information reporting the status and tracking the performance of the same standardized processes at each location.
Our industry has yet to learn the same lesson in clinical research. If you have managed a study that used a typical web-EDC system, you know it didn’t standardize key processes at all sites, or provide timely information to track performance of those processes. Unfortunately, EDC systems also usually leave data on paper for 48 hours to a week after collection. Such delays allow errors to proliferate before anyone realizes there’s a problem. Fixing those errors often requires doing work a second time, meaning previously expended time, effort and money have been wasted.
Web-EDC systems focus overwhelmingly on patient data. Such systems often lack the kind of detailed information that helps identify data quality issues. The systems offer managers little help improving major activities central to achieving study goals: ensuring rapid enrollment, minimizing rework (queries) and waste, cutting monitoring costs while assuring data accuracy, and achieving rapid database lock. Effective study management requires not just patient data, but a continuous stream of information reflecting the status of major operations such as enrollment, patient visits and assessments, case report form (CRF) submission, query processing, site monitoring, closeout, and progress toward database lock. If web-EDC systems collect such information at all, the information doesn’t reach study managers quickly.
The Path to Standardization
While producing thousands of identical items in factories that use standardized processes has worked wonders for manufacturing, standardizing operations in clinical studies, especially international studies, presents greater challenges. You can’t put identical sites all over the world and wouldn’t want to if you could. But we’re not talking about cloning physical facilities. Different industries standardize operations at distributed sites in different ways. For example, restaurants standardize operations through franchises, often with local variation to accommodate local culture and preferences.
We already standardize clinical studies in some important ways. We define protocols with great care to ensure adherence at all sites. We ask sites to perform the same medical procedures in the same way. We rely on standardized laboratory assays. We also define and follow Good Clinical Practices.
But protocols are about standardizing clinical research and Good Clinical Practices are about making sure every detail meets regulatory standards — not conducting an efficient study. We need to extend standardization to study operations and the way we manage studies. We have to be able to conduct studies wherever the patients are, and yet get the same performance and quality. Across all sites and study roles, we need uniform and timely tracking and reporting of enrollment progress, screen failures and the reasons for them, patient visits scheduled and missed, submission of CRFs from each patient visit, the number of queries, queries resolved, and queries outstanding, the average time to resolution, the amount of Source-Data Verification remaining at each site, the average number of fields that each monitor can verify in a day, and many other events and metrics to track progress and managers.
As an industry, we have been slow to adopt such a systematic, standardized approach to study and site management. Study managers and staff at all levels generally lack tools to track critical activities in many different places. Horror stories are all too common because managers may not understand the reality in the field until the common “pain points” of clinical research are causing major headaches. Enrollment may lag far behind goals at all sites in one country or some sites in many countries. Patient visits may be missed or behind schedule, and CRFs for patient visits may not be turning up when they should. The number of unresolved queries may reach disturbing levels at some sites. In short, before managers have any idea that there’s a problem, a study can drift out of control. This must change.
A Better Way: Networked CROs and Agile Systems
We can manage operations more efficiently in international trials through improved standardization and tracking. No, the answer is not cloning clinics, granting franchises, or even opening branch offices. We must combine the power of a new, more rigorous approach to study management with a distributed model that standardizes the way operations are conducted at numerous sites regardless of location. Furthermore, to ensure the ability to go wherever future patients may be found, we need to adopt a networked approach that can work not only across multiple sites, but across multiple CROs collaborating on the same study. A study reaching into an additional region can bring in a CRO with local expertise and connections.
The agile approach to managing international trials uses the same clinical development platform at all sites and for all participating CROs. The platform makes tracking activities at multiple distant sites quick and easy. Timely, objective information on trial status and site performance makes it possible to identify performance problems in their infancy. With such information, the study team can focus on preventing emerging issues from escalating rather than reacting after problems jeopardize the study. Study managers can also help sites perform better by identifying problems and suggesting solutions. To increase the productivity of the study team, the agile platform provides information to each team member based on individual roles at multiple levels, whether monitor, project manager, or program manager.
Standardized processes also provide benefits for staffing and individual productivity. Aided by the system, less trained people can be highly productive. Following standardized processes reduces subjectivity and increases consistency across the study. The emphasis on uniform training in standard procedures removes the emphasis on guild-like individualized training. The old guild-like transfer of skills increases the chances of different people doing the same task in different ways with different results.
An agile system also recognizes the importance of data collected by site-management software and the monitoring process. Such information includes the status of subject screening and reasons for screen failures, randomization, subject visits, discontinues, and CRF entry. Convenient tools to track these and other indices of study progress and staff productivity are invaluable to project managers, sponsors, and individual sites.
As with other globally distributed operations, training is an essential component in making the agile approach work at peak efficiency across the whole network. All CROs in the network and all sites need training in the use of the system, including both an initial company certification process and ongoing training to maintain proficiency.
Advantages of an Integrated Network of CROs
You may assume that the most effective way to ensure standardization in a global study with many sites in different regions is to assign the task to a single global CRO. But don’t forget the hard facts you learn when you work for almost any large global company in any field. These companies tend to become hierarchical and bureaucratic, and bureaucracies take time circulating information, pondering decisions and getting messages out. These companies also have numerous customers and prioritize the work for their biggest accounts. If you’re a new or small account, you may not get a big global company’s best effort.
Furthermore, there is no assurance that a large global company’s trial management system continuously tracks performance at all sites, ensures that data is keyed in promptly from paper CRFs, or provides quick and efficient data flow from collection into validation and query processing. A large, integrated company does not automatically integrate all the sites in a global study into a single operation that maximizes efficiency and minimizes timelines and risks. It is not the number of companies involved that brings about efficiency in a global study, but the ability of the trial management system to provide a flow of timely data and performance metrics for all sites.
Shared use of an agile trial management system can even enable a network of separate CROS to manage an international study more effectively than a single larger company. Integration of timely data collection, cleaning, and validation with data handling, analysis, and reporting allows more effective management of a networked approach than a traditional single company framework based on typical manual or EDC processes. Managers in the United States provide close central oversight based on performance metrics for all sites. Precisely defined processes and performance standards allow managers to improve efficiency across all locations. Partners with regional experience and expertise provide local oversight and insight. The ability to bring in CROs in new regions allows flexibility in seeking patient populations.
Rapid, integrated communications and reporting not only unite the partnership network, but also provide sponsors with transparency on timely information about study status throughout the world. Furthermore, earlier access to more detailed information from across the network enables sponsors to recognize issues earlier. Sponsors have greater control because they can make better-informed decisions to determine the course of their studies. The ability to make such decisions based on timely information reduces risk by preventing problems from escalating before the sponsor has a chance to react. In summary, the integrated network approach to global trial management provides transparency, increases sponsor control, and allows decision-making that reduces risk. The overall result is fewer false steps and shorter timelines.
The Rejected Drug
At the start of this article, the CEO of the company sponsoring the new biologic asked the FDA to approve the drug despite bad results from a few sites. The CEO should also have asked why the trial management system failed to alert study managers promptly when those sites failed to administer radiation in the period specified. Mistakes always happen in complex clinical studies. You don’t know where or when or exactly what the mistakes will be, but you know they will happen. No amount of person-to-person contact can track everything and identify deviations and trends at individual sites and across whole studies. Only a systematic approach that provides continuous tracking of all key operations and timely reporting to the study team can prevent, detect, and correct the many slip-ups large and small that can compromise a clinical study. The agile approach provides such a systematic approach and extends it to as many sites as necessary regardless of location.
Rick Farris is COO at Health Decisions, a full-service CRO focusing on high-efficiency adaptive solutions. Farris is leading the development of Health Decisions’ new Agile Network — an international community of CROs who are trained and certified in the company’s Agile Clinical Development methodology. Through 20 years of experience in clinical research on a global scale, Health Decisions has fine-tuned the Agile approach — which combines adaptive design with adaptive operations — to form a set of best practices which consistently cuts trial timelines, increases transparency, and delivers results ten to 40 percent faster than traditional approaches.
- Randomized phase III trial of Xcytrin® (motexafin gadolinium) injection for the treatment of brain metastases in patients with non-small cell lung cancer undergoing whole brain radiation therapy. 2003. Available from: ClinicalTrials.gov/show/NCT00054795
- BioWorld Week. 2005 Dec 26.
- Miller R. Cancer regression. The Wall Street Journal. August 1, 2007. Available from: http://online.wsj.com/article/ SB118593325021784255.html
- Harris G. Where cancer progress is rare, one man says no. The New York Times. September 16, 2009.
- Pharmacyclics press release. Pharmacyclics submits new drug application for Xcytrin for the treatment of lung cancer brain metastases. 206 Dec 22. Available from: http:// ir.pharmacyclics.com/releasedetail.cfm?ReleaseID=223235
- Redfearn S. More paper, more burden? ClinPage. 2009