In March 2006, the US FDA published the ‘Critical Path Opportunities Report’ – a discussion aimed at addressing specific actions that could be taken by the pharmaceutical and biotechnology industries to address what had become known as the ‘pipeline problem’. Data had shown that, despite the fact that over the previous decade R&D spending by the US pharmaceutical industry had more than doubled, the number of novel therapies reaching the market had declined. The drug industry was spending more but producing fewer successes.
One section of this FDA report focused on streamlining clinical trials, and it outlined a number of problems with traditional clinical trials. In the traditional approach, a clinical trial is designed based on what we think we know about the characteristics of the test product, before the trial is started and run to completion. We hope that when the trial is complete, it will have provided all of the information we need to progress to the next stage of the drug development process. Most importantly, during the trial, we do not allow ourselves to look at the information that is accumulating as the trial progresses, and so we cannot learn from the trial. This is clearly inefficient. Very often, we have limited information available to us at the trial design stage, and the information that accumulates during the trial will typically soon surpass, in both quantity and quality, the information we had before the trial started. Why ignore that accumulating information?
Among the many ideas suggested in the FDA report, the concept of a ‘learning trial’ was prominent. The report noted that learning trials "have a different conceptual framework and require a (different) statistical approach".
Adaptive trials: definitions and goals
At roughly the same time, the EMEA Think Tank on Innovative Drug Development also produced a report, and referred explicitly to the value of "adaptive/flexible designs that permit changes to important design characteristics based on accumulating interim data".
Encouraged by these regulatory initiatives, the Pharmaceutical Research and Manufacturers of America (PhRMA) set up a working party on adaptive clinical trials, which reported in 2006. This working party gave us the now standard definition of an adaptive clinical trial: "By adaptive design we refer to a clinical study design that uses accumulating data to decide how to modify aspects of the study as it continues, without undermining the validity and integrity of the trial. The goal of adaptive designs is to learn from the accumulating data and to apply what is learned as quickly as possible".
This flurry of activity in the early 2000s does not mean, though, that the concept of an adaptive clinical trial was new: ideas about adaptive trials actually have a long history. One of the earliest published academic papers on adaptive methodology appeared in 1933, in the statistical journal Biometrika, and much statistical work was done during the second half of the last century on methods for the design and analysis of various types of adaptive clinical trial.
But the FDA and EMEA’s explicit endorsement of the idea of adaptive design has given these methodologies a significant boost, and they have in recent years attracted the attention of pharmaceutical companies both large and small, in the hope that the drug development process can be made more efficient and effective, which might mean:
- stopping the trial earlier (either because of early success or clear evidence of failure)
- recruiting fewer patients
- identifying more accurately the appropriate number of patients
- to recruit
- assigning patients to more informative or relevant doses or treatments
- identifying a target objective (e.g. dose) more accurately and more precisely
- moving more quickly to the next stage of the drug development process
- making savings in time and money.
These efficiencies should arise from intelligent use of the information we gather as the trial progresses: in an adaptive trial, we analyse the accumulating data at intervals, and, using pre-defined formal statistical rules, we modify one or more aspects of the trial design to optimise its performance.
Various aspects of the study design might be addressed during this learning process, leading to a number of standard types of adaptive clinical trial:
- Sample size re-estimation designs: at one or more interim analyses during the trial, we recalculate the sample size likely to give the desired outcome from the trial: this allows us to correct the initial and often inaccurate estimates made using the information available prior to the trial.
- Adaptive interim analyses or group sequential designs: in this type of study, we analyse response data at intervals during the trial, and, based on these responses, we may terminate the trial early, either because of obvious success, or likely failure, or we may drop certain treatment arms or doses from the trial due to suboptimal performance.
- Adaptive randomisation designs: here we again analyse response data at intervals during the trial and, based on relative responses on different treatments, we modify the proportions of patients assigned to the treatment arms, with a view to recruiting more patients to the more effective regimens.
- Adaptive dose-finding designs: with these designs, we use formal statistical rules applied to accumulating data to decide which doses should be studied during the dose-finding process, and/or how many patients should be assigned to each dose, in order to optimise the process of identification of the target dose.
- Seamless adaptive trials: in a seamless adaptive design, we begin with an integrated protocol defining two or more stages in drug development – for example a dose-finding phase followed by a confirmatory phase in which the dose is tested in more patients. Using data accumulated during the first phase, we apply formal rules to help decide when to optimally transition one stage to the next, and then we move on without delay.
Various other types of adaptive design are also possible, but all have in common the application of formal statistical rules, based on interim data analyses, which help us to modify aspects of the trial design in order to optimise its performance.
Draft guidelines issued
Increasingly, pharmaceutical and biotechnology companies are considering these adaptive strategies as part of the process of clinical development planning, and increasing numbers of adaptive clinical trials are being implemented. In response to this, the FDA published a draft ‘Guidance for industry: adaptive design clinical trials for drugs and biologics’ in 2010, setting out the regulators’ perspective on the use of these designs as part of drug development programmes.
The key issue for regulators, of course, is that clinical trials must produce reliable and scientifically valid results, and so regulatory attitudes to adaptive clinical trials are concentrated on those aspects of the adaptive trial that might influence validity and interpretability. What most threatens this is the core process of the adaptive trial – looking at and analysing accumulating data. Clearly, the process of looking at accumulating data might introduce bias into the conduct of the trial. It might mean that changes are made to the type of patient recruited or that treatment identities are inadvertently unmasked. And the process of repeated statistical analysis of accumulating data, which is a key part of an adaptive trial, requires that special methods are needed, which will control the statistical properties of the trial so that reliable and valid answers are produced.
The application of adaptive methodology
So running an adaptive clinical trial requires special attention to the processes used to collect, review and analyse the accumulating data, and it is this that regulators are most concerned about. Fortunately, many standard techniques exist to allow these processes to be controlled to the regulators’ satisfaction, and statisticians are continuing to develop novel methodologies to deal with repeated analyses of data during adaptive processes. The application of Bayesian statistical methods is particularly useful in an adaptive context. Even so, because of these issues, adaptive methodologies are often easier to apply in the earlier stages of drug development when regulatory concerns will typically be less acute.
Although adaptive trials can often improve the efficiency of the drug-development process, it is important to understand that application of adaptive methodology can be demanding and will often require more time to be devoted to the planning phase of the trial. The logistical aspects of the study will often be more critical – adequate resources and systems are required to manage the potentially more complex processes of data collection, drug supply, patient recruitment and randomisation.
But even though adaptive trials have attracted a lot of attention and are becoming more widely used, it should not be assumed that such a trial is always the best option. Traditional trials, when designed with good-quality information, can out-perform adaptive trials. It is important to evaluate alternative designs and adaptive trials should always be compared with traditional alternatives. Even if an adaptive approach offers some potential benefits for a specific trial it may not be the best option when all factors are considered. Ask what is best for the overall development programme and not just the next trial.
Nevertheless, it is clear that in the future we will see adaptive methodologies applied more frequently in clinical trial design, and it is to be hoped that their use will bring the desired result for the industry – more drugs brought to the market quickly and efficiently, with associated reductions in clinical trial costs.
