The U.S. FDA has recently provided specific guidance[i] on the design and conduct of trials incorporating an external control group, sometimes known as a synthetic control arm. Their guidance represents the culmination of several other topics related to these trial designs and research themes, such as existing guidance on the use real-world data and real-world evidence,[ii] the use of electronic health records and medical claims to support regulatory decision-making,[iii] and guidance on demonstrating substantial evidence of effectiveness for drugs and biological products.[iv]
As such, in many ways, what the FDA provides is not a drastic change to several longer-standing policies regarding the appropriateness of such designs, and the critical caveats associated with such data. Their guidance is split into four principal sections, covering topics relating to design, data considerations, and analysis considerations for an externally controlled trial, alongside a section regarding communication and access to data with the FDA.
Design considerations
A key point that is explicitly stated by the FDA is that the design stage of any potential externally controlled trial is the best stage to address concerns regarding bias. Significant attention is brought with respect to topics such as estimands, assuring alignment of trial in/exclusion criteria and selection of covariates, as well as immortal time bias and appropriate selection of index date.[v] Specific examples are provided throughout of the types of challenges associated with overcoming these characteristics. The guidance notes that these are best dealt with at the protocol stage, and this emphasizes the importance of rigorous pre-study considerations for any eventual trial.
One critical aspect of the design considerations is a topic related to the choice of external data. The FDA directly asks that sponsors document and describe in any associated study protocol all data sources that were accessed and to provide information on why data sources are excluded. In the last two years, Cytel has supported over 5 regulatory-focused data landscaping activities both in the U.S. as well as the EU, covering multiple therapeutic indications ranging from oncology through to respiratory diseases. It is important to note that these activities can take time and should be built into any considerations for communication with the agency.
Data considerations
For externally controlled trials, there is no formal restriction on where and in what format data comes, provided it meets the associated quality criteria. Historical submissions to regulatory bodies have included data from other clinical trials, natural disease cohorts, as well as data obtained from routinely collected health data. Interestingly, the FDA does specifically highlight some potential challenges for certain data types. In particular, they emphasize concerns regarding clinical trials for which the outcomes of the trial are known a priori.
“A particular concern for bias would be the selection of an external control arm from a completed trial whose outcomes are already known,” the guidance details. “This would be especially problematic if the results of the external control arm are inconsistent with prior experience.”
For real-world data, they emphasize pitfalls that are well-known, such as missing data, and limited follow-up durations.
A critical table, which neatly distills many FDA topics regarding data concerns and biases, is provided and is essential reading for any group wishing to explore the potential of external evidence to support their comparative efficacy claims.
Analysis considerations
With regards to guidance on analysis considerations, the FDA is relatively non-specific on statistical approaches, consistent with earlier guidance and overviews of submissions to the agency.[vi] Rather, they favor the use of critical thinking and concepts relating to minimizing biases, which can occur during the generation of even a well-designed external control population.
“FDA does not recommend a particular approach to analyzing data from externally controlled trials. No single statistical or analytical method will be suitable for all trials involving external control arms, and potential approaches should be discussed with the appropriate FDA review division.”
Prior applications and Cytel experience indicates that demonstration of appropriate consideration and in/exclusion of certain statistical approaches can help to assure reviewers that a submission is guided by the best available statistical approach, whether this is Bayesian modelling or even more traditional propensity matching methods.
The FDA does note the importance of approaches to acknowledge limitations inherent in any externally controlled dataset, focusing on missing data and misclassification of data. Here, they encourage the use of quantitative bias analysis as a means to understand the potential for bias and what impact it may have on any conclusions. In this way, the guidance feeds back into the importance of understanding your data, your design, and then providing a means to acknowledge any limitations and explore the influence they have on your findings in a transparent and quantitative manner.
Communication with the FDA
As with any non-traditional design, communication is critical. The FDA asks that sponsors consult early during the development program of a drug to ensure that the agency can provide feedback on the appropriateness of any approach proposed. Here, they ask about four key topics: (1) reasons why the proposed study design is appropriate, (2) proposed data sources for the external control arm and an explanation of why they are fit for use, (3) planned statistical analyses, and (4) plans to address FDA’s expectations for the submission of data. While the FDA guidance does not address timelines, it is important to note that the development of these materials can be a time-consuming process. Ensuring that an appropriately detailed exercise has been undertaken to evaluate the appropriateness of the proposed design, as well as to fully evaluate all eligible data options is key for both the sponsor and the FDA. For the sponsor, assurance that the most high-quality data that is fit-for-purpose is key, as today there are many commercial data vendors and options, alongside more traditional evidence sources such as trials and registries.
Closing remarks
While the FDA guidance does not move the needle considerably with respect to their position on these unique design options, it does provide a well-described, clear, and concise insight into the agencies’ considerations. In laying out the multiple components of the process, it helps to provide sponsors with a clear acknowledgement that these designs necessitate a highly specialized, adaptive, and multidisciplinary team to ensure success.
For more information on synthetic and external control arms and leveraging the power of real-world data, or to contact us, click below:
[i] https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-design-and-conduct-externally-controlled-trials-drug-and-biological-products
[ii] https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-real-world-data-and-real-world-evidence-support-regulatory-decision-making-drug
[iii] https://www.fda.gov/regulatory-information/search-fda-guidance-documents/real-world-data-assessing-electronic-health-records-and-medical-claims-data-support-regulatory
[iv] https://www.fda.gov/regulatory-information/search-fda-guidance-documents/demonstrating-substantial-evidence-effectiveness-human-drug-and-biological-products
[v] https://www.sciencedirect.com/science/article/abs/pii/S1047279722003088
[vi] https://pubmed.ncbi.nlm.nih.gov/32440224/
Read more from Perspectives on Enquiry and Evidence:
Sorry no results please clear the filters and try again
Can RWE Help Restore Decades of Health Inequalities? Yes, and Here’s How
Developing Synthetic Control Arms Using Bayesian Models
Using Quantitative Bias Analysis in Real World Data Strategy
If you'd like updates on our blog posts, sign up for email updates below.
