Data Challenges (and Solutions) for Externally Controlled Trials

Real-world data and evidence are increasingly being used in health care decisions and publications. However, there are challenges in identifying suitable data for external control arms, and researchers need to consider solutions to address issues like data quality, bias, and selection bias when using real-world evidence for comparative efficacy analyses.¹

Data Challenges for Externally Controlled Trials

Real-world evidence from observational studies has historically had limited use for demonstrating therapeutic effectiveness for regulatory and reimbursement purposes. However, recent developments, such as the 21st Century Cures Act, increased accessibility to large-scale health data, improved standardization of data collection, and regulatory frameworks from bodies like the Food and Drug Administration and European Medicines Agency, have increased the demand for real-world evidence.²

Real-world data comes from sources like electronic health records, medical claims data, and registries, as well as data from mobile phones, wearables, and patient-reported outcomes.³ However, there are a number of technical challenges raised by regulatory and health reimbursement agencies when evaluating comparative efficacy. These include:

• Challenges in data source identification for rare conditions

Randomized controlled trials (RCTs) — long considered the gold standard to evaluate comparative effectiveness of a drug or biological product — may not reflect real-world settings, particularly in rare diseases where standard care is highly variable, recruitment is limited, and patient identification is challenging.

• Outcome and covariate challenges

Traditional clinical trials have established frameworks for monitoring and evaluating clinical effectiveness and analyses using data from RCTs rarely face issues of missingness or stochastic outcome measurements. Conversely, the use of real-world data poses a challenge in terms of availability of outcome measures, and proxies may not represent as clear of an outcome measure as those from a controlled trial, or prospective registry.

• Time selection challenges

The definition of start date for measuring patient outcomes can be challenging in routinely collected health data, leading to issues such as immortal time bias. Related are issues of general temporal biases associated with discordant timeframes of interest.

However, there are several practical solutions for researchers to consider to minimize these challenges.

How Can Challenges Be Minimized?

In my co-authors’ and my recently published article, we break down these challenges and provide practical solutions for researchers to consider through the approaches of detailed planning, collection, and record linkage to analyze external data for comparative efficacy.

• Solution 1: Transparent Prespecified Description of Data Element Definitions and a Detailed Data Analysis Plan
• Solution 2: Data Collection Leveraging Real-World Data
• Solution 3: Record Linkage/Tokenization

For more information on these solutions, click the “Access Publication” link below.

Key Takeaways

Particularly with the recent guidelines from the FDA on Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Projects, a heavy emphasis is placed on the choice of external data. The FDA asks sponsors to describe all data sources that were accessed and to provide reasons as to why data sources were excluded in efforts to reduce the perception of “cherry-picking” and ensure the suitability to the research plan.

Identifying and evaluating real-world data sources for comparative effectiveness studies has many challenges. To mitigate, researchers are encouraged to develop detailed analysis plans early and consider data-collection strategies through de novo collection and tokenization. The solutions suggested in the paper may minimize these challenges, but the selection and evaluation of a good real-world data source is not a straightforward endeavor.

For more information on the data challenges for externally controlled trials, including case study examples and applications of solutions, click below to read “Data Challenges for Externally Controlled Trials: Viewpoint”:

1. Jaksa A, Louder A, Maksymiuk C, Vondeling GT, Martin L, Gatto N, et al. A Comparison of Seven Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies. Value Health 2022 Dec;25(12):1967-1976.
2. Mahendraratnam N, Mercon K, Gill M, Benzing L, McClellan MB. Understanding Use of Real-World Data and Real-World Evidence to Support Regulatory Decisions on Medical Product Effectiveness. Clin Pharmacol Ther 2022 Jan;111(1):150-154; and Gabay M. 21st Century Cures Act. Hosp Pharm 2017 Apr;52(4):264-265.
3. Use of Electronic Health Record Data in Clinical Investigations Guidance for Industry. Food and Drug Administration.

Read more from Perspectives on Enquiry & Evidence:

Sorry no results please clear the filters and try again

FDA Guidance on the Design and Conduct of Externally Controlled Trials — What to Watch

The U.S. FDA has recently provided specific guidance[i] on the design and conduct of trials incorporating an external...

Read more

Demystifying Synthetic Control Arms

Last week, Cytel Director & Research Principal Louis Dron discussed new FDA guidance on the design and conduct of...

Read more

Can RWE Help Restore Decades of Health Inequalities? Yes, and Here’s How

Health inequalities are an enduring issue that can be exacerbated by clinical trial recruitment that does not reflect...

Read more

If you'd like updates on our blog posts, sign up for email updates below.