Real-world data and evidence are increasingly being used in health care decisions and publications. However, there are challenges in identifying suitable data for external control arms, and researchers need to consider solutions to address issues like data quality, bias, and selection bias when using real-world evidence for comparative efficacy analyses.1
Data Challenges for Externally Controlled Trials
Real-world evidence from observational studies has historically had limited use for demonstrating therapeutic effectiveness for regulatory and reimbursement purposes. However, recent developments, such as the 21st Century Cures Act, increased accessibility to large-scale health data, improved standardization of data collection, and regulatory frameworks from bodies like the Food and Drug Administration and European Medicines Agency, have increased the demand for real-world evidence.2
Real-world data comes from sources like electronic health records, medical claims data, and registries, as well as data from mobile phones, wearables, and patient-reported outcomes.3 However, there are a number of technical challenges raised by regulatory and health reimbursement agencies when evaluating comparative efficacy. These include:
Challenges in data source identification for rare conditions
Randomized controlled trials (RCTs) — long considered the gold standard to evaluate comparative effectiveness of a drug or biological product — may not reflect real-world settings, particularly in rare diseases where standard care is highly variable, recruitment is limited, and patient identification is challenging.
Outcome and covariate challenges
Traditional clinical trials have established frameworks for monitoring and evaluating clinical effectiveness and analyses using data from RCTs rarely face issues of missingness or stochastic outcome measurements. Conversely, the use of real-world data poses a challenge in terms of availability of outcome measures, and proxies may not represent as clear of an outcome measure as those from a controlled trial, or prospective registry.
Time selection challenges
The definition of start date for measuring patient outcomes can be challenging in routinely collected health data, leading to issues such as immortal time bias. Related are issues of general temporal biases associated with discordant timeframes of interest.
However, there are several practical solutions for researchers to consider to minimize these challenges.
How Can Challenges Be Minimized?
In my co-authors’ and my recently published article, we break down these challenges and provide practical solutions for researchers to consider through the approaches of detailed planning, collection, and record linkage to analyze external data for comparative efficacy.
Solution 1:Transparent Prespecified Description of Data Element Definitions and a Detailed Data Analysis Plan
Solution 2: Data Collection Leveraging Real-World Data
Solution 3:Record Linkage/Tokenization
For more information on these solutions, click the “Access Publication” link below.
Identifying and evaluating real-world data sources for comparative effectiveness studies has many challenges. To mitigate, researchers are encouraged to develop detailed analysis plans early and consider data-collection strategies through de novo collection and tokenization. The solutions suggested in the paper may minimize these challenges, but the selection and evaluation of a good real-world data source is not a straightforward endeavor.
For more information on the data challenges for externally controlled trials, including case study examples and applications of solutions, click below to read “Data Challenges for Externally Controlled Trials: Viewpoint”:
Jaksa A, Louder A, Maksymiuk C, Vondeling GT, Martin L, Gatto N, et al. A Comparison of Seven Oncology External Control Arm Case Studies: Critiques From Regulatory and Health Technology Assessment Agencies. Value Health 2022 Dec;25(12):1967-1976.
Mahendraratnam N, Mercon K, Gill M, Benzing L, McClellan MB. Understanding Use of Real-World Data and Real-World Evidence to Support Regulatory Decisions on Medical Product Effectiveness. Clin Pharmacol Ther 2022 Jan;111(1):150-154; and Gabay M. 21st Century Cures Act. Hosp Pharm 2017 Apr;52(4):264-265.
Use of Electronic Health Record Data in Clinical Investigations Guidance for Industry. Food and Drug Administration.
Read more from Perspectives on Enquiry & Evidence:
Sorry no results please clear the filters and try again