Randomized control trials (RCTs) are the gold standard for estimating the efficacy of a treatment. They allow us to investigate treatment effects in a controlled setting designed to ensure reproducibility of the results, quality of data, and minimization of selection bias. In contrast, comparative effectiveness from real world data (RWD) is traditionally regarded as inferior evidence, primarily due to its lack of randomization and limited availability of information on potentially relevant prognostic factors. As such, real world evidence (RWE) has been mostly used to corroborate RCT results and for post-marketing monitoring.
Recently, the growing body of data from the real world, which accompanied the digitalization of health, has drawn increasing attention to RWD and its potential, leading to questions on whether comparative effectiveness evidence from RWD could support regulatory and HTA decisions.
A brief history of RWE for Regulatory Approval
In 2016, with the mandate of the 21st Century Cures Act, the US Food and Drug Administration (FDA) has established the Real-World Evidence Program to evaluate the use of RWD in support of regulatory approvals and post-approval safety studies. In 2017, The European Medicines Agency (EMA) formed the HMA/EMA Joint Big Data Task Force to establish a roadmap for the use of RWD in regulatory assessments[1].
The framework for US FDA’s RWE Program explicitly mentions that replicating RCTs using RWD may shed some light into the opportunities and challenges of producing comparative evidence from observational studies [2]. Specifically, RCT emulation following a “target trial” approach (Hernán and Robins) [3] can help calibrate RWD against trials [4]. The target trial approach asks us to emulate a randomized experiment even when working with observational data. The target can be a thought experiment or an actual RCT already conducted. When targeting an RCT that has already produced results, the closer the observational study comes to emulating the results of the trial the more faith we can have in it producing scientifically rigorous results. A number of efforts are being made in this direction, such as the RCT DUPLICATE [5] project, which aims to replicate 30 completed Phase III or IV trials and to predict the results of seven ongoing Phase IV trials using Medicare and commercial claims data in the US.
The uses of RWD in Oncology
A particularly challenging area for incorporating RWD into regulatory evidence is oncology, where treatment decisions and efficacy results are dependent on a number of clinical characteristics that are normally not observed in common RWD sources. This includes disease staging, performance status, and results of mutation tests. While quasi-experimental methods like propensity score matching (PSM) aim to reproduce the effects of randomization in RWD, such techniques can only adjust for observable characteristics, and any residual bias from unobserved confounders remains. With a focus on US data, prior research has reached mixed conclusions regarding the replicability of oncology RCTs using RWD[6,7].
Innovative Approaches: Example
In a recent publication on the Journal of Comparative Effectiveness Research [8] we contributed to the literature emulating the CHAARTED trial [9,10] using German claims data. CHAARTED was an RCT in metastatic hormone-sensitive prostate cancer (mHSPC) comparing docetaxel in combination with androgen deprivation therapy (ADT) vs ADT monotherapy. In CHAARTED, docetaxel+ADT was associated with longer median overall survival (OS) compared to ADT alone in the intention to treat (ITT) population. We implemented the inclusion and exclusion criteria of the CHAARTED trial as closely as possible with the available information in the data. The real-world population included in the study was significantly older and had a shorter median OS compared to the trial population. However, the hazard ratio (HR) for OS was concordant with the trial and, after adjusting for baseline characteristics with PSM, the point estimate from the RWD (0.71) was very close to the trial estimate (0.72).
While sample sizes were small and findings need validation across different data sources and indications, our study suggests that after observable confounder adjustment claims data have the potential to produce unbiased estimates of relative efficacy for OS. This opens to the use of RWD estimates of relative efficacy within different contexts, such as within Bayesian hierarchical network meta-analyses or to inform outcome-based reimbursement agreements, as well as to support regulatory and HTA decision making.
Click the button to read more about this in our publication:
References
[1] Heads of Medicines Agencies. HMA/EMA Joint Big Data Steering Group [Internet]. [cited 2021 Feb 4]. Available from: https://www.hma.eu/509.html
[2] U.S. Food and Drug Administration. Framework for FDA’s Real-World Evidence Program [Internet]. 2018 [cited 2021 Feb 4]. Available from: https://www.fda.gov/media/120060/download
[3] Hernán MA, Robins JM. Using Big Data to Emulate a Target Trial When a Randomized Trial Is Not Available. Am J Epidemiol [Internet]. 2016 Apr 15 [cited 2021 Feb 4];183(8):758–64. Available from: https://pubmed.ncbi.nlm.nih.gov/26994063/
[4] Schneeweiss S. Real-World Evidence of Treatment Effects: The Useful and the Misleading [Internet]. Vol. 106, Clinical Pharmacology and Therapeutics. Nature Publishing Group; 2019 [cited 2021 Feb 4]. p. 43–4. Available from: https://pubmed.ncbi.nlm.nih.gov/30942896/
[5] RCT DUPLICATE - Home [Internet]. [cited 2022 Mar 25]. Available from: https://www.rctduplicate.org/
[6] Kumar SK, Dispenzieri A, Lacy MQ, Gertz MA, Buadi FK, Pandey S, et al. Continued improvement in survival in multiple myeloma: changes in early mortality and outcomes in older patients HHS Public Access. Leukemia [Internet]. 2014 [cited 2020 Feb 4];28(5):1122–8. Available from: http://www.nature.com/authors/editorial_policies/license.html#terms
[7] Soni PD, Hartman HE, Dess RT, Abugharib A, Allen SG, Feng FY, et al. Comparison of population-based observational studies with randomized trials in oncology. J Clin Oncol [Internet]. 2019 May 10 [cited 2021 Feb 4];37(14):1209–16. Available from: https://ascopubs.org/doi/10.1200/JCO.18.01074
[8] Ghiani M, Maywald U, Wilke T, Heeg B. Bridging the gap between oncology clinical trials and real-world data: evidence on replicability of efficacy results using German claims data. https://doi.org/102217/cer-2021-0224 [Internet]. 2022 Mar 22 [cited 2022 Mar 25]; Available from: https://www.futuremedicine.com/doi/abs/10.2217/cer-2021-0224
[9] Sweeney CJ, Chen Y-H, Carducci M, Liu G, Jarrard DF, Eisenberger M, et al. Chemohormonal Therapy in Metastatic Hormone-Sensitive Prostate Cancer. N Engl J Med [Internet]. 2015 Aug 20 [cited 2021 Apr 19];373(8):737–46. Available from: http://www.nejm.org/doi/10.1056/NEJMoa1503747
[10] Kyriakopoulos CE, Chen YH, Carducci MA, Liu G, Jarrard DF, Hahn NM, et al. Chemohormonal therapy in metastatic hormone-sensitive prostate cancer: long-term survival analysis of the randomized phase III E3805 chaarted trial. J Clin Oncol [Internet]. 2018 Apr 10 [cited 2021 Apr 19];36(11):1080–7. Available from: https://ascopubs.org/doi/10.1200/JCO.2017.75.3657
About Marco Ghiani
Marco Ghiani is Director & Research Principal at Cytel. Marco focuses on the design and execution of retrospective observational studies in Europe. He has led several real-world evidence studies using statistical and econometric techniques, and his experience expands across several therapeutic areas. Marco holds a PhD in Economics from Boston College with a dissertation titled Essays in Applied Health Economics.
