While randomized control trials remain the industry gold-standard for regulatory and reimbursement submissions, there are occasions when such trials are either infeasible or unethical. In such situations, regulatory approvals for products can be achieved based on single-arm trials where an external control creates a comparator arm, most commonly by using real-world data (RWD) such as, electronic health records or patient registries. However, this poses several challenges around the selection of an appropriate, fit-for-purpose real-world data source to construct the external control arm, in particular the trade-offs between data quality and locality.
At ISPOR US 2022, Cytel’s Lead Scientist Dr. Grammati Sarri will moderate an issue panel, “Is the Global Real World Data Supply Chain Broken?” Grammati has over 15 years of experience, specializing in evidence synthesis, epidemiology, and the use of real-world data in reimbursement decision-making. As a senior leader, she has held positions in academia, payer, and private consultancies, and has published over 60 peer-reviewed manuscripts and delivered several webinars and issue panels. As an internationally recognized thought leader, Grammati is currently chair-elect for the Comparative Effectiveness Research Special Interest Group at the International Society for Pharmacoepidemiology.
In this interview, we ask Grammati about the agenda of this panel discussion and get her insights on the challenges affecting single arm studies, the importance of real-world data and the selection of its source.
Can you tell us a bit about the premise of your issue panel. Why is there a choice between quality and locality in using RWD?
The growing number of highly specialized products (such as cell-based and gene therapies) and the focus on personalized medicine have transformed the way health technologies are developed in clinical programs and assessed by healthcare decision makers. The strong public need for these innovative technologies to enter the market early and access patients most in need has shifted the focus from traditional randomized clinical trials to single-arm trials due to ethical considerations or difficulty of recruiting patients. Although, these products often achieve regulatory approval based on single-arm trials, for a technology to demonstrate its comparative effectiveness value against standard of care as part of the health technology assessment (HTA), external control arm data are needed. Different sources can be used to construct the external control arm, but RWD is gaining increasing attention as it most closely represents clinical practice. During this selection process, HTA decision makers have shown a preference for local or national data reflecting clinical practice. However, this preference is not straightforward as it is accompanied with methodological challenges depending on the local availability of data and its quality, and presents operational difficulties when the industry is planning for multiple submissions across different countries.
Can you talk about the special challenges affecting single arm studies and how choices about RWD can lead to difference in their analyses?
Few HTA submissions based on single-arm trials have been successful in convincingly establishing comparative effectiveness for new technologies; the main reasons were data quality (biases), patient numbers and lack of comparability of selected control arm with clinical practice. Challenges related to data quality and source of external control groups have been widely communicated as factors leading to unsuccessful HTA submissions of single-arm trials. However, it is still unclear how different stakeholders (manufacturers, researchers, decision makers) address the dilemma of choosing between locality and quality of data that supports the external control arm in comparative clinical assessments of single-arm technology applications. Will HTA bodies trade-off RWD quality and subsequently compromise data analysis, in an attempt to reflect local populations?
Cytel is always saying that more RWD can be brought in for regulatory submissions to strengthen RCTs. Would any of the challenges you mention be important for a traditional two-arm approach?
Definitely. Issues around data sparsity, suitability to match the targeted population for decision-making, comparability of populations in a two-arm clinical trial and data quality (in terms of data missingness and completeness, and patient adherence to tested treatments) can all affect decision-makers’ confidence in the findings from the traditional two-arm trial approach. However, randomization in a two-arm trial aims to remove the selection bias which commonly occurs when single-arm trial data are compared with external control RWD.
Tell us a bit about the panelists and their contributions to this area of research.
Each of the panelists will bring a unique but highly relevant background as key stakeholders for the topic under debate: Dr Hernandez will bring the industry perspective and will outline the challenges in developing a local external control arm that meets the requirements of several health authorities while corresponding to the product’s targeted population. Professor Thorlund will discuss issues around bias versus precision in RWD and present on how to methodologically build a strong evidence base for an external control arm, if data is ‘similar enough’ to the local setting. Finally, Dr. Kent will discuss how HTA bodies (particularly, NICE in the UK) may assess the validity of non-local RWD to build the external arm for single-arm submissions, and share potential solutions for these situations by outlining recent guidance on this topic (NICE RWE draft Framework).
Cytel has spoken with Dr. Hernandez before about head-to-head trials. Ideas of causality seemed to be an important component of his research. How, if at all, does that play a role in the issues you will be discussing?
Causality is key to establishing the comparative effectiveness of new health technologies using RWD. However, ensuring that the methodological challenges related to RWD study design and analysis are overcome, is far from simple. It mainly depends on the understanding of the disease and the availability of information to apply more complex analytical approaches and ensuring causality is established. However, data availability may be sparse and span across different geographical settings, consequently doing little to directly reflect the clinical practice in each market.
What are some of the things you look forward to discussing with them?
It will be interesting to hear different perspectives on this topic and understand the contribution of key factors weighing in favor of one approach over another; how the manufacturers can ensure a successful market strategy while satisfying evidentiary requirements of different HTA bodies, in the absence of firm guidelines on what is considered convincing comparative evidence for single-arm trials? Can a single source, external control arm meet the requirements of several HTA bodies? Is it possible to establish threshold criteria of a “good” external control arm by balancing RWD quality and locality? How can HTA bodies ensure consistency and transparency in decision-making when the methodological challenges depend on clinical contexts, given the heterogeneity of populations and the rarity of diseases commonly seen in single-arm cases? These are some questions that we hope our panelists can address along with our audience.
Click below to download Cytel's full list of sessions at ISPOR US 2022.
About the Author of Blog:
Mansha Sachdev specializes in content creation and knowledge management. She holds an MBA degree and has over 12 years of experience in handling various facets of marketing, across industries. At Cytel, Mansha is a Senior Content Marketing Manager and is responsible for producing informative content that is related to the pharmaceutical and medical devices industries.
