Quantitative Bias Analysis – A Breakthrough for Transparency of RWE Applications in HTA
Quantitative bias analysis (QBA) is a potential and likely breakthrough in real-world evidence. This was the conclusion of the industry-sponsored Q-BASEL project presented during an Educational Symposium at the ISPOR Annual meeting in November 2022.
Panelists from Cytel, Roche, NICE, and Harvard School of Public Health discussed the role of QBA in addressing issues of internal validity in synthetic control arm (SCA) studies and presented positive findings from the application of QBA and external bias adjustment in 15 SCA studies in advanced non-small cell lung cancer (aNSCLC), the largest empirical evaluation of QBA applications to date. The overall findings: quantitative bias analysis methods can successfully identify and address concerns around internal validity and suspected unaddressed bias in real-world evidence (RWE) studies of comparative effects. The application of QBA in these settings was systematic, replicable, and transparent. Stakeholders in health technology assessment could benefit greatly from QBA as they will face more uses of RWE during the assessments of the value of medicines.
Kristian Thorlund of Cytel compared QBA today to network meta-analyses a decade ago – an emerging and newly endorsed methodology by the National Institute for Health & Care Excellence (NICE) but without widespread familiarity and acceptance across the research industry.
QBA findings proved to be robust based on Q-BASEL findings. In 14 out of 15 evaluated comparisons using trial data and RWD from Flatiron Health, QBA produced valid effect estimates after incorporating prespecified external information on bias. Non-randomized bias-adjusted estimates from the 15 evaluated SCAs were found to be close to randomized estimates from aNSCLC RCTs. External information on key sources of bias was systematically identified and prespecified before incorporating in SCA analyses to compute bias-corrected effect estimates. Researchers generated a template to support the use of QBA in future applications, a huge step forward for the application of RWD in HTA.
From the perspective of NICE represented in the panel, there was emphasis that conversations and expectations between developers and HTA organizations regarding key sources of bias and acceptable levels of uncertainty can be supported using a QBA process and facilitated by early engagement with HTA organizations.
For UK’s NICE and other HTA agencies, the question of robustness of RWE is at the forefront of the decision-making. The much-anticipated RWE framework published by NICE in June 2022 covered best practices for a transparent evaluation of potential sources of uncertainty, particularly in the context of single-arm trials and ECA studies, and endorsed the use of QBA methods bringing increased attention across the research community.
Panelist Miguel Hernan of the Harvard School of Public Health described how QBA, combined with an explicit emulation of the target trial of interest, improves causal inference from observational data by clearly identifying the elements of the emulation to which the effect estimates are particularly sensitive; the result is a better characterization of the uncertainty of the inferences and a more informative interpretation of the results for decision makers.
The positive results and lessons learnt from the Q-BASEL study bring a much-needed promise for the application of synthetic controls utilizing RWD considering the oncoming avalanche of single-arm trials in oncology and rare diseases that HTA agencies will face over the next decade.
Reflecting on the Q-BASEL project and the panel, Radek Wasiak, the head of Real World and Advanced Analytics at Cytel highlighted that an invaluable contribution of this extensive investigation was showing that if the right approach is taken, RWE can be at the core of the health technology assessment. The conclusion is clear: bias adjustment in real world data is feasible when quality external data are available.
Results from the Q-BASEL study will be published in early 2023.
Interested in learning more? Contact us today.
Cytel is the largest provider of statistical software and advanced analytics for clinical trial design and execution. For over thirty-five years, Cytel’s scientific rigor and operational excellence have enabled biotech and pharmaceutical companies to navigate uncertainty, prove value and make confident, evidence-based decisions. Its experts deliver industry-leading software, data-driven analytics, real-world evidence and strategic consulting. Headquartered in Waltham, Massachusetts, Cytel has more than 2,000 employees across North America, Europe and Asia. For more information about Cytel, please visit us at www.cytel.com