A complex methodological issue which arises in the production of real-world evidence involves the degree to which causal inferences made regarding a given population, can be transported to another population. Transportability Analysis is a set of quantitative methods by which the extension of such effects can be estimated. Those working with regulators and payers can achieve significant benefits from utilizing such methods to optimize data collected from trial participants, when interpreting real world evidence.
The second part of this two-part blog series with Professor Miguel Hernán of Harvard University, takes a deep-dive into the technical elements of the methods including issues surrounding data-pooling, weighting and heterogeneity.
Technical:
How can we conduct transportability analyses when data cannot be pooled between two sources? This would potentially be a common problem in commercial HEOR activities.
The problem is not so much the pooling of data but the availability of individual data in both studies.
What is a preferred approach for transportability analysis: outcome regression or trial participation modeling/weighting?
No method is generally preferred. Also, there is no need to choose a single method. We can conduct both types of analyses and check for the consistency of their estimates. Or, in some cases, we can use doubly-robust estimators that combine g-formula (outcome regression) and weighting by the inverse odds of participation.
What do you see as the pros/cons of Weighting vs G-formula approaches. Where and how would they differ in results?
Inverse weighting estimates usually have wider 95% confidence intervals than g-formula estimates. However, because investigators tend to rely on parametric working models, inverse weighting estimates may be less biased if the investigators know how to approximately specify the participation model. The results of both approaches will differ when either the model for participation or the model of the outcome, or both, are misspecified.
What is the role of double robust estimation in transportability analysis when results differ based on methods chosen?
When doubly robust methods are available, there is no reason not to use them. Doubly robust estimates may be slightly less precise than g-formula estimates but they offer greater protection against bias due to model misspecification. However, there is not much experience using doubly-robust methods in failure time outcome (survival analysis) settings.
What steps should be taken when transportability analysis suggests treatment effects are very different between two datasets?
Very different effects imply that there is great effect heterogeneity in different subsets of individuals. In this situation, in addition to extending the results to the target population, a reasonable course of action would be to identify subsets of individuals with different responses to treatment. Then the treatment can be prioritized towards those expected to have the greatest benefit or not administered to those with no benefit, or harm in case that qualitative effect modification exists.
Read Part 1.
About the Author of Blog:
Radek Wasiak, PhD is Chief Data Officer & Head of Europe at Cytel. As Chief Data Officer, he is in charge of developing and implementing Cytel’s informatics strategy for identifying and using a continuum of data assets to support the Cytel business, from clinical trial to e-health data and deploying these data via Cytel’s product portfolio. As Head of Europe, he provides strategic oversight and executive leadership to European business including input into communication, compensation, hiring, retention, facilities and benefits.
h
