A complex methodological issue which arises in the production of real-world evidence involves the degree to which causal inferences made regarding a given population, can be transported to another population. Transportability Analysis is a set of quantitative methods by which the extension of such effects can be estimated. Those working with regulators and payers can achieve significant benefits from utilizing such methods to optimize data collected from trial participants, when interpreting real world evidence.
This is a first in a two-part blog series with Professor Miguel Hernán of Harvard University, on Transportability Analysis, explaining its purpose, scope and relationship with more familiar statistical methods and with the idea of causal inference more generally. Part 2 of this series will then take a deep-dive into the technical elements of the methods including issues surrounding data-pooling, weighting and heterogeneity.
Non-technical:
How would you describe transportability analysis to a non-technical person first learning about it and what are some applications that could be relevant to a decision-maker?
Transportability analyses are designed to extend (or transport) the inferences made in a particular population to a target population. For example, suppose that a randomized trial estimated that a new treatment increases overall survival by 20% among patients with cancer. If the patients included in the trial had, on average, a better prognosis than patients in the real world, it would be unclear whether that 20% benefit can be applied to patients in the real world. A transportability analysis would help decision makers by using the results from the trial to quantify the average benefit in the target (real world) population.
People unfamiliar with the jargon of causal inference may assume that transportability analysis is a way to show that the treatment effects that they report are "generalizable". In your view, what is the important difference between generalizability and transportability for decision makers to understand?
Generalizability and transportability are different goals. The distinction depends on the relation between the study population and the target population of interest. To explain this, let’s return to our example in which we want to extend the inferences from a randomized trial to a larger target population. Suppose the target population is the population of patients who were eligible to participate in the trial. Some of them were not invited to participate or were invited but declined to participate. Then we refer to this as a generalizability problem because the target population coincides, or is a subset of, the trial-eligible population. Now suppose that the target population of interest is a more heterogeneous population of patients with cancer, some of whom did not meet the eligibility criteria of the trial. Then we refer to this as a transportability problem because the target population is not a subset of the trial-eligible population. Our preference is to avoid the distinction between generalizability and transportability by using the expression “extending inferences from trial participants to a target population”, but the use of the term transportability analysis has become widespread, even when the goal of the analysis can be viewed as generalizability.
Is transportability analysis restricted to only real-world data or can effects be transported between trials or from trial to RWD?
In principle, effects can be extended from any study population to any target population. Often the interest is in extending the effects from a randomized trial to real world population, but in some settings we may be interested in extending inferences between non-trial populations as well. A key condition to support the validity of the transportability analysis is that the study population and the target population live in settings that are sufficiently similar.
What are some assumptions that need to be made when trying to transport effects, and how plausible are these assumptions in practical applications in your experience?
We need two types of assumptions. Let me summarize them informally.
First, we need to assume that the causal effect can be estimated without bias in the study population from which we will extend the inferences. This is a reasonable assumption for randomized trials, but it needs to be justified if the data from the study population is observational, that is, real world data.
Second, we need to assume that individuals with the same characteristics in the study population and in the target population are comparable within levels of the relevant covariates, and that individuals with the same covariate values as those in the target population would have had a non-zero probability of being included in the study population. These are typically strong assumptions that cannot be empirically verified.
Presumably, the statistical methods we're discussing will always produce an answer but what kind of information (e.g., domain expertise) is needed to know whether such a transported effect is likely to be valid?
We cannot prove that the assumptions required to extend inferences hold true, but we can use domain expertise to evaluate their plausibility. Importantly, expert knowledge is necessary to propose the relevant covariates (effect modifiers) mentioned in the assumptions above.
Are transporting treatment effects possible when only summary-level data on patient covariates is available in the target population, that is, without individual patient data in the target population?
The short answer is yes: individual data on treatment, outcome, and covariates are needed. An alternative is having summary-level data for every combination of relevant covariates. However, that seems an unlikely scenario in practice.
About the Author of Blog:
Radek Wasiak, PhD is Chief Data Officer & Head of Europe at Cytel. As Chief Data Officer, he is in charge of developing and implementing Cytel’s informatics strategy for identifying and using a continuum of data assets to support the Cytel business, from clinical trial to e-health data and deploying these data via Cytel’s product portfolio. As Head of Europe, he provides strategic oversight and executive leadership to European business including input into communication, compensation, hiring, retention, facilities and benefits.
h
