Many thanks to Grammati Sarri and Michael Groff for their comments in developing this blog.
An indirect treatment comparison compares two trials A and B, by first comparing the results of Trial A to Trial C, and then comparing the results of Trial C to Trial B. Unlike head-to-head comparisons where these trials (A and B) would be directly compared with each other, indirect treatment comparisons, or ITCs, utilize at least one other trial to create a network for comparison.
Often, there are fundamental differences between the trials, such as population, or the interaction of the treatment with features of the population (e.g., age, sex, etc.). These are either caused by prognostic variables or by effect modifiers. The methods with which we handle the complexities raised by each differ slightly. Population-Adjusted Indirect Treatment Comparisons (PAICs) use individual patient data from one or more trials to adjust for characteristics of patients and ensure populations across all trials are reasonably comparable.
During a recent Cytel webinar, Dr. Michael Groff, Senior Research Consultant at Cytel, outlined five steps to adjust for effect modifiers when aiming for a PAIC.
1: Examine the totality of the evidence when considering which effect modifiers to include in the PAIC adjustment
The Updated NICE Methods Manual recommends identification of effect modifiers through review of related literature as well as by obtaining expert opinion. Subgroup analysis from previously conducted trials can also help identify the relevant effect modifiers for consideration for the targeted patient population. Issues such as small sample sizes, evidence on effect modifiers from related outcomes, and rarity of disease can all affect the validity of conclusions regarding the identification of effect modifiers.
2: Provide evidence that the ITC results are likely to produce less biased estimates
Not every effect modifier will produce biased estimates in a PAIC analysis, if not appropriately adjusted. Sponsor teams should ensure that effect modifiers are selected based on the consideration that these may produce biased results in the analyses, by applying a range of techniques examined during the webinar (access below).
When effect modifiers are selected, it is also important to demonstrate that there is a statistically significant distribution of the effect modifiers between the studies to justify proceeding with an adjustment. Failure to disclose this information could lead to producing more biased and non-transparent results than an unadjusted ITC.
3: Investigate for violations to the shared effect modifier assumption
The shared effect modification assumption is a highly restrictive assumption that states that the impact created by an effect modifier will be consistent across the range of treatments included in the PAIC. The assumption, if met, is highly beneficial in according an interpretation of the PAIC results within the sponsor’s study population, which is the relevant population of HTA discussions. Cytel statisticians rigorously examine the evidence for this assumption and provide alternative interpretative options when this assumption has been violated.
4: Adjust for all identified and confirmed effect modifiers
The base case PAIC analysis should rely on the strongest evidence of effect modifiers. A range of PAIC methods can be explored for use based on their strengths and weaknesses, including Network Meta-Analysis, Matching Adjusted Treatment Comparisons (MAIC), and Simulated Treatment Comparisons (STC). It is best practice to work with a statistician who is familiar with these techniques and can advise on which is well-suited to the task at hand.
5: Provide scenario analysis where uncertainty exists and describe trends
Where certainty remains, scenario analyses of different effect modifiers should be explored and examined for trends to add confidence to the results of base case analysis. A scenario analysis can offer context for such uncertainty, even if not able to fully model it. For more information, watch our webinar on the effect modifiers in indirect treatment comparisons.
Read more about this webinar series:
About the Author of the Blog:
Dr. Esha Senchaudhuri is a research and communications specialist, committed to helping scholars and scientists translate their research findings to public and private sector executives. At Cytel Esha leads content strategy and content production across the company's five business units. She received a doctorate from the London School of Economics in philosophy, and is a former early-career policy fellow of the American Academy of Arts and Sciences. She has taught medical ethics at the Harvard School of Public Health (TH Chan School), and sits on the Steering Committee of the Society for Women in Philosophy's Eastern Division, which is responsible for awarding the Distinguished Woman in Philosophy Award.
