Evidence Synthesis

Meta-analyses and network meta-analyses

Standard methods of meta-analyses require only two treatments for comparison, while network meta-analysis (NMA) applies when three or more treatments are connected through a network and compared simultaneously in a single analysis. Both standard meta-analyses and NMA are mainly based on aggregated data (AgD) from RCTs. These methods assume that the trials are similar enough and the distributions of effect-modifiers (i.e. those factors which can alter the treatment effect on outcomes) do not differ between trials (this is often called a “consistency” assumption). Our team has conducted several meta-analyses and network meta-analyses informing CEMs for HTA submissions and understand the challenges of testing for different assumptions in these analyses. 

Meta-regression uses a regression-based approach on aggregated data (AgD), a mix of AgD and individual patient data (IPD), or only IPD data to correct for potential effect modification in NMA. Meta-regressions are a standardized approach and well documented in NICE TSD3. https://nicedsu.sites.sheffield.ac.uk/tsds/population-adjusted-indirect-comparisons-maic-and-stc.

If meta-regressions are based on IPD-only RCTs, they are considered the golden standard for NMA. However, in practice, one often does not have IPDs for all trials in the evidence network. Meta-regression can also be applied for AgD-only datasets, which are not uncommon for HTA submissions. These AgD meta-regressions typically require strong assumptions, e.g. a covariate effect on the relative treatment effects over trials, which can be applied to all treatments in a network. These assumptions might lead to additional bias. As a result, other methods described below have been developed that perform population adjustment based on the covariate treatment effect observed within trials.

Multi-level network meta-regression (ML-NMR) is a novel population-adjusted, Indirect treatment comparison (ITC) approach that allows aggregate data trials to leverage information from IPD trials. In particular, it allows information on the association between effect modifiers and the relative treatment effect derived from IPD trials to be applied to  Aggregate Data trials. Contrary to MAIC and STC, ML-NMR can easily be applied to networks larger than two trial comparisons. Secondly, it produces endpoint estimates for the index treatment that can reflect any desired target profile in terms of baseline characteristics, e.g. pivotal trials or local real-world data. 

Propensity score weighing is a technique very similar to the MAIC. However, it requires IPD for all trials in the network. The latter is also true for PSM, which basically selects matched patients from the trials in the network to generate a “unbiased comparison”. 

RWE generated from non-randomized studies, including observational studies are increasingly recognized by key decision-making bodies as a valuable source of insights into the real-world performance of novel therapeutic products, particularly when traditional RCTs are impractical, and lack generalizability or to complement information from RCTs and potentially cover the ‘efficacy-effectiveness’ gap. Our team of statisticians and evidence synthesis consultants can understand the unique opportunities and challenges accompanying RWE studies and its considerations around reliably interpreting evidence from these sources, especially for reaching the high-quality standards expected by decision-makers. 

For example, our team can confidently identify the RWE potential in ITCs such as bridging disconnected networks, using evidence from non-randomized studies as priors in NMAs, in three-level Bayesian hierarchical models, and in bias-adjusted analyses. Additionally, RWE can inform the composition of external control arms in support of CER for single-arm trials. Our senior team members have produced internationally recognized guidance on the RWE use in CER which was endorsed by formal key stakeholders. Our team has also worked closely with clinical and methodological experts in several projects to understand the unique diseases and clinical considerations to avoid misleading results from combined analysis of non-randomized and RCTs that may increase the risk of poorly informed ITCs and subsequently of CEMs.