Skip to content

Evidence Synthesis

leftside leftside Hero image_580x302px
Cytel’s HEOR services are committed to the high-evidentiary standards established by HTA bodies like NICE and closely following guidelines like PRISMA and those from Cochrane Collaboration. Our evidence synthesis team is multidisciplinary and consists of researchers with training across the spheres of health and medical sciences. Collectively, we have decades of experience conducting and presenting evidence reviews for a wide range of research and commercial purposes. This includes producing landscape reviews to define the value demonstration of new products to support regulatory and reimbursement submissions to health authorities. A number of our senior members have previously worked in HTA bodies (NICE) and have had a strong methodological presence in international scientific communities.

Systematic Literature Reviews (SLRs)

SLRs follow a structured process that aims to identify, collate and synthesize evidence from different sources such as randomized clinical trials, real-world evidence [RWE], and qualitative studies on different outcomes (clinical, economic, patient-reported outcomes) in a systematic, reliable, and unbiased way. SLRs are driven by well-defined research questions and review protocols. Detailed evidence identification processes and quality assessments are considered an essential activity for HTA submissions. 

Summaries of epidemiological data, treatment patterns, and guideline recommendations should also follow a systematic approach for identification and selection if they are to successfully support products’ submissions to health authorities. We have a large group of trained consultants on the principles of SLR, but also senior members with extensive prior experience in key academic and industry organizations (Cochrane Collaboration). We also have HEOR experts previously employed in HTA decision-making bodies (NICE) who understand how to ensure that results from an SLR can meet the high evidentiary requirements from health care decision-makers, especially in new methodological areas such as Real World Evidence (RWE) reviews. When methodologically appropriate and feasible (i.e. when there is sufficient homogeneity of studies), a quantitative (meta) analysis and a statistical summary of effect estimates across the identified studies is performed. 

line-bg
SLR image_485x445px
Targeted Literature Reviews (TLRs) Read more
Early or Landscape Evidence Reviews Read more

Comparative Effectiveness Research (CER)

Comparative Effectiveness Research (CER) at Cytel extends beyond the basics of traditional CER to employ innovative, Cytel curated methods for proof of value. Cytel’s global evidence synthesis team consists of systematic literature reviewers, biostatisticians, modelers and multidisciplinary HEOR experts. Our comparative effectiveness work has been incorporated in several Cost Effectiveness Models (CEMs), supporting successful HTA submissions globally. We apply a wide range of statistical techniques to estimate comparative treatment effects between our clients’ technology and comparators currently used in clinical care.  These include a range of techniques, from standard meta-analyses to complex, Cytel curated methods like population adjusted indirect treatment comparisons. Our in-depth feasibility assessment of the availability and comparability includes quality assessment of the evidence retrieved from different sources (RCTs, single-arm trials, non-randomized studies), which then guides the selection of the most methodologically appropriate CER analysis. 

 

Comparative Effectiveness_580x474px
leftside leftside Anchored methods_580x302px

Anchored methods

Meta-analyses and network meta-analyses

Standard methods of meta-analyses require only two treatments for comparison, while network meta-analysis (NMA) applies when three or more treatments are connected through a network and compared simultaneously in a single analysis. Both standard meta-analyses and NMA are mainly based on aggregated data (AgD) from RCTs. These methods assume that the trials are similar enough and the distributions of effect-modifiers (i.e. those factors which can alter the treatment effect on outcomes) do not differ between trials (this is often called a “consistency” assumption). Our team has conducted several meta-analyses and network meta-analyses informing CEMs for HTA submissions and understand the challenges of testing for different assumptions in these analyses. 

Population-adjusted indirect treatment comparisons

Population-adjusted indirect treatment comparisons (ITC) are required when differences in population characteristics over trials in the evidence network may cause bias due to effect modification. Effect modification occurs when an exposure has a different effect among different subgroups. Several population adjustment ITC methodologies are available and are shortly described below. Cytel’s team of statisticians, health economists, and clinicians have extensive experience with these methods and are constantly trying to improve them to reduce bias in the HTA submissions. For instance,  we have recently introduced r a novel population adjustment ITC approach called network meta-interpolations. https://www.valueinhealthjournal.com/article/S1098-3015%2821%2902814-X/fulltext

leftside leftside Evidence Synthesis(2)_580x302px
leftside leftside STC_580x302px

Meta-regressions

Meta-regression uses a regression-based approach on aggregated data (AgD), a mix of AgD and individual patient data (IPD), or only IPD data to correct for potential effect modification in NMA. Meta-regressions are a standardized approach and well documented in NICE TSD3. https://nicedsu.sites.sheffield.ac.uk/tsds/population-adjusted-indirect-comparisons-maic-and-stc.

If meta-regressions are based on IPD-only RCTs, they are considered the golden standard for NMA. However, in practice, one often does not have IPDs for all trials in the evidence network. Meta-regression can also be applied for AgD-only datasets, which are not uncommon for HTA submissions. These AgD meta-regressions typically require strong assumptions, e.g. a covariate effect on the relative treatment effects over trials, which can be applied to all treatments in a network. These assumptions might lead to additional bias. As a result, other methods described below have been developed that perform population adjustment based on the covariate treatment effect observed within trials.

Matching Adjusted Indirect Comparisons (MAIC) and Simulated Treatment Comparisons (STCs)

Matching Adjusted Indirect Comparisons (MAIC) and Simulated Treatment Comparisons (STCs) are the most commonly used population-adjusted indirect treatment comparisons when there is a connected network between typically two RCTs comparing different treatments with a common comparator. Both require the availability of individual patient data (IPD) from at least one trial in the network. 

These methods allow for the “consistency” assumption in the network to be relaxed. This means that the availability of IPD permits the adjustment of effect modifiers across the different trials. This can occur through either population reweighting methods [in the case of MAIC] or outcome regression methods [in the case of STC], therefore allowing for valid comparative treatment effect estimates. These approaches produce relative endpoint estimates for the comparator treatment population for which Aggregate Data (AgD) is available, which is a disadvantage. Ideally one would have these for the pivotal trial for which typically IPD is available. 

leftside leftside MAIC_580x302px
leftside leftside Evidence Synthesis(3)_580x302px

ML-NMR

Multi-level network meta-regression (ML-NMR) is a novel population-adjusted, Indirect treatment comparison (ITC) approach that allows aggregate data trials to leverage information from IPD trials. In particular, it allows information on the association between effect modifiers and the relative treatment effect derived from IPD trials to be applied to  Aggregate Data trials. Contrary to MAIC and STC, ML-NMR can easily be applied to networks larger than two trial comparisons. Secondly, it produces endpoint estimates for the index treatment that can reflect any desired target profile in terms of baseline characteristics, e.g. pivotal trials or local real-world data. 

NMI

Network meta-interpolation (NMI) is an approach that uses subgroup data from all trials in the evidence network to inform the indirect treatment comparison. It embeds information on effect modification as reported in subgroup data from all trials in the network, irrespective of whether it's aggregated data or IPD. Similar to ML-NMR, this approach produces endpoint estimates for the index treatment that reflect any desired target profile in terms of baseline characteristics such as a pivotal trial or local real-world data. The advantage of NMI is that it is the only method that uses subgroup data from Aggregate Data (AgD) trials to inform the population-adjusted indirect comparison. Therefore, it does not rely on the shared effect modification assumption. 

leftside leftside Evidence Synthesis(5)_580x474px
leftside leftside Evidence Synthesis(4)_580x302px

Propensity score matching and weighing

Propensity score weighing is a technique very similar to the MAIC. However, it requires IPD for all trials in the network. The latter is also true for PSM, which basically selects matched patients from the trials in the network to generate a “unbiased comparison”. 

Unanchored Methods

Unanchored indirect treatment comparisons are performed when there is a disconnected network between RCTs, or due to the presence of single-arm trials. Depending on the IPD data availability several methods can be applied, naïve comparisons, STC, MAIC, propensity score matching, or weighing.  Unanchored comparisons require much stronger assumptions than anchored indirect comparisons on the consistency of treatment effects across the trials. Our team follows closely the recent methodological advances for this type of statistical analyses and consulting our clients on the best statistical approach to ensure adherence to published guidelines while also unraveling the potential of technology under assessment. 

leftside leftside Unanchored Methods_580x474px
leftside leftside Evidence Synthesis(5)_580x302px

Use of RWE in CER

RWE generated from non-randomized studies, including observational studies are increasingly recognized by key decision-making bodies as a valuable source of insights into the real-world performance of novel therapeutic products, particularly when traditional RCTs are impractical, and lack generalizability or to complement information from RCTs and potentially cover the ‘efficacy-effectiveness’ gap. Our team of statisticians and evidence synthesis consultants can understand the unique opportunities and challenges accompanying RWE studies and its considerations around reliably interpreting evidence from these sources, especially for reaching the high-quality standards expected by decision-makers. 

For example, our team can confidently identify the RWE potential in ITCs such as bridging disconnected networks, using evidence from non-randomized studies as priors in NMAs, in three-level Bayesian hierarchical models, and in bias-adjusted analyses. Additionally, RWE can inform the composition of external control arms in support of CER for single-arm trials. Our senior team members have produced internationally recognized guidance on the RWE use in CER which was endorsed by formal key stakeholders. Our team has also worked closely with clinical and methodological experts in several projects to understand the unique diseases and clinical considerations to avoid misleading results from combined analysis of non-randomized and RCTs that may increase the risk of poorly informed ITCs and subsequently of CEMs. 

Our Software

Cytel LiveSLR

Cytel’s LiveSLR provides a solution to the most important problem that HEOR professionals face: outdated Systematic Literature Reviews (SLRs). LiveSLR is powered by a unique human-machine partnership that combines the speed of an AI literature scan with the judgment of an industry-leading evidence generation team. Our SLR’s provide adaptable reporting and local adaptations to support reimbursement submissions to HTA bodies.

Cytel LiveNMA

Designed to work with LiveSLR, LiveNMA provides a comparative effectiveness network meta-analysis (NMA) roadmap based on interventional studies identified by LiveSLR for a selected patient population. LiveNMA allows clients to modify the targeted relative efficacy of their treatment and investigate how this will impact the relative efficacy of their products when compared with all other treatment options per treatment line and endpoint. LiveNMA can also calculate how much a product needs to improve efficacy in order to showcase its value relative to competitors and position its superiority against other treatments.

PubTracker

Also designed to work with LiveSLR, PubTracker fills in the gaps in your SLR database and provides you with pre-congress preparation and competitive intelligence. PubTracker monitors abstracts published online in the weeks leading up to a congress. These congress abstracts, often presenting preliminary results in advance of full-text articles, represent cutting-edge research that may be inadvertently excluded from systematic literature reviews.

Meet the Experts

ANNA FORSYTHE (1)

Anna Forsythe

VP Value & Access
Grammati photo

Grammati Sarri

Senior Research Principal, Head of RWAA External Research Partnerships
Maria Rizzo

Maria Rizzo

VP HEOR Europe
Screen Shot 2022-05-12 at 5.02.12 PM

Sharada Harricharan

Director HEOR

Our Publications

2022
2021
2020
2019
2018
2016
2015