Health economics and adaptive design methods share common ground in that they both aim to support more efficient and accurate decision making that can enable faster patient access to new health technologies. However, to date, there has been a limited understanding of how, if at all, the two approaches are being used together.
A paper, “A Review of Clinical Trials With an Adaptive Design and Health Economic Analysis,” exploring this important topic was published in the April 2019 issue of Value in Health (1) . In this blog, we catch up with Laura Flight, National Institute for Health Research (NIHR) Doctoral Fellow and the primary author of the paper for a deep dive into the objectives of the publication, key findings and the next steps for promoting better understanding in this area.
What were the key objectives of the publication?
My NIHR doctoral fellowship is looking at how we can use adaptive clinical trials and health economics together with the overall aim of trying to improve the efficiency of our health technology assessments, while also ensuring that the funding decisions that we make are accurate.
In our paper, we wanted to dig deeper and understand whether the methods of adaptive designs and health economics were being used together in practice. Both methodologies have the common objective of increasing efficiency but aren’t necessarily being considered together in practice.
We plan to use the findings to inform how to develop the methods further, by understanding the successes of case studies and thinking about why health economics hasn’t been used in adaptive designs in practice.
The paper builds on a previous University of Sheffield review by Hatfield et al. (2) that aimed to understand how many trials were adaptive and what types of adaptations were commonly done, in different therapeutic areas. The authors of this paper used the same search strategy as used in the Hatfield review to find trials that had adaptations and a health economic analysis component from sources including clinical trials.gov; known adaptive trials and the Health Technology Assessment (HTA) journal to identify. Once the trials were identified, the authors extracted information about the characteristics of the study and the role of health economics in the design, analysis, and reporting.
What were some of the key findings?
Overall, we identified 37 trials with an adaptive design and health economic analysis. We found three adaptive designs that discussed health economics at the design stage. None of the trials, where thought necessary, adjusted their analysis to account for the adaptive nature of the trial and overall reporting was suboptimal for the 17 trials with published results.
We found three case studies that were thinking about health economics to some extent in adaptive clinical trials. These trials are at the forefront of work in this area and are early examples of what is possible. There is, however, still work to do and the methods can be developed even more, but it was great to see that people are thinking about this already. We can use these findings and case studies to take the methods development and the broader project forwards.
Do you have any insights on why health economic aspects are not being considered in adaptive trials?
Following on from the review, I have used a qualitative study to understand whether there are practical and ethical issues that prevent people using these methods, or explore whether the main barrier is that there is not much methodological work in this area. The findings of this study, that included members of the public, researchers and decision makers, will be published over the next few months. These perspectives have been really valuable in guiding the direction of methods research.
How can adaptive designs particularly impact health economic analyses, and what do trial sponsors need to consider from a statistical or operational point of view?
We need to be aware of the potential for the adaptive trial to influence the health economic analysis and also think about opportunities to use health economics to design an adaptive trial and inform interim decision making. This can be achieved by ensuring the trial statistician and health economist are involved at the start of the project and have sufficient training in adaptive designs.
As a specific example, we've been focusing on the group sequential design, one of the most common and longest running types of adaptive trial. When we analyse a group sequential design, there is a potential for bias in the point estimate and confidence interval of the primary and secondary outcomes. This potentially has implications for our health economic analysis.
Imagine that we build a health economic model to establish the long-term cost-effectiveness of an intervention. We might use information about the primary and secondary outcomes from the trial to inform parameters in the health economic model. If we're using a point estimate from a group sequential design that hasn’t been appropriately adjusted, we might be using a biased point estimate or confidence interval in the model. This could in turn introduce bias into our health economic analysis.
Overstating or understating the cost-effectiveness of a particular intervention is concerning given that decision-makers, such as the National Institute for Health and Care Excellence (NICE), rely heavily on this information to make their decisions about which treatment should be made available to patients. Health economists and decision makers aren't necessarily aware that if they're using data from an adaptive trial, there may need to consider whether to adjust their results.
One of your recommendations was that health economic analysis should be outlined in a health economic and decision modelling analysis plan before the start of an adaptive design. What steps do you think need to take place for this to happen routinely in reality?
We found in our review that pre-specification of the health economic analyses was limited. It’s crucial if we're thinking about planning an adaptive design, where we’re going to use health economics to help inform some of the decisions we make along the way, that we pre-specify this, just as you would pre-specify your clinical stopping rules.
Dr Joanna Thorn and her team at the University of Bristol have been developing a framework for health economic analysis plans in clinical trials. Their focus is on all clinical trials, not specifically adaptive trials, but is a good starting point to get researchers developing a health economic analysis plan and we can then start to think about adding on this adaptive design layer.
If we can bridge this gap and get this interaction of health economics and adaptive trials working better, what benefits do you think the industry and patients, payers and different stakeholders, could derive from that?
Our ultimate goal is to try and make the most of the limited resources. I think everyone stands to benefit from integrating these two methodologies. Adaptive designs are becoming more popular because people are recognising that they're a more efficient way to run trials. However, we want to make sure that we're not compromising our health economic analysis by using adaptive designs and in turn preventing patients accessing the most cost-effective treatments. We also want to make sure we’re not missing opportunities to make our trials more efficient by neglecting cost-effectiveness when designing and analysing our adaptive designs. Patients stand to benefit if we can do things more efficiently, we can do more research, we can hopefully develop more interventions and get more interventions funded.
Conclusion
We thank Laura for sharing the team’s findings with us and look forward to seeing further research from the group that can advance the alignment of the two critical disciplines of adaptive designs and health economics.
Laura Flight is funded by a National Institute for Health Research (NIHR) Doctoral Research Fellowship for this research project. This report presents independent research. The views expressed are those of the author(s) and not necessarily those of the University of Sheffield, the NHS, the NIHR or the Department of Health and Social Care.
To read the paper, click the button below:
References
1) A Review of Clinical Trials With an Adaptive Design and Health Economic Analysis
Flight, Laura et al. Value in Health, Volume 22, Issue 4, 391 - 398
2) Hatfield, I., Allison, A., Flight, L., Julious, S. and Dimairo, M. (2016). Adaptive designs undertaken in clinical research: a review of registered clinical trials. Trials, 17(1).
3) Thorn JC, Ridyard C, Hughes D, Wordsworth S, Mihaylova B, Noble SM and Hollingworth W (2017) ‘Health economics analysis plans: the current state of play’ Trials, 18(Suppl 1):P144
About Laura Flight
Laura is a Research Associate in Health Economics and Statistics, at the School of Health and Related Research, University of Sheffield. Laura is interested in developing statistical methods for efficient health research by incorporating health economics into the design and analysis of trials. Her recent NIHR doctoral fellowship investigates how adaptive clinical trials and health economics can be used together in health technology assessments.
