Adaptive Designs in Practice: Interview with NIHR Research Fellow Munya Dimairo
NIHR and University of Sheffield researchers recently published a paper, ‘Adaptive designs undertaken in clinical research: a review of registered clinical trials’(Hatfield et al) which explores the current state of adaptive designs in practice. In this blog we catch up with Munya Dimairo, NIHR Research fellow and one of the paper’s co-authors to find out more.
The review involved searching the ClinicalTrials.gov database and the National Institute for Health Research database for Adaptive design trials, with the objective of gaining insights around a number of key area including the number of trials are being designed and conducted as Adaptive Designs, the type of adaptive designs being implemented, and the trends we can see in the use of adaptive designs by therapy area and phase. Importantly, the reviewers also wanted to assess how adequate Clinicaltrials.gov is currently at capturing Adaptive design trials and to highlight case studies which could be used by researchers as a learning resource.
The research found that there were a number of significant limitations in the way Clinicaltrials.gov records information around adaptive designs. However, despite these limitations the review was able to highlight a number of key findings regarding the use of adaptive designs.
Cytel: What was the catalyst for developing this piece of work? Can you tell us more about how it developed?
Munya Dimairo (MD) I’ve been funded by NIHR (DRF-2012-05-182) to work on adaptive designs and try to understand the barriers, and come up with solutions if possible in order to improve the use of adaptive designs. I was mainly working on Phase 3 trials but of course adaptive designs can be extended to any phase.
"Adaptive designs are not new in terms of their theory but there is lack of transitional, practical knowledge and people want to understand a little bit more"
So I did some initial work which was quite interesting. I did a presentation at the PSI conference last year, and one of the main barriers that came up was a lack of practical knowledge among researchers and the research community. And there was a feeling that they needed more case studies on adaptive designs. It’s kind of a new thing. Of course, adaptive designs are not new in terms of their theory but there is lack of transitional, practical knowledge and people want to understand a little bit more. So one way to go about it is to say ‘Have we got case studies we can learn from?’ ‘Where can we find these case studies? And if we’ve got these case studies are they being reported well for people to understand key aspects of the design, and to replicate if possible. Because some of the designs are really complicated and it means the bar is high in the way you report these designs for someone to understand exactly what’s going on. So that was the foundation.
"If we want people to know more about adaptive designs, then we need to do more in terms of reporting."
It’s difficult to go through the normal literature, to go through journals and search for adaptive designs. And we’re trying to avoid publication bias and reduce the time-lag to access case studies in the literature. So we decided to search for trials through clinical trial registers instead of going through the normal journals and then went back to the journals to try to link those trials. It’s quite interesting because the reporting of adaptive designs is really worrying. It’s not really good enough. It’s something we need to work on as a research community. If we want people to know more about adaptive designs, then we need to do more in terms of reporting.
Cytel: Was there anything particularly unexpected in your findings?
MD What was really interesting is that we were expecting to see a lot of simple adaptive designs such as sample size re-estimation and futility analysis. But because of poor reporting of those kind of adaptive designs we couldn’t find many as we expected. So it goes back to the issue of indexing of adaptive designs– how do we index these adaptive designs when we are reporting them?
There is also an increase in Phase 2/3 seamless designs but another issue is that it’s difficult to differentiate between inferentially seamless and operationally seamless designs. In operationally seamless designs you are not combining the data from Phase 2 and 3 in the analysis- you analyse the data separately as independent trials. But for inferentially seamless designs you are saying, for example, if you start with 6 doses and drop maybe 2, then you need to keep the patients that you enrolled in the first part of the study and combine that dataset for the analysis of the future phase. So the methodology is different. For inferentially seamless designs you need more complex statistical methodology to do it. We need to report well enough to say which seamless design we are using- is it inferentially seamless or operationally seamless?
Cytel :How do you think what the paper describes as the ‘vicious circle ‘of underutilization’ can be overcome?
MD We’ve been trying to have this kind of discussion here in the UK. There’s an appetite for adaptive designs. Clinical investigators want to use adaptive designs more often, especially the public funders. I’m talking from my experience on the public side, but it could be applied even in the private sector. They’ve realised the shortcomings of the current traditional approach to the conduct of clinical trials.
"It’s not just about the positive lessons, the negative lessons are really key as well."
So they’re saying ‘Can we improve the efficiency of trial designs? What is the value of money in the research, can we gain something?' There’s an appetite for it but there are a lot of challenges we need to go through. The first issue is mainly about the practical knowledge. People are more comfortable with things they have done before. So if we can say OK, someone has done it and here are the results- whether positive or negative it doesn’t really matter. What matters is the practical experience of doing it. What have they learned? What are the positive and negative lessons? It’s not just about the positive lessons, the negative lessons are really key as well. But we are seeing an improvement in the application of these designs. We’ve got some case studies here in the UK in the public and private sector and I think in the coming 5 years they’ll start to come through.
Cytel: Ideally what would you like to see happen next?
MD The tricky bit is that sometimes we’ve got this disjointed way of operating between private and public sector. First of all we need to have a cross sector discussion, rather than just working from a public or private sector perspective. There’s a need to work on reporting guidance of adaptive design. We need to reach an agreement and look at every type of adaptive design- sample size re-estimation, group sequential etc. and consider what are the key issues that researchers, regulators, other stakeholders need to know about that design. Then we could have a platform where we can share resources- by which I mean implementation resources. If we can have that kind of platform, and we’ve got the reporting guidance, and can look at how we index these designs so people can find the case studies- I think that would be really useful.
We thank Munya for discussing these findings and the goals for the future. The principle of collaboration is an important one, which we at Cytel certainly agree with as we work to further advance the application of adaptive designs in practice.
Further reading
Adaptive designs undertaken in clinical research: a review of registered clinical trials Isabella Hatfield1,2, Annabel Allison2,3, Laura Flight2, Steven A. Julious2 and Munyaradzi Dimairo2*
Missing steps in a staircase: a qualitative study of the perspectives of key stakeholders on the use of adaptive designs in confirmatory trials Munyaradzi Dimairo1*, Jonathan Boote1,2, Steven A. Julious1, Jonathan P. Nicholl1 and Susan Todd3
Cross-sector surveys assessing perceptions of key stakeholders towards barriers, concerns and facilitators to the appropriate use of adaptive designs in confirmatory trials Munyaradzi Dimairo1*, Steven A. Julious1, Susan Todd 2, Jonathan P. Nicholl 1 and Jonathan Boote1,3
Abigail Stevely1, Munyaradzi Dimairo2*,Susan Todd 3,StevenA.Julious 2, Jonathan Nicholl 2,Daniel Hind 2,Cindy L. Cooper 2
Cytel Blog: Inferentially and Operationally Seamless Designs
