In this blog we turn to some reading matter, and interview Gautier Paux and Alex Dmitrienko about the recent book 'Clinical Trial Optimization with R'. The book explores a unified and broadly applicable framework for optimizing decision making and strategy selection in clinical development, through a series of examples and case studies. To learn more, read on for Paux and Dmitrienko's insights.
Who is the audience of the book and why is the topic important to this audience? This book is intended for biostatisticians and other scientists who are involved in the design and analysis of clinical trial designs or development programs. Development of new treatments is becoming more and more challenging with increasing costs, trial duration and failure rates. We know that the use of innovative trial designs and analysis methods has considerably increased over the past decades to help improve the probability of success in clinical trials. However, clinical drug development remains a risky endeavor with numerous complex decisions related to the choice of doses, patient populations and many other factors. Due to the competitive landscape, there is increasing pressure to make these decisions as quickly and efficiently as possible. It is critical to carefully evaluate the decision-making processes throughout the development cycle and identify optimal solutions to decision-making problems. In today’s world, clinical trial optimization should play a central role at the trial level as well as at the development plan level. To give examples of how optimal approaches to decision making could be applied to confirmatory Phase III trials, we first need to recognize that most modern trials are designed to pursue multiple objectives. These objectives are formulated in terms of multiple endpoints, multiple doses of experimental treatments or multiple patient populations. To maximize the probability of success, the trial’s sponsor needs to select an optimal analytical strategy, e.g., identify the most efficient multiplicity adjustment that accounts for all sources of multiplicity in the trial. In addition, multiple clinical objectives may be evaluated at several decision points (interim and final analyses). Multi-stage designs of this kind support many flexible options, including early termination rules, rules for adaptively selecting the best doses or patient populations at an interim look, and offer a slew of advantages over traditional designs with a fixed sample size. If a multi-stage design is employed in the trial, various design parameters such as the timing of interim looks and data-driven decision rules need to be pre-defined and sponsors are often interested in finding an optimal configuration of these parameters. Our book focuses on practical approaches to clinical trial optimization, which is illustrated using multiple case studies based on real-life late-stage clinical trials.
What key challenges in clinical development does the book seek to address? Drug developers often find themselves in situations where they have to deal with a myriad of different options related to the details of a trial’s design, analysis models for key trial endpoints, decision rules, etc. To make things worse, there is typically incomplete information or considerable uncertainty around the underlying assumptions. This can be quite overwhelming and, if there is enough external pressure, development teams may be essentially forced to make quick decisions without carefully examining all relevant options. In other cases, the design and analysis choices may be dictated by some rules of thumb. In this book, we focus on quantitative methods for accurately evaluating all available information and quantifying risks associated with important decisions.
The quantitative framework described in the book is applicable to a broad class of late-stage clinical trials. The book provides detailed examples of how the framework can be applied to Phase III clinical trials with multiple objectives, e.g., clinical trials designed to investigate the efficacy profile of a novel treatment using several endpoints. In addition, illustrations are provided using clinical trials with multi-population designs, where an experimental treatment is evaluated in the overall trial population as well as in one or more pre-defined subsets of the overall population. We approach this general set of problems from a frequentist as well as a Bayesian perspective. It is well known that the latter enables clinical trial sponsors to efficiently account for the uncertainty around key design parameters estimated from previously conducted trials. The quantitative decision-making framework presented in the book is based on Clinical Scenario Evaluation (we will tell you more about it in a second). The most important feature of Clinical Scenario Evaluation is that it enables a project team to transition from decisions made using various rules of thumb to an evidence-based approach to designing and analyzing clinical trials. We argue in the book that a clinical development team’s goal is a comprehensive assessment of all clinically relevant design and analysis options rather than a back-of-the-envelope calculation to compute the number of patients in a multimillion-dollar clinical trial. Without any doubt, it takes patience and focus to go about trial design and analysis in a careful and systematic way but a committed development team will definitely reap significant rewards.
What methods does the book propose to resolve these challenges? The book explores a framework known as the Clinical Scenario Evaluation (CSE) framework which was initially introduced by Benda et al. (2010) and Friede et al. (2010), and refined in other publications. This framework offers an efficient quantitative approach to assessing the performance of candidate designs and analysis methods in clinical trials. It can also be applied to a more general setting to evaluate different strategies at the development plan level. The key idea behind the CSE framework is to deconstruct the complex problem of clinical trial evaluation into three components. The first component is the data model, which describes the process of generating patient data. A data model defines, for example, the sample size, parameters of the endpoint distribution or the patient enrollment and dropout processes. It is important to ensure that a data model covers all plausible sets of statistical assumptions ranging from more conservative to more optimistic assumptions. The second component is the analysis model, which specifies the strategies that will be applied to analyze the data generated from the data model. Analysis methods such as statistical tests and multiple testing procedures are specified in this model. Clinical development teams are encouraged to consider multiple candidate strategies such as several applicable multiple testing procedures. The combination of all design and analysis elements defined in the two models represents the clinical scenarios that will be evaluated in a given trial. The third component, known as the evaluation model, is the component describing the metrics that will be used to investigate the performance of the individual clinical scenarios. Examples of metrics defined in an evaluation model include marginal power or more complex metrics such as disjunctive power (probability that at least one test will be significant in a family of tests). Lastly, Clinical Scenario Evaluation is defined as the application of the evaluation model to the pre-defined clinical scenarios. Assessment of a large number of clinical scenarios can become extremely complex and the CSE framework enables a systematic approach to performing complex evaluations of plausible of data and analysis models selected by a development team.
Based on our experience with the CSE framework, we believe that it serves as a valuable tool available to clinical development teams. We have seen on multiple occasions that this framework helps teams transition from empirical decision-making to a more disciplined approach to designing clinical trials and clinical trial programs.
What potential obstacles are there to implementing these methods? It is well known that, with any new methodology, there is sometimes an important gap between a theoretical framework and its practical applications. This could be a potential obstacle for clinical trial researchers who are interested in implementing the CSE principles to compare multiple designs and analysis methods in their trials. To bridge this gap and facilitate the implementation of simulation-based CSE approaches by statisticians and development teams, we have developed an R package called Mediana (Paux and Dmitrienko, 2017). This open-source software has been built to mimic the three models of the CSE framework and streamline its implementation in a broad class of clinical trial applications. The package supports a large number of data, analysis and evaluation models and can be used to perform a systematic and critical review of designs and analysis methods. Case studies presented in the book were implemented using Mediana and the R code from the book is available on the package’s web site.
How are the methods currently being implemented in practice? Based on our experience, the simulation-based approaches to CSE and clinical trial optimization have found numerous applications in late-phase clinical trials. We have seen multiple cases where the Mediana package has been used to support complex evaluations in Phase III trials. We receive emails from clinical statisticians around the world and are happy to learn that lots of statisticians take advantage of this package. A recent example that comes to mind is a comprehensive evaluation of candidate trial designs in an event-driven trial with non-exponential endpoint distributions and non-homogenous treatment effects over the trial’s population due to the presence of predictive markers.
We are also glad that statisticians across the biopharmaceutical industry have started using this package to perform clinical trial optimization to find optimal design parameters or optimal analysis strategies. We have provided multiple examples of optimization exercises in the book, in the context of clinical trials with multiple objectives or trials with pre-defined patient subgroups. In addition, applications of the general CSE approach to optimal selection of key parameters of a simple adaptive design were presented in a recent paper (Dmitrienko et al., 2016).
Thank you to the authors for their insights. They look forward to future developments and receiving further feedback and case studies.
To learn more about how to access a copy of the book click here.
About the authors of Clinical Trial Optimization with R
Gautier Paux, MSc, has over 7 years of experience in pharmaceutical industry and is currently working as Biostatistics Project Manager at Servier in France where he is responsible for Phase I to III clinical trials in Oncology. His interests in biostatistics include multiplicity issues in clinical trials, subgroup analysis, clinical trial simulations/optimization and early phase design in Oncology. He has actively collaborated with international experts and published several publications on these topics. He has played a leading role in the development of open-source software (Mediana R package) and has recently contributed to the book Clinical Trial Optimization Using R published by Chapman and Hall/CRC Press.
Alex Dmitrienko, Ph.D., Founder and President of Mediana Inc, has over 20 years of clinical trial experience and has been actively involved in biostatistical research with emphasis on multiplicity issues in clinical trials, subgroup analysis, innovative trial designs and clinical trial optimization. He has authored/edited two SAS Press books (Analysis of Clinical Trials Using SAS and Pharmaceutical Statistics Using SAS) and two Chapman and Hall/CRC Press books (Multiple Testing Problems in Pharmaceutical Statistics and Clinical Trial Optimization Using R). Dr. Dmitrienko has served as an Associate Editor for The American Statistician, Biometrics and Statistics in Medicine and is a Fellow of the American Statistical Association.
Interested in learning about clinical trial decision-making approaches? Click the button below to gain access to our webinar replay ' Dual Target Methods for Go/No-Go Decision-Making'.