As the use of advanced and innovative clinical trial designs continue to rise, sponsors often wonder which estimation methods are best used for measuring treatment effects with novel trial designs. A variety of estimation methods exist, and it is essential to understand their strengths and limitations. However, the FDA’s most recent guidance on adaptive designs only mentions controlling Type I error, not estimation. More research is needed to evaluate methods for point estimates and confidence intervals within adaptive settings.
Cytel statisticians and colleagues at Harvard University and Vertex Pharmaceuticals, recently employed extensive simulations to examine the strengths and weaknesses of various estimation methods for adaptive settings. Their findings place sponsors on the path to preparing even stronger evidence packages for regulatory submissions involving complex designs.
Adaptive group sequential designs enable clinical trial sponsors to update the statistical design of their trials, based on evidence collected during a trial’s implementation. This allows sponsors to stop early when trials are either underperforming or performing better than expected, or to improve power by increasing sample size.
Typically, when using an adaptive design, statisticians must make adjustments when calculating point estimates and confidence intervals to preserve Type I error. While methods for controlling Type I error in adaptive settings are well-established, there is less familiarity with the best methods for computing point estimates and confidence intervals.
At the end of an adaptive trial, one may calculate the Repeated Confidence Interval, which provides conservative coverage of the true treatment effect (e.g. a greater than 95% confidence interval when one wants a 95% confidence interval), or the Backward Image Confidence Interval, which is an extension of the more familiar Stagewise Adjusted Confidence Interval, and provides exact coverage of the true treatment effect (e.g. a 95% confidence interval when one wants a 95% confidence interval). The Backward Image Confidence Interval method enables the calculation of the median unbiased estimate. Note, the median unbiased estimate is ‘exact’ in the sense that it over-estimates treatment effect about as often as it underestimates it.
Nelson et al. (Pharmaceutical Statistics 2021) conducted extensive simulations using East 6.5 to evaluate the performance of the Repeated Confidence Interval, Backward Image Confidence Interval, and naïve (e.g., un-adjusted point estimate and Wald confidence interval) methods following adaptation at an interim look of a group sequential clinical trial.
As expected, naïve estimation methods performed inconsistently, the Backward Image Confidence Interval provided exact coverage, and the Repeated Confidence Interval provided conservative coverage. The authors also noted considerable asymmetry for the Repeated Confidence Interval: while only 2.5% of the Repeated Confidence Intervals had an upper limit below the true treatment effect, far fewer than 2.5% had a lower limit above the true treatment effect.
When conducting an adaptive group sequential clinical trial, the authors recommend using the Backward Image Confidence Interval if possible, or otherwise using Repeated Confidence Intervals, and urge avoiding the naïve Confidence Interval.
About the Author of Blog:
Dr. Esha Senchaudhuri is a research and communications specialist, committed to helping scholars and scientists translate their research findings to public and private sector executives. At Cytel Esha leads content strategy and content production across the company's five business units. She received a doctorate from the London School of Economics in philosophy, and is a former early-career policy fellow of the American Academy of Arts and Sciences. She has taught medical ethics at the Harvard School of Public Health (TH Chan School), and sits on the Steering Committee of the Society for Women in Philosophy's Eastern Division, which is responsible for awarding the Distinguished Woman in Philosophy Award.
