The Risk of Under-exploring Trial Design Options: A New Case Study
Earlier this year, Cytel founder Cyrus Mehta observed that clinical trial design is often treated like an art rather than a science .
Those guiding design selection are often following their intuitions rather than any firm principles of exploration and prioritization of design options. A new Cytel case study demonstrates how misleading those early insights can actually be.
When Cytel began working with French biotech Da Volterra, the biotech had already spent several months exploring sample size re-estimation designs to overcome one of their resource constraints. Da Volterra assumed that a sample size re-estimation would offer them a sound chance of distributing their enrollment to meet the demands enrollment size. After all, the entire point of a sample size re-estimation is to devote a portion of the sample size to the early part of a trial, and then determine the full sample based on conditional power at an interim look. When conditional power suggests that a trial will be underpowered, only then does a higher sample size enter the picture. This is a common tactic for sponsors seeking to manage enrollment targets.
Yet a common tactic might not be the one necessary for a specific scenario.
Working with Cytel’s biostatisticians, Da Volterra was able to use Solara’s powerful simulation engines to explore over 1500 design options and scenario variants within a couple of hours. This was achieved by first creating a ‘design space’ of all 1500 possible designs, and then ranking them with a number of quantitative decision tools.
When the designs were created and ranked, it turned out that a group sequential design actually optimized for speed and power, given a particular sample size. The group sequential design maintained the sample size restrictions that a sample size re-estimation would have achieved, but also increased other performance characteristics by 10-20%.
The lesson here is one that is familiar to those with a statistical background, but which sometimes is overlooked in trial design: simply because a rule or hypothesis works for a typical (average) specimen hardly means that it will work for all specimens. Similarly, a trial sponsor is not seeking principles that optimize the average trial, but one that optimizes a specific trial.
Not long ago, it was impossible to discover the optimal trial for a specific situation within a reasonable framework of time. As it took several weeks to design and optimize any given trial, examining thousands of trial designs was simply not an option. When intuitions are the best place to begin, the traditional response to sample size re-estimation would have made sense.
Now statisticians can quickly examine thousands of designs, making it more reasonable to aim for a context-specific exploration of optimal designs.
About the Author of Blog:
Dr. Esha Senchaudhuri is a research and communications specialist, committed to helping scholars and scientists translate their research findings to public and private sector executives. At Cytel Esha leads content strategy and content production across the company's five business units. She received a doctorate from the London School of Economics in philosophy, and is a former early-career policy fellow of the American Academy of Arts and Sciences. She has taught medical ethics at the Harvard School of Public Health (TH Chan School), and sits on the Steering Committee of the Society for Women in Philosophy's Eastern Division, which is responsible for awarding the Distinguished Woman in Philosophy Award.