Earlier this summer, we published a series of articles on the need to utilize weighting and prioritization tools in the selection of optimal clinical trial designs. One of the methods for which we advocated included using Pareto Optimization techniques to identify the set of designs over which to deliberate. Cytel’s position paper highlighted how Pareto could be used in conjunction with scoring functions and eNPV considerations to help trial sponsors and statisticians select the design most suited to their objectives. This piece highlights some of the ways this technique can save decision-makers from expending time and energy on designs that do not deliver on their promise.
A quick review of the Pareto Principle
A clinical trial design is said to be Pareto optimal when it is not possible to improve a specific performance characteristic of the trial, without reducing the performance of another. Given the four design options found in the table below, Designs B and D would be considered Pareto optimal. For Design A, all the parameters can be improved by choosing Design B or C, and two can be improved using Design D. For Design C, three design options can be improved using Design B, without compromising the fourth. Therefore, choosing Design C leaves something on the table.
Note that when comparing Design C and Design D, there might be a number of reasons to choose Design C over the longer and more expensive trial. The fact that Design D is Pareto optimal may not be sufficient reason to choose said design. All Pareto-optimality says is that those who find Design C superior to Design D should really consider Design B as their design of choice. Design B contains all the benefits of Design C, plus more.
Pareto and Pairwise Comparisons
One of the underappreciated benefits of Pareto is that the judgments that would arise during pairwise comparisons of designs, still play a function in the final selection of design, but without the strenuous process of actually conducting pairwise comparisons. The four designs above would actually require six sets of pairwise comparisons (Design A to Designs B, C and D; Design B to Designs C and D; and then Designs C and D with each other). Even within the smaller scale of our example, the necessary comparisons collapse to just one using Pareto optimization (Design B to Design D).
This is because those sponsors considering Design A will always be better off with one of the other designs. Similarly, those considering Design C will either be better off with B or D. This means that five of the six comparisons no longer have to be done, without worry that something will be overlooked. The Pareto optimization does the comparison before sponsors have to begin weighing and prioritizing designs.
This might seem like a small achievement when considering only four designs. Many sponsors though, like those using Cytel’s new selection platform Solara, have to compare thousands of design options. As the number of options increase in a linear fashion, there will be an exponential increase in the number of pairwise comparisons. Sponsors, therefore, have to depend on technology that whittles down design options to a small set for consideration, without worrying that something will be overlooked or a good option eliminated.
Clarifying Tradeoffs and Developing Strategic Aims
Many clients come to Cytel fairly certain what their strategic aims are. They will tell our statisticians that they are committed to a high-powered trial or an innovative design that facilitates acceleration of trial timelines. Not long ago, telling this to a statistician would have meant that statisticians would only focus on generating those designs that appeared to meet these specific goals. Considering the table above, had a sponsor told a statistician to focus on only accelerated design options, a statistician working under time-pressure, might have only provided the sponsor with Designs B and C for consideration. The option for Design D would not have been on the table.
Yet when presented with Option D, even those who believed themselves to be committed to an accelerated design might begin to question whether speed is the best strategic aim. Perhaps they did not realize that a 92% power was achievable. Perhaps the extra time required to meet with venture capitalists and other potential investors is now worth the effort.
A separate but often important question is also the worth of an accelerated trial to the sponsor leadership team. Might there be a non-optimal design that maintains a power in the range of 90-95, but which reduces time be a couple of months? Would such a design be worthwhile? The answer will differ from sponsor to sponsor, but having the optimized options all presented will often elicit the discussions necessary to have these strategic conversations in a structured and tangible way.
Tying Clinical Trial Design to Return on Investment
Cost and Revenue Models are complex, and do not easily correspond with trial design selection. Still, by converting a conversation about trial selection into one about sample size, duration and cost, Sponsors can begin talking about expected net present value much earlier during clinical development and gain early wins for downstream benefits.
To learn more about how to employ the Pareto Principle during clinical trial design selection, click the position paper below:
About the Author of Blog:
Dr. Esha Senchaudhuri is a research and communications specialist, committed to helping scholars and scientists translate their research findings to public and private sector executives. At Cytel Esha leads content strategy and content production across the company's five business units. She received a doctorate from the London School of Economics in philosophy, and is a former early-career policy fellow of the American Academy of Arts and Sciences. She has taught medical ethics at the Harvard School of Public Health (TH Chan School), and sits on the Steering Committee of the Society for Women in Philosophy's Eastern Division, which is responsible for awarding the Distinguished Woman in Philosophy Award.