At the recent CMO Summit East James ( Jim) Bolognese, Cytel’s Senior Director of Strategic Consulting, and Lou Vaickus,MD, FACP, Founder and President of aktaPD jointly explored how the critical connection between a CMO and a biostatistician can help enhance communication with internal and external stakeholders, ensure patient safety and ultimately improve clinical development efficiency. Below you can catch up with some of the key topics they covered and watch the video of the presentation.
P Values –dead, dying or misused?
Within scientific circles, the p-value has been the subject of well documented controversy. In Bolognese’s view, the issue is really one of misunderstanding and misinterpretation. It’s not that the p-value is invalid as a tool, but rather that categorizing results as significant or not significant based solely on the p-value can be misleading. Vaickus alluded to this as the ‘seductive certainty of significance'. Bolognese points out that the more important question for research is: ‘What is the probability of success for the drug?’
Probability of success
This can be defined as the probability that the true difference between outcomes of patients treated with a placebo versus a drug meets or exceeds the target. In other words, what is the likelihood that the results will meet or exceed expectations? The probability of success (POS) for a drug can be influenced by many factors including:
- The sample size
- Power of a study
- Randomization procedure
- Informative priors
- Study conduct
- Measurement quality
In the presentation, Bolognese at 7:23 works through an example to explore the distinction between the p-value approach and the probability of success approach in practice and at 11:24 compares the performance characteristics of these two approaches.
In essence, using a decision analysis approach we would compute the probability of success – the probability that true difference exceeds the target. For example, where the POS calculated is >70% the trial should be continued whereas if the PoS < 50% then the trial should not be continued. On the other hand, if the PoS falls between 50 and 70% then more information is needed. This could be obtained in various ways including looking at secondary endpoints, conducting another trial, or doing sample size re-estimation to enlarge the trial.
Vaickus described how he and Bolognese have worked closely together on projects involving interim analyses which can be helpful for go/ no go decision making in situations with a larger than affordable ‘N’. He advocated that from a CMO point of view, interim analyses are useful for ‘news flow’- improving communication between the clinical and corporate stakeholders and helping to inform the allocation of budget spend. Where interim analyses are used, information can be obtained earlier in the trial with indications of likely success or otherwise. If success appears improbable, there is the opportunity to stop the trial before further resources are expended.
When should the CMO involve a statistician?
It’s clearly critical to involve a biostatistician early in the process. Certainly, Bolognese recommended that CMOs consult a biostatistician before an IND is submitted. Vaickus noted that increasingly IRBs, ethics committees, and PIs, require the creation of a statistical analysis plan before patients are enrolled in a trial.
Cytel works extensively with specialty pharmaceutical, biotechnology and virtual companies supporting the statistical and data collection aspects of their clinical studies. Very often, in these cases, we are working closely with the Chief Medical Officer to ensure the best approaches are taken. To find out more about our clinical research services click below.
Lou Vaickus MD, FACP is Founder and President of aktaPD. He has 30 years in medicine and the pharmaceutical industry and 15 years of executive pharma leadership with extensive experience at start-up, small-, medium-, and large-sized companies in all aspects of drug development and clinical medicine including new chemical/molecular entities, biologics, devices, diagnostics, assay development, and biosimilars.
James Bolognese, MStat is Senior Director, Cytel Strategic Consulting. His expertise encompasses experimental medicine, proof-of-concept study design, and biomarker development.