At the East User Group meeting (EUGM) on 25th and 26th October, we will welcome a number of renowned industry speakers to discuss some of the most promising approaches in innovative trial design. Over the next few weeks, we will profile a number of the speakers and their talks. In this blog, we talk with Bob Beckman M.D in advance of his presentation ‘Design Concept for a Confirmatory Basket Trial'. This, the first part of a two-part interview will reveal Bob’s insights into basket trials and their potential. The second part of the blog will delve more deeply into the design concept itself and will be published after the East User Group Meeting. To learn more, read on.
.jpg?width=320&name=R%20Beckman%2013-Oct-09%20(4).jpg)
About Robert Beckman
Robert Beckman, M.D., an oncology clinical researcher and mathematical biologist, has played significant leadership roles in developing new oncology clinical research groups at 4 pharmaceutical companies and in 5 cross-company collaborations, brought 23 oncology therapies into first-in man studies, 5 through phase 2, and 2 to market. He has co-invented novel clinical strategies for proof of concept studies and for early and late biomarker driven clinical development, including a confirmatory basket trial design. He has developed a new approach to cancer precision medicine based on tumor evolution, which has the potential to dramatically enhance survival and cure rates. His versatile publication record, comprising greater than 200 articles and abstracts, ranges from computational chemistry to clinical oncology, emphasizing quantitative approaches. Dr. Beckman is currently Professor of Oncology and of Biostatistics, Bioinformatics, and Biomathematics at Lombardi Comprehensive Cancer Center and the Innovation Center of Biomedical Informatics, Georgetown University Medical Center.
Cytel: Could you tell us a little about your background and how you became interested in this area of research?
Robert Beckman (RB): I'm an oncologist and I became interested in math when I was pretty young. While I taught myself some math, I never actually ever formally learned statistics. As I went through my career I was an interdisciplinary person doing a lot of work at the interface between biology and mathematics. So when I was in a team with biologists I would do some math, and when I was on a team with mathematicians I would be more of a biologist. This work arises from a collaboration I've had for about 10 years with Cong Chen who is an excellent professional statistician at Merck. I was in the industry for 18 years as a clinical research executive wearing an oncologist hat but I always liked to talk to the statisticians. I came to Cong originally with an idea about drug development and he was very interested in my ideas and taking them further from a mathematical point of view. We published a lot of papers together and became close collaborators. Since then a number of other joint ideas have emerged from my collaboration with Cong Chen. This is one of them.
Cytel: Why is the basket trial approach important in drug development today?
RB: Much of the research community has been very interested in exceptional drugs. And of course, exceptional drugs are producing exceptional and exciting results. But there are a lot of drugs that are effective but not exceptional. The exceptional drugs get an enormous amount of hype so that someone from the outside might believe that we are entering an age where we're going to have nothing but exceptional drugs. I think if you work down in the trenches you understand that exceptional drugs are great if you have one, but you have to have a plan for 99 percent of reality which is drugs that are effective but not exceptional. The cost of drug development is unsustainable. And as science produces more and more possibilities, it's becoming even more unsustainable in its cost. The success rate has not gone up as much as the number of interesting possibilities you could look at. And so we need a more efficient way to develop drugs.
Cong and I had originally looked at proof of concept studies, and we found a way to improve the efficiency of Phase 2 and 3 by between 10 to 30 percent by optimizing Phase 2 studies and the go/ no -go decision to phase 3. We thought that was important because 10 to 30 percent of billions of dollars spent in development would be an enormous saving. Basket trials take these savings to a whole other level.
Cytel: Could you explain what a basket trial is?
RB: In a basket trial, you can actually ask the question of whether a drug is effective in multiple indications at once. Let’s say you have three to five indications in the 'basket' and all these indications are grouped by some other fundamental characteristic- such as they all have the same mutation or the same molecular biomarker, or they have the same disease pathogenesis. Instead of saving 10 to 30 percent which we were excited about from our earlier work, in a basket trial you might be able to conduct trials for three to five indications in the basket for a cost that is approximately the same as doing one. So you have a several 100 percent increase in efficiency.
In the past, basket trials have been used generally for two purposes. One is for exploratory work to get an idea of where there is potential activity early in development. And the other is for confirmatory studies of exceptional drugs. This is why I distinguish between exceptional drugs and drugs which are effective at a more normal level.
"The question is -what could you do in the confirmatory space for drugs that are useful and effective but not exceptional? "
What has happened in several instances with exceptional drugs is that the health authorities looked at the level of activity and benefit from these drugs, and at the extraordinary level of scientific evidence, and they accepted, quite reasonably, a different level of evidence than would be normally acceptable.
So, for example, the first basket trial was for Glivec, the drug which made chronic myeloid leukemia into a curable disease and one of the first targeted agents that were designed based on scientific knowledge. This is really a spectacular drug.
There was a basket trial in which a number of very rare cancer types were gathered together into a small trial. These cancer types were very rare, and they might have had just 5 or 10 patients in some of these groups. There was no control group and they looked for tumor shrinkage, a short term endpoint that may or may not correlate with survival -which is what we're of course really interested in. Under ordinary circumstances, this would not have been sufficiently rigorous for a phase 2 study. However, because these very rare tumor types didn't have any other options, and because it's hard to enroll a larger study in rare tumor types, and because there is an enormous body of scientific data suggesting that Glivec might be helpful –all of these attributes of the study that made it seem only exploratory were accepted for FDA approval for those indications where responses were seen. There was one indication where they had 5 patients and the tumor shrunk in 1 of the 5 patients. And the FDA approved Glivec for that indication. In all, only 4 of 40 indications represented in the trial were approved, demonstrating the heterogeneity in response although the indications all shared the same molecular features. There’s a trial with Vemurafenib ongoing that has a similar design. Vemurafenib is a spectacular drug for melanoma that has the b-raf V600E mutation. And again, recently, an immunotherapy was approved for any tumor that has a high mutation burden- this was Merck's Keytruda. These are exceptional cases.
"Could we take the confirmatory part of drug development for either rare diseases or biomarker defined subsets and make it affordable by combining three to five studies into one basket?"
But now the question is -what could you do in the confirmatory space for drugs that are useful and effective but not exceptional? You could forget about the confirmatory space and just use basket trials for the early exploratory space. But most of the money in drug development is spent in the confirmatory space. I also have an interest in rare diseases, and for rare diseases, it may be hard to do a conventional confirmatory trial. You may not be able to find enough patients and you may have a hard time convincing a company to do the trial because it's both very challenging and then there is only a very small market. So our question was - could we take the confirmatory part of drug development for either rare diseases or biomarker defined subsets and make it affordable by combining three to five studies into one basket?
Cytel: What challenges need to be overcome in implementation?
RB: There are two major statistical issues with basket trials that we need to address in the design. One of them is the false positive rate or Type I error. The other is the false negative rate (or Type II error). Imagine you take a bunch of indications that you think are going to be positive based on a hypothesis that they all share a molecular marker. Let's say you take five of them and include in the basket trial. And you find you were right about three of them but wrong about the other two. The difficulty is that at the end of a basket trial you have a pooled analysis of whatever is in the basket. The negative results from the two false indications may dilute the positive results and you would get nothing from having pooled all five of them. That's an important risk.
Vemurafenib, the spectacular drug for melanoma with a b-raf V600E mutation, does not work in colorectal cancer with the same mutation. And nobody predicted that in advance. Once it was observed, the molecular biologists were able to go back and investigate further and find out the reasons why. But it was not predicted beforehand. And so if someone had done a basket trial with melanoma and colorectal cancer indications they may have ended up with a negative trial and thrown the drug away. This is the Type II error issue, one of the issues we are trying to overcome in our design.
At the East User Group Meeting, Beckman will discuss the the specifics of the proposed design, including the methods for control of type I and II error and other aspects of rigor required for confirmation of drugs that are effective but not 'exceptional'. To register for the East User Group Meeting and hear Beckman’s presentation, click the button below.
Many thanks to Dr. Beckman for his insights.
