Data-Driven Trial Planning: An Interview with Pfizer's Chris Conklin
Data driven decision-making can ensure that every feasibility team achieves its enrollment milestones. By transforming how pharmaceutical companies and CROs conduct feasibility studies, new techniques in recruitment planning are affecting every aspect of trial planning and clinical development strategy.
We sat down with Chris Conklin, the Director of the Feasibility Center for Excellence at Pfizer (pictured below) to discuss innovative ways to think about enrollment factors, and how he uses high-precision forecasting tools like Cytel’s EnForeSys®.
Cytel: Let’s talk a little bit about feasibility studies for the non-specialist. Beyond the general question of is a trial feasible, what are the more specific questions you tend to worry about?
Conklin: The chief aim is to accurately estimate enrollment durations. Our key objective at Pfizer is to reduce variability between our planned enrollment and actual enrollment. We do that through a number of ways. One of the ways is just to understand how protocol design elements will drive enrollment, so we look at historical internal data and external data (through industry collaborations or third party data), and assess how our protocol compares to those reference studies, and try to understand the extent to which certain design elements are influencing enrollment.
Secondly, we aim to optimize enrollment, [based on] how our study clinicians have defined the patient population and inclusion and exclusion criteria just to see if there are opportunities to accelerate. Primarily we justify these arguments through real-world data. For example, we might point out, “This last requirement will make it difficult to find patients. Can you loosen that?” Then focus our efforts on identifying the best countries, investigators, to conduct that specific protocol.
Cytel: Could you tell us a bit about your projects?
Conklin: We’ve recently spent a lot of time assessing the probability of success of achieving on-time enrollment for several studies in one of our key development programs after implementing some mitigations designed to improve recruitment. Our usual planning tools use straight line calculations and projections based on single point input parameters. By using Enforesys, we were able to account for variability by modeling across a range of input parameters. We were able to leverage the actual data that we had at that point, and also to account for uncertainty by incorporating our assumptions about the range of potential impacts our mitigations would have. The result of doing that was a much more thorough understanding of the factors that were driving enrollment.
Cytel: What did you learn from looking at enrollment in such a way? What did you learn from these relationships of inputs?
Conklin: Obviously we knew that the number of sites, screening rates, screen failure ratio, were all important factors driving enrollment, but we didn’t truly understand the degree to which each of these factors was driving enrollment. It turned out that in this particular case the screen failure ratio was the single most important factor that was impacting our enrollment, and that we could have a greater impact on enrollment by lowering the screen failure ratio by 10 points than we could by adding a significant number of sites. We learned that if we did not significantly reduce screen failure ratio, we would have a very low probability of achieving our goal.
If we can do this even before the study starts…if we go through the process and plan this way, chances are that more often than not, the actual should be close to what plan we put in place.
Cytel: Doesn’t this make planning more complex?
Conklin: No, it makes the planning more thorough. These are inputs you should be considering anyway. We try to have evidence behind these inputs, and where we don’t have evidence we come to agreement, whether it be from a clinician or a regulatory person, or whomever on the team needs to have input. The process makes for a more thorough plan. Because you provide a range of input parameters it forces you to articulate the assumptions behind your inputs whether they be data driven, or truly assumptions, and align on a plan.
Cytel: What are the advantages of simulation such as that in EnForeSys?
Conklin: The reason I like EnForeSys specifically or use of the Monte Carlo simulation in general is that it allows you to deal with uncertainty and it also allows you to express outcomes in probabilistic terms. So many things influence the enrollment duration, protocol design, country footprint– investigator excitement of protocol, competitive landscape. There is rarely certainty around in any of these things. Simulation allows you to deal with the variability.
Cytel: Could you tell us a bit about your use of conventional methods compared with EnForeSys? What does EnForeSys add?
Conklin: Enforesys allows you to use data when you have data as inputs. Also, when you don’t have data, you can reach some consensus around possible ranges; 30% to 50% screen failure, for example. Most planning tools allow for only single point inputs as opposed to a range. If you are off on one input of your plan then your plan is bad.
EnForeSys allows you to handle a range of input parameters so the model can consider all possibilities rather than only the opinion of the loudest voice in the room, for example. By working to gain consensus on the range of input parameters and aligning on those inputs you are able to incorporate opinions of all relevant study team members. This method of modeling also helps you to better understand where the risk is. Also if you reach consensus on inputs it means that the team has to accept the outputs.
Cytel: For example?
Conklin: One output is the range of likely enrollment durations and probability of achieving a given enrollment duration. The team can decide on targets: We might want to be aggressive and select a target that we have a 10% chance of achieving. The difference here is that we can communicate that to the organization so that it understands it’s an aggressive target. Everyone is going in with the same expectations. We also have great insight into the conditions that need to be present in order to achieve that target and the levers that we can pull to make hitting that target more likely. If there isn’t a compelling business need to be that aggressive, we can choose a more conservative target. Other planning tools offer a single point estimate, and little insight into how likely it is to achieve your targets or what you can do in order to increase the probability of doing so.
Cytel: So how did you use EnForeSys to build consensus?
Conklin: Well it is time-consuming, seeking input from people not accustomed to it. Everyone’s got a perspective on a valuable input. Say we need 12-months or 18-months for a trial. We review inputs one by one. I have one view, a clinician another. Now we can incorporate both views, into the range of input. If everyone agrees on the input then they have to accept the output.
Cytel: Why do this in-house rather than use a CRO?
Conklin: We can use this tool in the early planning stages of a program or before we even have a contract with the CRO, and then later we can use it to begin to validate the work the CRO has done after a contract has been signed.
I think if you have this function in house you can get started much earlier. As soon as you have a development program you can start talking about timelines, countries to go to and things like that. The main advantage is beginning earlier and being able to be present for a lot of conversations.
Cytel: How much statistical know-how would you say a person needs to use EnForeSys?
Conklin: So none of us are statisticians in this group. I guess we have varying degrees of statistical know-how. So you don’t need to be a statistician. That being said, it is helpful to have a statistician to consult with. One thing I like about EnForeSys is that I have a choice of statistical distributions to input into my parameters, so it’s helpful to have a statistician determine which distribution your historical data follows, and then help the team choose the right distribution.
So yes, it is easy to use but you certainly don’t need to have a statistician. We have a team developing our model, but we have a statistician helping us to validate our own data, helping us provide inputs according to the distribution that we select.
Cytel: You will be speaking at the SCOPE conference later this year. What will your talk be about?
Conklin: It’s going to be very similar to this discussion, focused on the value of planning this way, the value of having discussions and coming to consensus, making your best assumptions when you don’t, the value of probabilistic ranges, really setting goals that make sense for the study and for the business. If there’s a business need there’s reason to be aggressive, but at least now everyone knows that it’s quantified.
You also get a deeper understanding of what factors are really driving enrollment, so you know where to focus your resources and what you can reasonably expect to achieve. There is the value of process and value of planning, and the value of expressing projections in probabilistic terms.
We’re convinced that this is the way we’re supposed to be doing this, but we’re still working through the kinks of the process.
Cytel: Thanks so much for sharing your thoughts on trial planning and model-based enrollment forecasting. We are looking forward to hearing you speak in Orlando.
Abstract of Chris Conklin's talk at the SCOPE Conference in Orlando: We'll discuss the value of working with study team members to align on the range of input parameters for use in simulations and the benefits of expressing output enrollment cycle times in probabilistic terms. The EnForeSys™ software uses these inputs to generate a more realistic enrollment plan.
Related Items of Interest
3 Key Trends in Clinical Trial Enrollment Forecasting
How to Take Control of Your Enrollment Woes
