Estimands 101: Interview with Mouna Akacha

It’s been hard to miss the prevalence of estimand-related discussions in the last year.  This is a topic which is very much at the forefront of statistics discussions right now.  We are lucky enough to welcome Mouna Akacha to the blog to give us the lowdown on estimands and the problems and opportunities they represent for the global biopharma industry. 

Mouna is a Consultant in the Statistical Methodology Group of Novartis Pharma AG, based in Basel, Switzerland. She has a wide range of research interests including topics on missing data, longitudinal data and recurrent event data and is an active participant in the current estimand discussions.

Read on to find out everything you ever wanted to know about estimands but were afraid to ask…..

Cytel Welcome Mouna. First of all, can you explain for us- what is an estimand?

Mouna Akacha ( MA) In my understanding, an estimand clearly defines, what is to be estimated. Or in other words, an estimand defines the target of estimation to address the scientific question of interest.

"To define the estimand precisely, one needs to specify at least four attributes."

Usually, in our clinical trials, we are interested in assessing treatment effects. For that, we have to be very precise on which treatment effect we plan to estimate as treatments usually have several effects. For example, a treatment can result in adverse events severe enough to cause treatment discontinuation; it can result in unsatisfactory efficacy and the need for rescue medications , but, of course, it could also lead to acceptable efficacy and tolerability. With so many different treatment effects, which effects are we really interested in? And which effect should be of primary interest? The estimand language and framework will help us to state this more clearly.

To define the estimand precisely, one needs to specify at least four attributes. First of all, we have to say, "What is our population of interest for which we want to address a specific scientific question? This is usually defined through the inclusion and exclusion criteria of a given study.

Then we also have to ask ourselves “What is the variable of interest?” Or, in the terminology we currently use, "What is the endpoint of interest?" So for example, this could be the blood pressure measurement after 12 weeks of treatment.

Also, we have to state which summary measure we want to base our comparison on - this could be for example the mean or the median in the case of a continuous variable.

Finally, we have to precisely define which intervention effect we are interested in. So we have to state how we plan to capture the treatment effect, and for that we have to keep in mind that usually, a treatment has several effects and that after randomization several things can happen that are themselves related to treatment. Such as adverse events, the need for rescue medication and so on.

So we have to ask ourselves, in the presence of these post-randomization events, which are themselves potentially related to the treatment, how do we want to judge whether one treatment is better than another one? And it's particularly this point which, maybe in the past, has been neglected a bit. We tended to be quite vague in saying how we would take these events into account. The details were usually only specified implicitly through choices made about the statistical analysis. The aim of this whole estimand topic is to encourage us to be very specific upfront on what our trial objectives are.

Want to continue to read offline?  Click the button below to download a pdf copy of the interview.

Why has this topic emerged right now as being very important?

(MA) The fact that we sometimes specify the trial objectives too vaguely has led to various challenges. In some cases, this ambiguity has led to misalignment between the trial objectives, the design of the study, the conduct, and then effectively it caused problems in interpretation of the clinical trial results. In some cases, this also caused regulatory challenges.

So there was something happening in the background, but nobody was really putting their finger on what exactly was wrong, until the first time this estimand framework was brought to the clinical trial community through the publication of a document on missing data (1) by The National Academy of Science which was commissioned by the FDA in 2010.  In this document, they highlight the need to first clearly specify the target of estimation, so the estimand, before starting to discuss different methods to handle missing data.

The NAS document focused on missing data which often leads to challenges in the interpretation of clinical trials results, but the relevance and need of the estimand topic was reinforced for an even broader setting by the ICH Steering Committee which published a concept paper in 2014 (2) with the goal of developing a new regulatory statistical guidance. This guidance is suggested to be an addendum to the ICH E9.

As you probably know the ICH E9 is like the holy grail of pharmaceutical statistics. And it's a pretty big deal that this document is going to be amended for the first time since its publication in 1998.  The addendum itself is expected to focus on two main aspects, firstly on estimands, and secondly on sensitivity analysis.

We’ve discussed the problem for the industry and what it's been. As this document develops and the discussion develops, what would you see as being the opportunity for the industry in terms of how it can help clinical trials in general?

 (MA) I think there are plenty of opportunities. First of all, I really think it will improve the transparency and also ensure alignment between the trial objectives, and then the design and conduct of the clinical trial as well as the statistical analysis. In particular, the language that we are provided with through this estimand framework will help us to discuss the relevant questions at the right stage with relevant stakeholders. Agreement on the estimand choice and design can be reached in a more informed way so that we avoid tough discussions at a stage where things are too late because we have already designed and conducted a study in a certain way.

So I think this transparency is really an opportunity to get agreement and fruitful discussions with all the relevant stakeholders at the right time.

In addition, the estimand discussions have highlighted that maybe some established paradigms in the pharmaceutical industry, at least when it comes to the statistical concepts, are maybe not any longer in line with the type of studies that we run nowadays. In the last twenty to thirty years studies became more complex.  For example, they are longer in duration; in many settings there are other treatments on the market such that patients may not stick to their randomized treatment for the whole duration, but they may rather take additional treatments, or switch treatments. To a certain extent, these aspects were not fully captured in the original ICH E9 guideline.

"as statisticians this means that we may need to get out of our comfort zone"

              The discussions around the estimand framework have highlighted a couple of these, let's say, shortcomings or changes in the way that we conduct our studies. And I think we now have the opportunity to really discuss and hopefully use more targeted designs and statistical analyses that investigate objectives and questions that are of interest to us as sponsors, but also to the regulators, to payers, and the patients.

              Of course, for us as statisticians this means that we may need to get out of our comfort zone and learn more about innovative designs and about innovative statistical methods but I think that’s good news.

One opportunity that is maybe missed or not sufficiently pushed at the moment is to involve the clinical folks in these discussions. The estimand framework essentially focuses on what we want to estimate in a clinical trial, or how we want to quantify treatment benefits. And this is of course also or even especially a clinical question. We as statisticians can guide the discussion, moderate it, and raise important questions. But at the end we will always need the input from our clinical colleagues to decide on clinically meaningful estimands.

 Do you see any regional differences with how things are developing in Europe and in the US for example?

 It's a relatively new topic, so of course, many of us in the industry are thinking about it and wrapping our head around it. And there are of course different opinions. But in terms of the regulatory guidance, given that this is driven by the ICH, which has members from all the main regions including the FDA, PMDA, and EMA I would expect agreement between these parties before the document is published.

              But time and experience will show whether we will get recommendations and requests for specific projects, which go into different directions. 

For people who'd like to learn more about this topic, what reading and resources would you recommend?

The concept paper is, of course, worth reading because that was the starting point for this ICH effort. Since then there have been quite a few publications in the field. The PSI (Statisticians in the Pharmaceutical Industry, UK) has created an expert working group that has published a paper. There are papers by regulators from the US and Germany which are certainly worth reading- Tom Permutt and Lisa LaVange published papers in "Statistics and Medicine" last year and Ann-Kristin Leuchs and colleagues have published two to three papers in this area as well. I have also published a couple of papers with some of my colleagues.  (Editor: These publications may be found in the recommended reading section below.)

So there are around ten recent publications that touch upon the current status of the topic from a clinical trial perspective. The topic of estimands, however, is of course not confined to this setting. For example, the causal inference community has already talked about estimands long before the publication of the NAS document. Also, as the addendum comes out, it's guaranteed that there will be more literature and more discussions especially with regard to designs and analyses that target certain estimands.

Thank you Mouna, we look forward to seeing the further evolution of this topic.

Mouna Akacha is a consultant in the Statistical Methodology Group of Novartis Pharma AG, based in Basel, Switzerland. She has a wide range of research interests including topics on missing data, longitudinal data and recurrent event data and is an active participant in the current estimand discussions.

References and recommended reading are listed below.

Liked this?  Click the button below to download a copy to review at your leisure.

References

1)The Prevention and Treatment of Missing Data in Clinical Trials Panel on Handling Missing Data in Clinical Trials; National Research Council ( 2010)

2) Final Concept Paper E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials October 2014

Recommended reading

Akacha, M., Bretz, F., Ohlssen, D., Rosenkranz, G. and Schmidli, H. (2015) ‘Estimands and their role in clinical trials’, 22(3), pp. 1–4.

Akacha, M., Bretz, F. and Ruberg, S. (2016) ‘Estimands in clinical trials - broadening the perspective’, Statistics in Medicine, . doi: 10.1002/sim.7033.

Koch, G.G. and Wiener, L.E. (2016) ‘Commentary for the missing data working group’s perspective for regulatory clinical trials, estimands, and sensitivity analyses’, Statistics in Medicine, 35(17), pp. 2887–2893. doi: 10.1002/sim.6954.

LaVange, L.M. and Permutt, T. (2015) ‘A regulatory perspective on missing data in the aftermath of the NRC report’, Statistics in Medicine, 35(17), pp. 2853–2864. doi: 10.1002/sim.6840.

Leuchs, A.-K., Brandt, A., Zinserling, J. and Benda, N. (2016) ‘Disentangling estimands and the intention-to-treat principle’, Pharmaceutical Statistics, . doi: 10.1002/pst.1791.

Leuchs AK, Zinserling J, Brandt A, Wirtz D, Benda N. Choosing appropriate estimands in clinical trials. Therapeutic Innovation & Regulatory Science. 2015;49:584-592

Permutt, T. (2015a) ‘A taxonomy of estimands for regulatory clinical trials with discontinuations’, Statistics in Medicine, 35(17), pp. 2865–2875. doi: 10.1002/sim.6841.

Permutt, T. (2015b) ‘Sensitivity analysis for missing data in regulatory submissions’, Statistics in Medicine, 35(17), pp. 2876–2879. doi: 10.1002/sim.6753.

Phillips, A., Abellan-Andres, J., Soren, A., Bretz, F., Fletcher, C., France, L., Garrett, A., Harris, R., Kjaer, M., Keene, O., Morgan, D., O’Kelly, M. and Roger, J. (2016) ‘Estimands: Discussion points from the PSI estimands and sensitivity expert group’, Pharmaceutical Statistics, , p. n/a–n/a. doi: 10.1002/pst.1745.