Medical researchers and public health experts are becoming more reliant on meta-analyses to capture in summary form, the expansive set of research findings across numerous clinical trials. Data collected from these trials are often heterogenous, requiring advanced statistical tools to remove bias. Unfortunately, for data involving rare events, such statistical tools are few and far between. The result is that heterogeneity in rare event meta-analyses often goes undetected.
A new paper, co-authored by Dr. Pralay Senchaudhuri, Cytel’s Senior Vice President for Research and Product Development, demonstrates an evaluation of many innovative new methods for meta-analyses for rare events, all involving exact methods. These exact methods avoid many of the biases created by beginning with asymptotic assumptions, but in doing so also curtail many of the biases that continue to challenge those conducting meta-analyses for rare events.
Meta-analyses of rare events pose at least two challenges that other forms of meta-analyses do not. Firstly, most studies where there are no observable events use ‘zero’ or ‘undefined’ for effect measures. Oftentimes a continuity correction (a small adjustment) is made to these measures, to avoid excluding them. Unfortunately, when employed for meta-analyses such continuity corrections can introduce bias.
Secondly, as in all meta-analyses, statistical heterogeneity can arise when aggregated studies reflect somewhat different treatments administered, different population samples or even different study designs. Yet the challenges that arise with heterogeneity are compounded in the case of rare events. These challenges accrue in the rare event setting, regardless of whether one tries to measure heterogeneity variance, or whether one tries to build in assumptions about homogeneity (to avoid complexities raised by heterogeneity.) The first approach is statistically quite complex. The latter makes results more prone to inflation of Type I error.
The use of confidence distributions for rare event meta-analyses is achieved in two steps. First, every study is represented by a confidence distribution, using exact methods. Then all the confidence distributions are aggregated into a single measure. A number of different approaches for the use of confidence distributions were evaluated in the recently published paper, using simulations.
Overwhelmingly these methods performed very well for homogenous studies, i.e. when heterogeneity was not a factor. Unfortunately, statistical heterogeneity continues to go undetected in a number of meta-analyses involving rare events. The degree to which this occurs when using confidence distributions is important to understand.
The recently published findings demonstrate that many existing methods that employ confidence distributions for rare event meta-analyses still performed poorly in the face of heterogeneity, though not in the face of homogeneity. Three methods in particularly performed well in the face of heterogeneity, compared to related methods but they were devised to take more appropriate consideration of random effects.
To learn more click below:
About the Author of Blog:
Dr. Esha Senchaudhuri is a research and communications specialist, committed to helping scholars and scientists translate their research findings to public and private sector executives. At Cytel Esha leads content strategy and content production across the company's five business units. She received a doctorate from the London School of Economics in philosophy, and is a former early-career policy fellow of the American Academy of Arts and Sciences. She has taught medical ethics at the Harvard School of Public Health (TH Chan School), and sits on the Steering Committee of the Society for Women in Philosophy's Eastern Division, which is responsible for awarding the Distinguished Woman in Philosophy Award.
