When conducting a clinical trial with small or sparse data sets, statistical methods meant for large sample sizes may fail to obtain an accurate interpretation of data. This is where computationally challenging exact methods often come into play.
Exact methods, however, are inferentially conservative in the sense that due to small sample sizes, the actual Type 1 error rate is often smaller than the nominal (intended) rate . There exists an array of strategies to combat this troublesome feature of exact tests, each of which varies along the parameter of computational complexity.
In a June 2015 webinar, Professor Chris Corcoran of the University of Utah, illustrated three strategies to reduce conservatism along with simple software techniques to compute these results. He considers these strategies with a view to optimizing type 1 error control, maintaining high statistical power, and ensuring rapid computation.
Chris considers the following strategies to reduce conservatism.
Strategies to reduce conservatism:
- Blaker’s Method (Blaker 2000): Refines exact distribution to enable actual Type 1 error rate to come closer to nominal, thus improving statistical power.
- Bayesian Approach for Two Proportions: The proportions are treated as prior distributions, while the difference between them is the posterior distribution.
- Bias Correction of MLEs (Firth 1993): Corrects bias in maximum likelihood estimators, so that one can employ a more robust estimator without the computational challenges of exact methods.
The video below shows Chris employing these strategies to clear, believable examples, showcased using Cytel’s StatXact 11 suite. Click to watch.
 Firth, David. "Bias reduction of maximum likelihood estimates." Biometrika 80.1 (1993): 27-38.