When conducting a clinical trial with small or sparse data sets, statistical methods meant for large sample sizes may fail to obtain an accurate interpretation of data. This is where computationally challenging exact methods often come into play.
Exact methods, however, are inferentially conservative in the sense that due to small sample sizes, the actual Type 1 error rate is often smaller than the nominal (intended) rate [1]. There exists an array of strategies to combat this troublesome feature of exact tests, each of which varies along the parameter of computational complexity.
In a June 2015 webinar, Professor Chris Corcoran of the University of Utah, illustrated three strategies to reduce conservatism along with simple software techniques to compute these results. He considers these strategies with a view to optimizing type 1 error control, maintaining high statistical power, and ensuring rapid computation.
Chris considers the following strategies to reduce conservatism.
Strategies to reduce conservatism:
The video below shows Chris employing these strategies to clear, believable examples, showcased using Cytel’s StatXact 11 suite. Click to watch.
[3] Firth, David. "Bias reduction of maximum likelihood estimates." Biometrika 80.1 (1993): 27-38.