Frequently Asked Questions

StatXact

Statistical Issues
Binomial Differences and Ratios, Exact Confidence Intervals
Censored Data, Power and Sample Size
Comparing Two Proportions in a Single Multinomial Population
Confidence interval, one-sided Confidence interval, one-sided
Confidence Intervals When Observed Value is a Boundary
Design of Experiments
Equivalence and Non-Inferiority Tests
Exact vs. Non-Exact
LePage Test
Log-rank Test and Wilcoxon-Gehan Test
Logrank Test
Odds-Ratio Formula
One Binomial:Computation of P-Values
Pitman's Test
Poisson Regression, Conditional/Stratified
Proportional Hazards
Relative Risk
Wilcoxon Signed Rank Rest

Data Handling Issues
Binomial Counts, Entering in SAS
Case Data vs. Table Data
Large Data Sets
Exporting files to Excel
Monte Carlo Simulations, Setting the Number
Printing Asymptotic P-Values (SAS)
Proc-StatXact vs. Standalone StatXact
Running Proc-StatXact

LogXact

Alternative Exact Approaches
Binary Logistic Model
Classification
Collinearity
Covariance Matrix
Degeneracy
General Contrasts
Imbalanced Data
Interactions
Missing Data
Multinomial Polytomous Outcomes
Odds Ratio, Interpreting Point Estimate
Multinomial Polytomous Outcomes
Power and Sample Size
Predictor variable required
P-Values, Two-Sided
Separability
Stratification
Summing Betas
Temp Files

StatXact PROCs & LogXact PROCs
Installing and Configuring

Product Validation
Egret

East
Asymmetric Boundaries
Binomial Endpoint: Exact Distribution
Changing Log Scale to Natural Scale in a Survival Study Plot
Data Driven Interim Monitoring
Designing a Three-Arm Study with East
Implication on Critical Points in Interim Analysis
Inflation Factor Comparison
Interim Monitoring: Post-Hoc Power: Error in Display
One Sample Study Design
Survival Non-inferiority Trials with Indirect Superiority Alternative
What is Allowed for Test Statistic in Interim Monitoring

Binomial Differences and Ratios, Exact Confidence Intervals
Q. I have seen several variants of exact tests for differences and ratios of binomial parameters. How do they differ?
A. An early variant was based on inverting a test based on a very inefficient statistic due to Santner and Snell (1980). More recently, StatXact has introduced (1) Exact confidence intervals for differences and ratios of binomial parameters based on inverting the exact 1-sided score-test (Due to Chan and Zhang, 1999), and (2) Exact confidence intervals for differences and ratios of binomial parameters based on inverting the exact 2-sided score-test (due to Agresti and Min, 2001). This test is more efficient than the Chan and Zhang test because it reduces discreteness by being 2-sided.

Censored Data, Power and Sample Size
Q. Does StatXact compute power and/or sample sizes based on censored (survival) data? How about censored data with fixed sample sizes?
A. We have a very inexpensive DOS product called Egret-Siz that can compute these quantities.

Confidence interval, one-sided, StatXact PROCs
Q. I am using PROC Binomial to calculate the sensitivity of a study drug relative to a gold standard.  The statistical analysis plan states that a one-sided 95% lower confidence bound for the estimate will be given.  If GS is the gold standard, SD is the study drug, 1 = abnormal and 0 = normal, I would use the following code:

PROC BINOMIAL DATA=TEST;

WHERE GS = 1;
BI/BS;
OUTCOME SD;
RUN;

This code would give me 2-sided exact 95% confidence intervals using Clopper-Pearson and Blyth-Still-Casella.  Is there an option in Proc-StatXact that would allow a 1-sided 95% confidence interval?

A. Proc Binomial produces a two-sided confidence interval. If you need a 95% one-sided confidence interval, you will have to specify the confidence level as 90% and then take either the lower limit or upper limit as one end of one-sided 95% confidence interval.

For your example, you will be specifying this as
PROC BINOMIAL DATA=TEST ALPHA=0.90;WHERE GS = 1;

BI/BS;
OUTCOME SD;
RUN;

Top

Confidence Intervals When Observed Value is a Boundary
Q. My company has been using StatXact for many years. We have been looking at the results generated by the current version of StatXact for exact one-sample binomial confidence intervals, specifically at the extreme cases (0/N or N/N successes).
It seems that StatXact implements 2-sided C.I.'s at the extremes. Another possible approach is to use 1-sided C.I.'s at the extremes.
By way of example: suppose you have 10 trials in which you observe no successes. Version 2.11 produces (0, 0.2589) as the C.I. while the current version of StatXact, implementing a 2-sided C.I., produces (0, 0.3085).

A. We have had mixed feelings about using up all the alpha instead of alpha/2 whenever a boundary value of the test statistic is observed. This question arises in all our exact procedures such as one binomial, two binomials, stratified 2x2 contingency tables and logistic regression. At one time we were in fact using up all the alpha. But now, for consistency we use alpha/2. Most statisticians want to see consistency between the two-sided p-value and the confidence interval. That is, if the two-sided p-value exceeds 0.05, they would like the corresponding 95% confidence interval to include the value of the parameter under the null hypothesis. Conversely if the two-sided p-value is less than 0.05, they would like the corresponding 95% confidence interval to exclude the value of the parameter under the null hypothesis. Although the earlier StatXact method will preserve the desired coverage of the confidence interval and produce shorter intervals when an extreme value is encountered, it won't have the above consistency property.
For example, suppose you want to test the null hypothesis that p=0.5. And suppose you have observed 5 successes in 5 tosses of a coin. The exact two-sided p-value is 0.0625. Since it exceeds 0.05, consistency requires that the 95% CI should include 0.5. And indeed it does. The 95% CI is (0.4782, 1). If we were to abruptly shift to a 90% CI in this case (i.e., use up all the 5% error in the lower confidence bound), we would get a CI of (0.5493, 1). And this would not be consistent with the two-sided p-value.
But there is nothing wrong with your approach either. If you prefer to use up all the error in one tail when an extreme result is encountered, you can always do so by changing the confidence coefficient to 1-2*alpha instead of 1-alpha.

Design of Experiments
Q. I am doing a design of experiment. Two of them. A 2x3 and a 3x3. Two factors at two levels and three levels, and two factors each at three levels) the sample sizes are very small (20-30). The responses are number of failed in 20 devices. The engineer running the experiment expects very few failures. Can StatXact solve this problem and if so by what method? What other types of designed experiments can StatXact analyze? Can it analyze data from a factorial experiment if the responses are pass/fail? Fractional factorial if the experiments responses are pass/fail?

A. StatXact can handle factorial designs using stratified R x C analysis. For example, consider two factors at two levels with a response variable that is binomial (failure/no failure). The data can be looked at as a stratified 2 x 2 data. The stratum variable can be var 2. For each of the 3 levels of var 2 you have a 2 x 2 table data: response vs. var 1 levels. This type of data can be analyzed using the procedures available under stratified 2 x 2 tables in StatXact.
Another example: Two factors each at three levels and the response variable is binomial (failure/no failure). The data of this study can be viewed as stratified 2 x 3 data and the analysis can be using the procedures available for stratified 2 x C data in StatXact.
If your factorial design data can be meaningfully cast into stratified a R x C data structure, StatXact will be of help to you. There is no procedure available in StatXact for analyzing data of fractional factorial design.

Equivalence and Non-Inferiority Tests
Q. Is there guidance on how to use exact confidence intervals for tests of equivalence and non-inferiority?

A. Using a confidence interval to derive an exact p-value for a test requires some thought. Many people get confused about the choice of confidence level, and which tail of the confidence interval to use, when trying to derive a p-value from a confidence interval. We have to
also worry about whether it is a one-sided or two-sided p-value. It is simpler just to run the exact tests of non-inferiority and equivalence that StatXact now provides.

Top

Exact vs. Non-Exact
Q. When should I use an exact test? For example, in a 2x2 table, when should I use Fisher's Exact, as compared to a chi-squared or corrected chi-squared procedure?

A. Now that exact tests are generally available in SAS, SPSS and StatXact, use them all the time regardless of whether you have a small or a large sample. The exact test is guaranteed to preserve the type-one error whereas the asymptotic test is not. This business of using an exact test only for small samples is too tricky. The fact is, you often won't know when you have a small sample. In our StatXact manual we have numerous data sets where the total sample size is in the thousands, and yet the asymptotic answer does not match the exact answer because of the imbalances in the treatment allocation. The only way you know you have a large sample is if the exact and asymptotic answers match. So why bother worrying about it. Use the exact test and you know you have a valid result. And forget about continuity corrections that may or may not work.

LePage Test
Q. I was trying to implement the LePage (1971) test cited in section 5.3 of Hollander and Wolfe. This test performs a permutation test on the sum of the standardized values of the Wilcoxon and Ansari-Bradley scores. Will the MERT transformation of these scores would give me the LePage test?

A. Yes, essentially the MERT option gives you the equivalent of the LePage test for free.

Log-rank Test and Wilcoxon-Gehan Test
Q. Are StatXact's log-rank test and Wilcoxon-Gehan test results different from those obtained by the standard asymptotic method?  

A. The exact logrank and exact Wilcoxon-Gehan tests in StatXact are linear rank tests based on permuting the generalized exponential and generalized Gehan scores, respectively, as specified in Kalbfleisch and Prentice ( The Analysis of Failure Time Data , John Wiley, 1980, Section 6.2). With this choice of scores for the uncensored case, the logrank scores specialize to the usual Savage exponential scores test and the Wilcoxon-Gehan test specializes to the usual Wilcoxon rank-sum test.
The logrank test in StatXact converts the censored survival data into logrank scores as described in Kalbfleisch and Prentice, then performs a two-sample linear rank permutation test on these scores. This is asymptotically equivalent to taking a 2x2 table at each failure time, but is not equivalent in small samples.
Performance does differ between the exact and test and the standard asymptotic test. One popular implementation of the latter in the case of censored data, for example, uses the asymptotic variance derived from a proportional hazards model whereas StatXact uses the exact (permutation) variance derived from the appropriate permutation test. It is shown in Brown (1984, Biometrika , Vol 7, 65-74) and Davidov and Zelen (1998, Lifetime Data Analysis , 4, 309-327) that the power of the test based on the permutation variance is slightly higher than that based on the proportional hazards model.
The exact tests and their linear rank scores, as well as the StatXact output are fully documented in the StatXact User Manual.

Top

Logrank Test
Q. In Kalbfleisch, Section 6.4.2 (Variance Estimation), they write "A permutation approach is commonly used to calculate the variance of a linear rank statistic. This approach does not extend in a convenient way to arbitrarily censored data..." They then say the observed Fisher information provides a variance estimator that is generally appropriate. Does StatXact do the "inconvenient" thing and extend the permutation approach to censored data?

A. StatXact uses the Kalbfleisch and Prentice approach of replacing the survival time and censoring indicator with a logrank score. It then permutes these scores across the 2 treatments. But the underlying assumption of this approach is that the censoring distributions are the same for both treatments. For the case of unequal censoring in the two distributions, it is not possible theoretically to eliminate the censoring distributions and obtain an exact permutation test. There are, however, asymptotic tests that handle the unequal censoring. But they are not valid in small samples. So it's not a question of StatXact doing the right thing. In this instance no exact permutation test is even possible. Otherwise we would have by now provided an exact estimate for the hazard ratio, and for other coefficients in the Cox proportional hazard model just as we do for the problem of exact logistic regression.

Medians, Confidence Intervals
Q. Can StatXact calculate confidence intervals for median differences in:

- 2 related samples (with Wilcoxon signed rank test),
- 2 independent samples (with Wilcoxon Mann-Whitney test)

A. Yes, these procedures are available in StatXact and are referred to by the name Hodges-Lehmann estimation procedures. In the case of 2 related samples, this procedure is an extension of Wilcoxon signed rank test and in the case of 2 independent samples it is an extension of Wilcoxon Mann-Whitney test.

Odds-Ratio Formula
Q. The StatXact manual gives a formula for the empirical odds ratio which is the inverse of what is found in Agresti's An Introduction to Categorical Analysis .  Can you explain this inconsistency?

A. StatXact's formula for the empirical odds ratio is consistent with StatXact's formula for the true odds ratio. We recognize that some textbooks use the reciprocal of our definition for the odds ratio in a 2x2 table. That is, whereas we use column 1 as the un-exposed group and column 2 as the exposed group, they might use column 1 as the exposed group and column 2 as the un-exposed group. Either convention is ok. As long as one is consistent with the definition and enters the data appropriately into the table, there should be no confusion.
We prefer our definition because it extends in a natural way to ordered 2xc tables. In these tables, column 1 is once again the unexposed group and columns 2, 3, 4 ... represent progressively greater exposures relative to column 1.

Top

One Binomial: Computation of P-Values

Q. From the procedure for One Binomial available in StatXact version 6 , I noticed that two exact Confidence Intervals (CI) are provided, Clopper-Pearson (CP) CI and Blyth-Still-Casella (BSC) CI . But no p-value is reported for BSC CI. For example, n=10, observed value is 4 and the null hypothesis to be tested is Ho: p=0.14. What is the p-value corresponding to BSC CI for this example?

A. In the StatXact-6 log file that you can get in the link, the procedure to compute the one-sided p-value correponding to the Blyth-Still-Casella interval as well as for Clopper-Pearson interval is explained.

One Binomial TXT 3KB

Top

Pitman's Test
Q. Is it possible to do Pitman's test (permutation tests for two groups) in StatXact?

A. Yes - use "Permutation Test with General Scores."

Q. How to carry out Pitman test for the case data shown below? Can these data also be converted to Table data format and analyzed?
Click here to view case data

A. In StatXact, the Pitman test is more commonly referred as ‘Permutation test with General Scores’. This test appears in StatXact (in both the software and the user manual) under “Two Independent Samples” as well as under “Two Ordered Multinomials”.
You can convert the CaseData file into TableData format by using the menu item “DataEditor>Convert to TableData”. The
Click here to view TableData

Poisson Regression, Conditional/Stratified
Q. I need to analyze data derived from the case-series design (Farrington FP, "Relative Incidence Estimation From Case Series for Vaccine Safety Evaluation," Biometrics , 51:228-235, 1995 March). This essentially requires performing conditional Poisson regression or a related procedure. Can I do this in StatXact?

A. StatXact provides the following procedures for Stratified Poisson Samples (the type of data that this design yields):

1. Homogeneity of Relative Risks.
Use the Homogeneity of Relative Risks test to determine if there is indeed a common relative risk across the strata. (See Breslow and Day, The Design and Analysis of Cohort Studies , IARC Scientific Publications No. 82, Lyon, France, 1987, page 110.)

2. CI on Common Relative Risk.
If you have not rejected the hypothesis that the relative risks are the same across the strata, use the CI on Common Relative Risk procedure to estimate the confidence interval for the common relative risk and to test that it equals unity. (See Breslow and Day, 1987, page 107.)

3. Trend Test for Ordered Poisson Rates.
If you have ordered exposure groups then use this procedure to test whether the rates are the same versus the alternative that the rates are increasing (or decreasing) as exposure level increases. This procedure also allows for stratification, if necessary. For example, one may want to check for this trend while also adjusting for factors such as age group, gender, race, etc. (See Breslow and Day, 1987, page 110.)

Proportional Hazards
Q. I would like to know if StatXact has the capability of doing exact proportional hazards modeling (including Confidence Intervals).

A. The logrank test available in StatXact-5 software may meet your need partially. The logrank test can be useful in a testing situation, though no estimation of CI is possible.

Top

Relative Risk
Q. How can I calculate the relative risk with confidence limits from a 2x2 Table.

A. StatXact provides you a procedure to compute the exact confidence interval for the odds ratio, which can be used as an approximation to relative risk under certain conditions.

Survival Analysis, Confidence Intervals
Q. For survival analysis, are estimates of cumulative survival and exact confidence intervals given, or only the exact test results?

A. To the best of our knowledge exact confidence intervals for the hazard ratio in survival analysis cannot be obtained as there is no coherent conditioning argument that will eliminate the censoring distribution under the alternative hypothesis.

Wilcoxon Signed Rank Test
Q. I am interested in calculating a sample size for a study that will be analyzed by a matched paired Wilcoxon signed rank test.  I have two questions for you. Does your program calculate a sample size when a study will be analyzed by the matched paired Wilcoxon signed rank test?   Does it calculate an exact p-value for the matched paired Wilcoxon signed rank test?

A. StatXact does calculate an exact p-value for the matched paired Wilcoxon signed rank test, but does not offer sample size calculations for this test.

Data Handling Issues

Binomial Counts, Entering in SAS
Q. I want to find a binomial confidence interval for the proportion of successes to trials. I see that this is possible using proc binomial given that the variable in the data is in the form 0 0 0 1 0 1, where 1 represents a success.
Is there a way to explicitly tell proc binomial that I have 2 successes out of 6 trials, i.e. I have data set with one variable denoting the number of successes and another variable denoting the number of trials?

A. You can use the weight variable to specify the number of success and number of failures. As an example see the following code.

data test;
input code count;
cards ;
1 2
0 12
;
proc binomial ;
bi ;
ou code;
weight count;
run ;
The output is below

The SAS System

Output created by Cytel Software Corporation
ESTIMATION OF BINOMIAL PARAMETER (PI)
Number of Trials = 14
Number of Successes = 2
Maximum Likelihood Estimate of PI = 0.1429
95.00% Confidence Interval for PI : (Clopper-Pearson) = ( 0.0178 , 0.4281)
Exact P-values for testing PI = 0.5000
One-sided : Pr { T .LE. 2 } = 0.0065
Pr { T .EQ. 2 } = 0.0056
Two-sided : 2 * One-sided = 0.0129

Top

Case Data vs. Table Data
Q. I entered the following data in a two-row table:

Conventional 65 79 90 75 61 85 98 80 97 75
New 90 98 73 79 84 81 98 90 83 88

I went to statistics, selected 2-sample comparison, permutation test. It told me I couldn't do the exact test by Monte Carlo means. When I did the asymptotic test, the values of the test statistic, presumably the sum of the values in the first row, were totally off the wall, in the 3000 range instead of less than 1000.

A. You have used the table editor to enter your data, when you should have used the case editor. You only have 20 data points. But by typing these large numbers into a 2x10 contingency table you are telling StatXact that you actually have over one thousand data points.

Large Data Sets
Q. When using StatXact with a big dataset we get the following error message: "exact method not possible, use the asymptotic method". Is this a software failure or an operating error? How can we get rid of this problem?

A. Neither. Exact methods may not be possible for some large data sets because of large memory requirements. You may try increasing the memory specification in the options. You may also try using memory freeing utility programs like "MaxMem" which you can download free from the website: www.analogx.com . You have not mentioned under which software or test you are having this problem. If you are on our support program and would like, you can send us the data set with the details of the analysis you are trying to do and we will test it here.

Exporting files to Excel
Q. Is it possible to dump results from StatXact into a file? Specifically, if we need to construct hundreds of confidence intervals, can they be generated by StatXact and then dumped into an Excel file?

A. Please note that using Proc-StatXact, which runs under SAS, you can have all your data (e.g. several sets of binomial distribution parameters) together in one file and all the output written to a single OUT file which can then be imported into Excel.

Monte Carlo Simulations, Setting the Number
Q. I want to use PROC STRATIFY to do Zelen's test for homogeneity of odds ratio for stratified 2-by-2 tables:

PROC STRATIFY DATA=RUNDATA OUT=ZOUTP SEED=100 NOPRINT;
HO/MO;
RO GRPID;
CO RESPONSE;
STRATUM SITEID;
/* proc print data=outp; */
RUN;

The number of sites (each site is represented by SITEID) is quite large, e.g. >= 1000, while each table is relatively small, e.g. 3 persons per group. Can the above procedure handle this situation correctly? I have run this procedure in my company's UNIX machine and it didn't finish after about an hour of running and I have to manually terminate it.
I assumed Monte Carlo approach would be used in this situation.
1. Is there a way that I can control the number of Monte Carlo sampling so that it will not take so long to do a test?
2. And what's the guideline to choose such number?
3. Is it possible to let Proc-StatXact run nearly as fast as StatXact in this situation?

A. 1. Yes, there is a way. In Proc-StatXact, the default number of samples used in the simulation is 10,000. This number can be changed by the options specification NSAMP= in Proc Stratify statement. As an example you may specify the Proc statement as

PROC STRATIFY DATA=RUNDATA OUT=ZOUTP SEED=100 NSAMP=100 NOPRINT;

2. You may use the desired precision of the confidence interval as a guide in deciding about the number of simulations. The formula for the confidence interval and other details are discussed in the StatXact user manual.

3. Proc-StatXact runs under the control of SAS system, whereas StatXact runs directly under Windows operating system. Because of the overheads of SAS and because of the differences in the processor speeds of the Unix and Windows computers you have tested, there can be differences in the job execution times for Proc StatXact and StatXact and we do not have much control on this aspect.

Top

Printing Asymptotic P-Values (SAS)
Q. I am running Proc-StatXact on HP-UX. My code is simple enough:

proc twosampl data=foo out=new;
by visit ;
sm/ex ;
po trt ;
re sbjasen ;
run ;

Problem:
1. Output has NO by-variable.
2. When I print dataset NEW and look at what I assume to be the exact p-value (VALUE), it does not mach what is in the output. It lists the value 1 (or 0) for all visits.

A. Your problem is due to the way Proc Print of SAS works and we have no control over it. But the solution is simple. You can specify a format statement in your code for printing the OUT file.

Proc-StatXact vs. StatXact Standalone
Q. I am running SAS on Windows 2000. I also have licensed StatXact standalone in my computer. Now how can I conduct an exact test on risk difference of two proportions?

A. StatXact is a stand-alone Windows software product, and Proc-Statxact is related version that works under SAS. To run proc Binomial you will have to purchase and install Proc-StatXact first. Once Proc-StatXact is installed, procedures like proc binomial can be run in the same way all the SAS procedures are run.

Here is an example:
Data:
GroupA GroupB
yes 5 9
no 5 1
Total 10 10

To compare the proportions of yeses (5/10 and 9/10) of the above data, you may use the following SAS code
(Groups: GroupA=1, GroupB=2, Outcome: Yes=1, No=2):

Q. 1. Does Proc-StatXact include all procedures available in StatXact standalone?
2. We work with SAS using one server installation for two users. Is a single license of Proc StatXact within this framework sufficient?

A. 1. Yes, though there may be a brief lag between the release of a new standalone and the release of a corresponding Proc version.
2. You need to have a network user license for the required number of users, which may be 2 or more.

Running Proc-StatXact
Q. I have received the CD and installed it without any problem. However, no one here knows how to use this new software. Could you please let us know how to run a simple job before our trial version expires.

A. Proc-StatXact runs under SAS. Once SAS is up and running, in the program window of SAS, any of the sample programs of Proc-StatXact can be opened and executed.

Top

Alternative Exact Approaches
Q. I'm a psychometrician. I enjoy and respect very much your line of software. I recently finished writing a paper on exact logistic regression, drawing upon Cytel's huge contribution to the area. My question is, could stats packages like SPSS theoretically handle exact logistic regression through a merging of a logistic procedure and an exact procedure? Or would the computations typically become too unwieldy?
A. Yes, theoretically it may be true, but practically no.

Binary Logistic Model
Q. Does the binary logistic regression procedure in LogXact model the probability of Y=1?
A. Yes.

Classification
Q. I am dealing with a problem of learning to discriminate between objects of two classes. The training set is very small (500 objects of one class, and 500 objects of another), but a very large number of attributes (> 2000) can be generated (perhaps some of them are correlated). I will greatly appreciate if you give me any ideas on whether your product can be used in this framework.

A. For classification (discrimination) procedures, our products LogXact and Egret, which have logistic regression capability, can be used. So can our product XLMiner (www.xlminer.net), which also has sampling capabilities for partitioning into training and validation sets. None of these products will be able to handle problems with this many variables (attributes), though. However, you can use a data reduction technique (such as principal components analysis, available in XLMiner) to reduce the very large number of attributes by grouping sets of correlated variables together.

Collinearity
Q. Is collinearity examined?
A. There is no direct procedure in the program to examine collinearity.

Degeneracy
Q. We tried to fit an exact logistic regression model on a data with 20 patients using LogXact. The problem is: the estimate of the coefficient of the intercept is always degenerate. Can we get a real estimate of the intercept?

A. You may try the following: (data set provided by customer):
1) Round off the covariates lgpacdose and lgschdose to 6 decimals by creating two new variables in the data editor and using the menu item CaseEditor-transform variables. The two variables may be named as lgpac and lgsch.
2) Define further two new variables as lgpacx=lgpac-minvalue=lgpac-4.905275 and lgschx=lgsch-minvalue=lgsch-4.60517.
3) Change the 'accuracy of input' to 6 in the dialog box 'options-global'.
4) Now you can fit the model ytox=intercept + prechemo + lgpacx + lgschx, and obtain the intercept value.

Top

Degeneracy
Q. In my data, a bivariate logistic regression using both factors resulted in degeneracy for the second factor, a binomial one. In the final analysis, two more factors are planned to be included, leading to degenerate solutions for all factors. Is there any way to avoid degeneracy?

A. One solution is to selectively discretize one or more covariates to avoid overconditioning, which in turn may avoid degeneracy.
For your data, a new variable (fact1X) is created in the case editor and then using the menu item "caseEditor-Transform variable" the new variable is defined as fact1x=round(fact1,-2)
This built-in function rounds off the values of "fact1" to the nearest 100's (as specified by the "-2"). You can use this argument in the command statement above to round to the desired number of digits.
Carrying out the analysis using fact1x and fact2 as covariates gives results without degeneracy for fact2.
Some additional information: To see what happens when degeneracy occurs, you can do the following: In the 'Binary Logistic Regression' dialog box, define 'outcome' as the response, 'number' as stratum, 'fact1' and 'fact2' as model terms (with fact1 selected as "no" for testing, fact2 selected as "yes" for testing), choose 'Test' and 'exact' options, click on exact distribution check box to specify distribution file path and finally click on 'OK'. After you get the results, click on the "distribution" tab to look at the contents of the distribution file. There you will find the distribution file having a single point for the test statistic with probability 1. This is the degeneracy situation. In the non-degeneracy situation, you will have more than one point in the distribution file.

General Contrasts
Q. I am using LogXact and I'm wondering what tips you might have for deriving contrasts that are of much more general interest than simply getting the estimated parameters using the corner parameterization of LogXact. For example, if there was a way of obtaining MCMC (Markov Chain Monte Carlo) samples using the MCMC option, then I could consider contrasts of any sort by processing those samples.

A. In Section 3.4 of our paper "Exact Logistic Regression: Theory and Examples" we discuss how to perform inference for general contrasts. It requires that you generate the permutation distribution of the sufficient statistics for the beta vector of interest. You can download this paper from our Web site.

Imbalanced Data
Q. What is meant by "imbalanced?" Is this in terms of the balance of response/non-response on the dependent measure or balance of response/non-response on the predictor measures?

A. Either one. If the proportions of responses and non-responses differ greatly, it is one kind of imbalance in the data. The other kind of imbalance is that the distribution (univariate or multivariate) of predictors under response (or non-response) is highly asymmetrical. Exact methods are called for in either case.

Interactions, Automatic Detection
Q. Does LogXact have an option for automatically searching for interactions among predictor (independent) variables? 

A. There is no such automatic procedure available in the program. The interaction terms have to be specified in the model by the user.

Top

Missing Data
Q. This question concerns missing data. In some packages, observations which have one or more missing value in any variable within a model, are deleted (case-wise deletion) before the model is fit. How does LogXact handle this?

A. LogXact automatically deletes the observation if one or more variables of the model have missing values in the observation. The default missing value for any variable is a dot (.). One can also define, individually for each variable, other values or ranges as missing values using the Data Editor.

Missing Data
Q. How can I permanently save a data set with missing value definitions in LogXact? I tried to save under Global plus save file, however I have to redo all variables whenever I open the file.

A. Once you define the missing values for variables in the data editor, you can save the file with File-Save menu item. This will save the dataset with the missing value definitions.

Multinomial, Polytomous Outcomes
Q. Do any of your programs handle a logistic regression for the case of an ordered dependent variable with > 2 levels? Thus, instead of having a 0,1 dependent variable, one might have a 0,1,2 scoring.

A. Our software Egret and LogXact deal with only dichotomous dependent variables, Nevertheless, in the Appendix of the Egret user manual, a special procedure devised by Dr.Harvey Risch, of the Department of Preventive Medicine and Biostatistics at the University of Toronto, Ontario, is outlined; this procedure can be used to carry out multinomial logistic regression in Egret and other packages.

Multinomial, Polytomous Outcomes
Q. Is it possible to specify in LogXact a polychotomous variable as response variable? From a practical point of view, my problem is as following: some patients, over 80 in age, developed acute myocardial infarction and physicians have several treatments to offer them. In fact, there are no predefined rules as to which treatment is best. They asked me to find what risk factors are associated with each of the treatment. There are four possible treatments. The number of patient is very low in one of the categories (52) making exact inference necessary. When specifying "treatment" (coded from 1 to 4) as response variable, LogXact issues a warning message: "invalid value for response".

A. It is not possible to have a polytomous variable as a response variable (as of January 2004, this is a feature that will likely be offered in the next release). Logxact allows only a binary variable (taking values 0 or 1) as the response variable.
In your problem situation, having treatment as a response variable does not seem to be appropriate. The response variable should be the one which measures the status of the patients (like 'alive' or 'dead') after administering the treatment. In such cases, the treatment will be a model term and, since it is a categorical variable, treatment should be defined as a factor variable, before including it as a model term.

Top

Odds Ratio, Interpreting Point Estimate
Q. We used LogXact to calculate odds ratio estimates and confidence intervals based on exact estimation. The following shows the cell sizes for the unstratified 2 by 2 table.

Coding:
Exposure no=0, yes=1
Case-control status: control=0, case=1
Matching factor: 56 strata (coded 1-56)
Case-control study results:
Exposed Cases N=0
Exposed Controls N=1
Unexposed Cases N=74
Unexposed Controls N=144

The LogXact program calculates the following estimates taking the matching into account ("stratified"):
OR = 3.3
Lower Limit CI95% minus infinity (that is zero for our analysis)
Upper Limit CI95% 130.0

Usually, we would only present the limits of the confidence interval and would not interpret the point estimate. Do you have a meaningful interpretation of this OR estimate?

A. Since there are zero exposed cases for all 56 strata, the conditional maximum likelihood estimate of the common odds ratio does not exist. What we provide instead is the median unbiased estimate of the common odds ratio. In your special case, suppose that t1 is the sum of entries in cell(1,1) of all the 56 2x2 tables and t2 represents the table margins for all the 56 tables. Suppose theta is the odds ratio. The median unbiased estimate of theta is the solution to the equation:

f(t1|t2,theta) = 0.5

Suppose theta* is the solution. The physical interpretation of theta* is that the true value of theta is equally likely to be above theta* as it is to be below theta*.
Incidentally, if you are only interested in inference on stratified 2x2 tables you might prefer to use StatXact rather than LogXact. The algorithms in StatXact are much more efficient than those in LogXact for this special case of conditional logistic regression. Consequently you will be able to solve larger problems and solve them faster.

Power and Sample Size
Q. Do you know how to work out power and sample size calculations in logistic regression?

A. LogXact does not provide any facility for power and sample calculations. However, we do have a DOS based software called Egret Siz, which provides sample size and power estimates (using asymptotic methods) for logistic regression , Poisson regression and Cox Proportional Hazards regression models. If you are interested in this product, please contact sales@cytel.com .

Predictor Variable Required
Q. I want to use LogXact to provide an exact confidence interval for a proportion.
An example of what I have in mind is as follows (10 events in 100 trials):

Set up data as
V1 V2
0 90
1 10

Then specify V1 as the outcome variable, with V2 as a case weight.
I can then translate the exact 95% CI limits for the intercept into probability values.
However, I am unable to select the exact method (it's greyed out) unless I also specify a predictor variable. Any way round this?

A. Since the exact method under LogXact is based on conditional inference, there needs to be at least one variable in the model for the conditional inference to be based on. This is the reason for not being able to get the CI for the const term alone in the model.

P-Values, Two-Sided
Q. I have a question about the p-values in LogXact. I was trying to replicate the first example (section 5.1) in the paper by Mehta and Patel entitled "Exact Logisitic Regression: Theory and Examples" which I downloaded from your website.
In the example, two-sided p-values using the scores test are calculated, however, I cannot arrive at the same results using the LogXact software since it is giving me 2*1-sided p-values. These two calculations will not give the same answer because the distributions of the sufficient statistics are not symmetric. Is there any way to get the correct 2-sided p-values (instead of 2*1-sided p-values) in the software?

A. You will be able to get the correct 2-sided p-values in the following way.

In the Statistics-Binary Logistic Regression dialog box, you will have to choose the options 'Test,' 'Score Test,' and 'Exact'. The model terms to be included are LI, SEX and AOP. But the variable selected for testing should be selected one variable at a time. For example LI-Yes, SEX-No and AOP-No. Now clicking on OK will give the exact p-value as reported in the paper.

Top

Separability
Q. I am using LogXact's exact method to determine the association of my exposure to outcome. Quasi-complete separation will not allow me to use the asymptotic test. Is it possible to obtain an odds ratio using the exact method? 95% CI?

A. If you have separability in the data the upper confidence bound of your 95% CI will be infinity but you will be able to obtain a finite lower confidence bound. The MLE estimate of the odds ratio won't exist but you can obtain a median unbiased estimate in its place. If your data consist of several 2x2 tables StatXact can solve this problem for you. If you have additional covariates besides a stratification factor, you will need LogXact.

Stratification
Q. I have found that LogXact shows an error message of "too many variables," even when the number of variables is less than the limit. Is this because I have a stratification variable?

A. Most likely. If the stratum variable has n strata, it is equivalent to having n-1 additional (dummy) variables in the exact logistic model.

Top

Summing Betas, Covariance Matrix
Q. I would like to calculate exact confidence limits for the sum of betas (expected beta value for the joint effect of two exposures using a model with interaction). If there is an asymptotic solution for the model, the postfit window gives only the asymptotic covariance matrix. I would be grateful, if the exact covariance matrix were also available.

A. If we understand you properly, you can define and add a new variable in the DataEditor. This new variable can be the sum of two variables you are interested in. The beta of this new variable and its CI can provide you the required information. Asymptotic estimates fall under the category of unconditional inference and the exact estimates under conditional inference. For the latter, covariance matrix is not defined.

Temporary Files
Q. I am running into some hard drive space problems when fitting exact models in LogXact - there is no room for the temporary files the program wants to create.

A. LogXact attempts to write files to the temp directory on your C drive. If you have another drive with more space, you can specify an alternate temp path through the options dialog box.

A. Several situations can require reconfiguring your Cytel PROCs. These include

•  Upgrading the hardware on servers or clients (e.g. changing the system hardware for better memory management, speed, or security).
•  Changing or upgrading the operating system on servers or clients.
•  Changing administrator.
•  Conflict of Cytel PROCs with other PROCs. In rare instances, a Cytel PROC name (in full or abreviated form) may be identical to that of another SAS PROC. One such example is PROC ASSOCIAT in Cytel PROCs which conflicts with an identically-named procedure in SAS/Enterprise Minor software.

You should contact Cytel tech support in these circumstances. Note that sometimes the test server and production server are different, in which case the administrator may need to first install and test the application before deploying it on the production server. Though the procedures for the two servers are similar, any change in either of the servers may require reconfiguring the Cytel PROCs.

Top

Q. I'm getting the error "ERROR: PROCEDURE <Cytel PROC name> not found" - what can I do?

A. This may be due to one of the above problems. You can do some trouble-shooting as follows: a) Check the definition of "Cytelroot" in the SAS configuration file. Cytel PROCs must be available in that folder. The PROC-StatXact modules are copied in the "PROCsx5" subdirectory of "Cytelroot" and PROC-LogXact modules are copied in the "proclx" subdirectory of "Cytelroot". The format is -set cytelroot <path where Cytel PROCs are installed>. E.g. -set cytelroot "c:\cytelPROCs". Note that "Cytelroot" will not include "PROCsx5" or "proclx" subdirectory in its definition.

b) Check the PATH in the SAS configuration file. It is available normally at the end of the file along with couple of other SAS related paths. Format of the PATH statement is -PATH ( <path where Cytel PROCs are installed> )

In case of PROC-StatXact, you find -PATH ( "!cytelroot\PROCsx5") while for Proc-LogXact you find -PATH ( "!cytelroot\proclx") in the SAS configuration file.

c) Check to be sure that valid licenses are available. For the WINDOWS platform, validate.dll is the license information for Proc-StatXact, validlx.dll is the license information for Proc-LogXact. For UNIX (Sun Solaris) platform, validate.so.1 is the license information. These files should be available in "PROCsx5" or "proclx" subdirectories of <Cytel PROCs> directory for PROC-StatXact and PROC-LogXact respectively.

d) For UNIX (Sun Solaris) platform, the environment variable LD_LIBRARY_PATH setting is extremely important. It should be set to "PROCsx5" subdirectory of the <Cytel PROCs> directory for PROC-StatXact.

e) Cyteltemp in the SAS configuration file represents the directory reference for temporary file generation while executing Cytel PROCs (a different location than the SAS temporary area). The syntax is set cyteltemp <path where temporary files to be generated during execution>

e.g. -set cyteltemp "c:\temp"

Q. I upgraded my SAS software, and now my Cytel PROCs don't work.

A. If you upgrade your major version of SAS, you will need to upgrade your Cytel PROCs as well. The same is true in reverse - PROCs for SAS8 cannot be used on SAS6. (This is due to limitations in the SAS/TOOLKIT software, which is used to develop the Cytel PROCs.)

Q. I've installed my Cytel PROC on the server and client, but it will not run properly on the client.

A. First, make sure you have SAS configured to run in a client-server configuration. Next, make sure you have installed the PROCs using the client-server option. Note that simply copying the server's configuration files to the client will not work. When you use the client installation option for Cytel PROCs, the install procedure will take care of the proper path reference between the server and the client. If, for some reason, this is not working, you can use a UNC reference or (preferably) a mapped drive reference. Contact Cytel tech support if you are accessing SAS via a Windows terminal server or via a published application such as citrix metaframe.

Q. Do Cytel PROCs require a database?

A. No. Data files can be saved on individual PCs or on a network drive, as needed.

Top

Q. When I install a new Cytel PROC, can I keep my old version intact?

A. Yes. The install procedure gives you the choice of a "new install" or "upgrade." In the latter case, the install procedure replaces existing files. If you want to retain your old version, choose "new install" and be sure to specify a different directory from the existing one.

Q. When I renew my license, what happens to keep my PROCs running?

A. We will send you a new license file. Just overwrite the old license file with the new one. (Note: In the past you would have used a utility to update your license; that system is no longer in use.)

Q. Is there a way to know if the installation went correctly?

A. Yes. We provide a validation procedure for the installed software. At the end of validation procedure, a report is printed which indicates the success or failure of the installation. In case of errors, please contact Cytel tech support.

Validation
Egret Software Validation
The statistical results computed by Egret engines have been subjected to rigorous and extensive quality-assurance testing for purposes of validation. A database of large number of datasets has been compiled at Cytel Software Corporation. These datasets have been collected from standard textbooks, published articles, Egret users and beta testers as well as constructed by us in order to test across the range of possible input values.

The main methods used in the validation process are:

Method 1: Comparison of results from StatXact for the analysis of 2xk Contingency Tables. The algorithms used in StatXact are quite different than those used in Egret.

Method 2: Comparison of the results for Logistic Regression with those in LogXact. Here again, the algorithms used in LogXact are quite different from those used in Egret.

Method 3: Comparison of results with corresponding procedures available in SAS and SPSS

Top

A. Although East does not support a 2-sided asymetric design directly, you can certainly evaluate the properties of the above design through the simulation module. You have to trick East into doing it. Proceed as follows:

(1) Design any 1-sided H0-H1 design.
(2) Click on the enhanced simulation button and go over to simulation worksheet.
(3) Edit the upper and lower stopping boundaries so that they match what you want them to be.
(4) Set the value of delta to be zero if you want to simulate under the null hypothesis. Set it to some other value if you want to simulate under the alternative hypothesis.
(5) Simulate the study 10000 times. In this case the simulation results in the (H1) column tell you how many times the upper boundary was crossed in 10000 trials. The results in the (H0) column tell you how many times the lower boundary was crossed in 10000 trials. The sum of these two quantities will be less than 10000. The difference is the number of times that neither boundary was crossed.

Inflation Factor Comparison
Q. There seems to be a difference between the Inflation Factor given in Jennison & Turnbull and that computed by East. In our Info-Based design, we had specified equally spaced information for 2 look 1-sided test with an alpha of 5% and power of 80% with Pocock boundary. For this design, J&T have an Inflation Factor of 1.121 (Table 2.2, looking at the 10% level 2-sided test), but East seems to give a value of 1.126. Why is this difference?

A. If you use Wang-Tsiatis boundary family with shape parameter value as 0.5 for Pocock, you will find the resulting inflation factor in East matching with what is given in J&T. You have compared Lan-DeMets (PK) value in East with the value in J&T, which is not an appropriate comparison.
Click here to view the details with screen shots.

Interim Monitoring: Post-Hoc Power : Error in Display
In IM sheet, when a study terminates, final outputs are shown in the section titled "Final Outputs at Look # ".
The Post-Hoc Power displayed in this section is correct only when the study terminates at the maximum sample size or when an intermediate look is forced to be the last look by choosing the menu item "Tools-Last Look".
In other situations, the Post-Hoc Power displayed is incorrect. This error will be fixed in the next version of East.
But for now, as a workaround, you can get the correct value from the "Post-Hoc Power Chart" in all the situations of study termination.

Top

One Sample Study Design
Q. How to use East for designing one sample problems, both with normal and binomial data.

A. You use the Information Design-module of East for 1-sample problems. For example, suppose you want to design a study to test the null hypothesis H0: P=0.1 versus the alternative hypothesis H1: P=0.2. Suppose this is a group-sequential study with 5 looks and an O'Brien-Fleming boundary to reject H0, and no early stopping to accept H0. You can enter the design parameters into the Information Design-module. Suppose the Effect Size is Delta = P - 0.1 and you wish to achieve 90% power to detect Delta = 0.2- 0.1 = 0.1. East tells you that you need to commit up-front to gathering 1075.0006 units of information in order to achieve 90% power on a 2.5% significance group sequential one-sided test. That is, unless a stopping boundary is crossed, you must keep the study open until [se(Delta_hat)]^(-2) = 1075.0006. Now Delta_hat = P_hat - 0.1. Therefore se(Delta_hat) = sqrt{(P)*(1-P))/N} = sqrt{(0.2)*(1-0.2)/N}. Therefore the desired sample size is obtained by solving the equation [sqrt{(0.2)*(1-0.2)/N}]^(-2) = 1075.0006. Thus N = 0.2*0.8*1075.0006=172.

Binomial Endpoint: Exact Distribution
Q. Is East 4 group sequential design and analysis for Binomial endpoint based on exact binomial distribution?

A. No. At present, East does not have an exact group sequential test for the two-binomial problem. However, we do have a simulation module that allows you to simulate from a binomial population and verify that the operating characteristics provided by East are valid. Our experience with this module shows that for sample sizes of 100 or more the asymptotic theory works out very accurately (unless the treatment arms are severely unbalanced).

Q. I used information-based method to calculate sample size for a trial defined as follows:

alpha=0.1, power =0.75, H0=0.05, adn H1=0.25, effect size=0.20.

With East, I got a sample size of 18. But my colleague employed his method with binomial probability distribution and got a sample size of 10. Which is correct?

A. The method suggested for one sample binomial design in East is
a) essentially for multiple look (k > 1) study design
b) based on asymptotic method and
c) conservative in nature.

Under East Info Design approach, you would get results as shown below:

If you use mid-way p value of 0.15, you will get a sample size estimate of 12. The problem in Binomial design is that the variance is unknown.
The method suggested in East is to choose a sample size which will be the maximum of the above two values, which is a conservative approach.

The method used by your colleague is exact one and the same method is available in software like StatXact or nQuery.
StatXact and nQuery also give the sample size estimate as 10, under exact method, for your study design.
nQuery gives a sample size estimate of 9 by asymptotic method (Chi-Square approximation) for the same design.

Our recommendation for one binomial study design would be:
1) for single look (fixed sample) study: use exact method available in StatXact or nQuery
2) for multiple look, sequential study: use the asymptotic method of East.

Top

Implication on Critical Points in Interim Analysis
Q. For a  binomial design with an overall Alpha (P-value) of 0.05 and with 1 interim analysis (totally 2 analyses), we would assume that the critical value for both 1st and 2nd analyses should be smaller than 0.05. This would be considered as a penalty for interim analysis. But in my case (see the attached file), the critical point for the 2nd analysis is only 1.910, even smaller than 1.96. That means you are more likely to reject the H0 at the final analysis if you perform an interim analysis. Is this correct? Please advise.

A. (1) If you were designing a two-look study with early stopping only to reject H0, the critical point at the second look would indeed be larger than 1.96. That would be the penalty you would have to pay because you have one extra chance of declaring significance by rejecting H0 at the first look.

(2) But in this case you have designed a two-look study with early stopping to either reject H0 or reject H1. There is certainly one extra chance of declaring significance by rejecting H0 at the first look. For this you should indeed pay a penalty. But there is also one extra chance of declaring non-significance by entering the inner-wedge and being forced to reject H1 (accept H0) at the first look. For this you should receive a reward. Sometimes the penalty exceeds the reward and the critical point at the last look exceeds 1.96. At other times the reward exceeds the penalty and the critical value at the last look is less than 1.96. It all depends on how the mathematics works out for the specific choice of parameter values. For example, if you were to alter your design and ask for 90% power instead of 70% power, the critical value at the second look would be 1.96955.

Designing a Three-Arm Study with East
Q. East provides facility for designing two-arm studies. Is there a way to use East for designing a three-arm study?

A. It can be done this way: Design a three-arm study with two separate comparisons: A versus C and B versus C. Each of the two comparisons can be designed with East. But you should use alpha=0.025 instead of 0.05 to correct for the multiplicity.

Top

What is Allowed for Test Statistic in Interim Monitoring
Q.I am conducting interim monitoring with normally distributed data. According to page 72 in the manual (3rd edition), Z-scores must be entered in the column for the "test statistic". We used t-tests for data analysis. What is the procedure to convert the t-value from the test results into a z-score for entry into this window?

A. You can enter any standardized statistic in the column for the test statistic. You could use a Z-statistic or a t-statistic. You can even use an exact p-value, based on Fisher's exact test, and convert it into the corresponding Z-statistic using the formula Z=Phi^{-1}(1-p/2).
If the standardized test statistic crosses the stopping boundary you can terminate the trial.

Comparing Two Proportions in a Single Multinomial Population
Q. I have a single multinomial population where I would like to compare 3 proportions, p1, p2,and p3. The interest is to get exact test for testing p1=p2 vs. p1 not equal p2. I am also interested in getting exact power and sample size calculation under this scenario.

A. The trouble here is that the common value of p1=p2 under the null hypothesis is not specified. Therefore the likelihood function will contain this common value as a nuisance parameter. You have data r1, r2, r3 for probabilities p1, p2, p3. For testing p1=p2, it would be sufficient to look at the distribution of r1 conditional on (r1+r2). By conditioning on r1+r2 we can eliminate the common value of p1=p2 from the conditional likelihood under the null hypothesis. Then we can permute r1 and r2 in all possible ways such that r1+r2=c.
Although StatXact does not at present solve this problem, we will consider including it into a future version of StatXact.

Top

Changing Log Scale to Natural Scale in a Survival Study Plot
Q. In the power curve plot in a survival study, can I change the x-axis scale from log(HR) scale to HR scale?

A. A work-around is as follows:
(1) Design the survival study
(2) Click on the IM icon to go to the interim monitoring module. Notice that bottom right table on the Interim Monitoring workbook displays the effect size (on the log hazard ratio scale) versus conditional power (CP). Since we are at the start of the trial and have not entered any data into this worksheet, this conditional power table is the same as the unconditional power table.
(3) You can use the usual Excel copy and paste facility to copy all the numbers from this table and paste them into an Excel spreadsheet (or you can paste them into an East Scratch Sheet using Insert>Scratch Sheet from the East menu bar.)
(4) Then you can easily transform the effect size column by exponentiating with an Excel formula and you can plot the graph with Excel.

Top

Survival Non-inferiority Trials with Indirect Superiority Alternative
Q. In a non-inferiority design using East, I have some concerns about the design with alternative hypothesis “delta <> 0”. Most papers using non-inferiority are with alternative: delta=0, not superiority. Below is a brief description of my hypotheses:

Null Hypothesis:
hazard ratio for A/B<=0.82 (28 months PFS median for A vs. 23 months PFS median for B)

Alternative Hypothesis:
hazard ratio for A/B>1.18 (28 months PFS median for A vs. 33 months PFS median for B).

I am concerned about the nature of the comparison and the asymmetry of the error rates. As designed, there is a 20% chance of concluding inferiority (that is, of not rejecting the null) when the true hazard ratio is 1.18, and I believe a slightly greater than 50% chance of concluding inferiority when the true hazard ratio is 1.

There is also possibly a problematic interpretation at the final analysis, since if the estimated hazard ratio is 1 at that time, the test statistic would be approximately:
(log(1)-log(0.82))/sqrt(4/245)=1.55 < 2.054 (the expected boundary value)

so an outcome with no difference would be seen as evidence for inferiority.

A: You have to use the reciprocals of your hazard ratios in order to correctly interpret the output from East. Unfortunately in East version 4 we have defined the hazard ratio as lambda_control/lambda_experimental for non-inferiority trials, instead of lambda_experimental/lambda_control as is the case for superiority trials. This may be a source of confusion. We will change this input specification in East 5.

In your example, it seems that A is the control drug and B is the experimental drug, since the PFS for A is better than the PFS for B under the null hypothesis of inferiority. Therefore, the null hypothesis states that A/B = 0.82 (control is less hazardous than treatment) and the alternative hypothesis states that A/B = 1.18 (control is more hazardous than treatment. When the alternative hypothesis is true, the estimate of -ln(A/B)= -ln(1.18) = -0.165. Therefore the study has 80% power at this alternative as shown by the power curve below:

Top

Data Driven Interim Monitoring
Q. The study protocol requires first interim analysis to be based upon the data of 270 patients. The maximum sample size required (at final look) was set to 390. Due to good recruitment, not just 270 but in fact 280 patients will be available at the time of interim analysis. In this situation, how should one proceed? I could think of the following two possible options.
1. Perform statistical analysis with 270 patients as planned in the protocol, applying the adjusted significance level determined in the protocol based on the information fraction avaliable with 270.
2. Perform the first interim look with 280 patients, applying the recalculated significance level based on the information fraction available with 280 patients.

A. I am assuming that you have response data on all the 280 patients at the time of the first interim analysis. In that case, the proper procedure is to perform the interim analysis on the basis of 280 patients, not 270. The information fraction will be 280/390 and East will re-compute the stopping boundary at that information fraction based on the spending function that you have specified in the protocol. In general, it is not realistic to assume that the interim looks will take place exactly as was specified in the protocol. Administrative difficulties might force you to make changes. The whole point of the spending function methodology is that you should have full freedom to deviate from the number and spacing of the interim looks that were specified at the design stage. The regulatory agencies understand this and the ICH-E9 guidance permits you to use spending functions to recompute stopping boundaries.

Q. If for example, in the case of fast recruitment, the first planned interim analysis with exactly 270 patients leads to a borderline adjusted p-value which just exceeds the required significance level, then one could wait for the next x patients and reanalyze the data. In case of significance, one could then declare this significant analysis to be the first interim analysis and describe the additional patients to be arosen just due to overrunning. Will this be the right strategy?

A. There can be two possible strategies:
Strategy 1 (Recommended): Data driven interim analysis. In this strategy, you take one unblinded look after 270 patients and a second unblinded look after 280 patients. As long as you report both the looks in the interim monitoring worksheet, it is very difficult to violate the type-1 error. The spending function approach is extremely robust. If you take a second look very soon after the first look, the stopping boundary will actually become harder to cross.

Strategy 1 screen shot:

In this example, the first look was almost at the boundary. We took a second look after only 5 additional patients. Notice that the second look boundary, Z2=2.931 was actually much stricter than the first look boundary, Z1=2.887. Many statisticians have studied the data driven choice of taking the second look very close to the first look, in a data driven manner, after seeing that the value of Z1 is very close to the boundary. They all conclude that it is almost impossible to "fool the system" and exceed the type-1 error. See, for example, Section 7.4 of Jennison and Turnbull (2000) where they talk about "data dependant timing of analysis”.

Strategy 2 (Not Recommended): Do not report the results of the first unblinded analysis; simply pretend that no previous interim analysis took place up to the time that the boundary is crossed. In the example above this means that you pretend that you never took an interim look after 250 patients, but that you took your first look after 255 patients. This time the boundary will be crossed at look 1. But it would of course be unlawful to deliberately supress the information that you had taken an unblinded interim look at the data.

Strategy 2 screen shot:

Top