next up previous


Postscript version of these notes

STAT 350: Lecture 29

Power and Sample Size Calculations

Definition: The power function of a test procedure in a model with parameters $\theta$ is $P_\theta(\mbox{Reject}).$

Definition: The non-central distribution t with non-centrality parameter $\delta$ and degrees of freedom $\nu$ is the distribution of

\begin{displaymath}\frac{Z}{\sqrt{U/\nu}}
\end{displaymath}

where Z is $N(\delta,1)$, U is $\chi^2_\nu$ and Z and U are independent When $\delta=0$ we get the usual, or central t distribution.

Fact: If a is a vector of length p and a0 is some scalar then

\begin{displaymath}\frac{a^T\hat\beta-a_0}{\sqrt{MSE}\sqrt{a^T (X^TX)^{-1}a}}
\end{displaymath}

has a non-central t distribution with non-centrality parameter

\begin{displaymath}\delta =\frac{a^T\beta-a_0}{\sigma\sqrt{a^T(X^TX)^{-1}a}} \, .
\end{displaymath}

Power of two sided tests from table B 5. Normally computed before experiment based on assumptions about a0, $\sigma$and XTX.

Sample Size determination

Before an experiment is run it is sensible, if the experiment is costly, to try to work out whether or not it is worth doing. You will only do an experiment if the probability of Type I and II errors are both reasonably low. The simplest case arises when you prespecify a level, say $\alpha=0.05$ and an acceptable probability of Type II error, say 0.10, for testing a null hypothesis like $a^T\beta=a_0$. Then you need to specify

F tests

Simplest example: regression through the origin (no intercept term.)

Suppose now that the null hypothesis is false.

FACT:

If W is a $MVN(\tau,I)$ random vector and Q is idempotent with rank p then WTQW has a non-central $\chi^2$ distribution with non-centrality parameter

\begin{displaymath}\delta^2 = E(W^TQW)- p = \tau^T Q \tau
\end{displaymath}

and p degrees of freedom. This is the same distribution as that of

\begin{displaymath}(Z_1+\delta)^2 + Z_2^2 + \cdots + Z_p^2
\end{displaymath}

where the Zi are iid standard normals. An ordinary $\chi^2$ variable is called central and has $\delta=0$.

FACT:

If U and V are independent $\chi^2$ variables with degrees of freedom $\nu_1$ and $\nu_2$, V is central and U is non-central with non-centrality parameter $\delta^2$ then

\begin{displaymath}\frac{U/\nu_1}{V/\nu_2}
\end{displaymath}

is said to have a non-central F distribution with non-centrality parameter $\delta^2$ and degrees of freedom $\nu_1$ and $\nu_2$.

POWER CALCULATIONS

SAMPLE SIZE CALCULATIONS

Examples

POWER of t test: SAND and FIBRE example. See Lecture 11

Consider fitting the model

\begin{displaymath}Y_i = \beta_0 + \beta_1 S_i + \beta_2 F_i + \beta_3 F_i^2 +\epsilon_i
\end{displaymath}

Compute power of t test of $\beta_3=0$ for the alternative $\beta_3=-0.004$. (This is roughly the fitted value. In practice, however, this value needs to be specified before collecting data so you just have to guess or use experience with previous related data sets or work out a value which would make a difference big enough to matter compared to the straight line.)

Need to assume a value for $\sigma$. I take 2.5 - a nice round number near the fitted value. Again, in practice, you will have to make this number up in some reasonable way.

Finally at=(0,0,0,1) and aT(XTX)-1a has to be computed. For the design actually used this is $6.4\times 10^{-7}$. Now $\delta$ is 2. The power of a two-sided t test at level 0.05 and with 18-4=14 degrees of freedom is 0.46 (from table B 5 page 1346).

Take notice that you need to specify $\alpha$, $\beta_3/\sigma$ (or even $\beta_3$ and $\sigma$) and the design!

SAMPLE SIZE NEEDED using t test: SAND and FIBRE example.

Now for the same assumed values of the parameters how many replicates of the basic design (using 9 combinations of sand and fibre contents) would I need to get a power of 0.95? The matrix XTX for m replicates of the design actually used is m times the same matrix for 1 replicate. This means that aT(XTX)-1a will be 1/m times the same quantity for 1 replicate. Thus the value of $\delta$ for m replicates will be $\sqrt{m}$ times the value for our design, which was 2. With m replicates the degrees of freedom for the t-test will be 18m-4. We now need to find a value of m so that in the row in Table B 5 across from 18m-4degrees of freedom and the column corresponding to

\begin{displaymath}\delta = 2\sqrt{m}
\end{displaymath}

we find 0.95. To simplify we try just assuming that the solution mis quite large and use the last line of the table. We get $\delta$between 3 and 4 - say about 3.75. Now set $2\sqrt{m}=3.7$ and solve to find m=3.42 which would have to be rounded to 4 meaning a total sample size of $4\times 18 = 72$. For this value of m the non-centrality parameter is actually 4 (not the target of 3.75 because of rounding) and the power is 0.98. Notice that for this value of m the degrees of freedom for error is 66 which is so far down the table that the powers are not much different from the $\infty$ line.

POWER of F test: SAND and FIBRE example.

Now consider the power of the test that all the higher order terms are 0 in the model

\begin{displaymath}Y_i = \beta_0 + \beta_1 S_i + \beta_2 F_i + \beta_3 F_i^2
+\beta_4 S_i^2 + \beta_5 S_i F_i +\epsilon_i
\end{displaymath}

that is the power of the F test of $\beta_3=\beta_4=\beta_5=0$.

You will need to specify the non-centrality parameter for this Ftest. In general the noncentrality parameter for a F test based on $\nu_1$ numerator degrees of freedom is given by

\begin{displaymath}E(\mbox{Extra SS})/\sigma^2 - \nu_1 \,.
\end{displaymath}

This quantity needs to be worked out algebraically for each separate case, however, some general points can be made.


next up previous



Richard Lockhart
1999-04-09