Hypothesis Testing: Type I and Type II Errors, P-Values, and the Student-t Distribution: Practice Problems and Solutions

A Type I error occurs when one rejects the null hypothesis H₀, even though H₀ is true. Thus, the probability of a Type I error occurring is Pr(Type I error) = Pr(reject H₀│ H₀ is true).

The level of significance of a hypothesis test is precisely the probability of committing a Type I error. The symbol α is often used to denote the level of significance.

A Type II error occurs when one fails to reject the null hypothesis H₀, even though H₀ is false. Thus, the probability of a Type I error occurring is β =Pr(Type II error) = Pr(Do not reject H₀│ H₀ is false).

The complement of β, 1- β, is the probability of rejection of H₀ when H₁ is true. This value is known as the power of the test. It is often intuitively easier to find 1- β first and then determine β from it.

According to Larsen and Marx, "The p-value associated with an observed test statistic is the probability of getting a value for that test statistic as extreme or more extreme than what was actually observed (relative to H₁) given that H₀ is true" (437). If the p-value of a given test is less than or equal to α, then we can reject the null hypothesis at the significance level of α. Another way to define the p-value is as "the smallest α at which we can reject H₀" (437).

Now we will discuss the Student t distribution.

Following Larsen and Marx, "Let Z be a standard normal random variable and let U be a chi square random variable independent of Z with n degrees of freedom. The Student t ratio with n degrees of freedom is denoted T_n, where T_n = Z/√(U/n)" (476).

In this section, we will be focusing on hypothesis testing using the Student t distribution and more particularly, the following property.

Following Larsen and Marx, "Let Y₁, Y₂, ..., Y_n be a random sample from a normal distribution with mean µ and standard deviation σ. Then

T_n-1 = (Ŷ - µ)/(S/√(n)) has a Student t distribution with n - 1 degrees of freedom." (Ordinarily, when circumstances permit it, the "Y" has a straight line over the top.)

Reminder: Here, S is the sample standard deviation, and σ is the standard deviation of the given normal distribution. These may or may not be the same!

In any particular test involving the Student t distribution, you will have some random sample of size n, drawn from a normal distribution. You can find the test statistic t, where

t = (µ_actual - µ_hypothesized)/(S/√(n)). Here, µ_actual is the observed sample mean and µ_hypothesized is the mean value predicted via the null hypothesis H₀.

Once you find the value of the test statistic t, you can find the p-value associated with the hypothesis test. The p-value of t is equal to Pr(|T| > t) or Pr(T > t) or Pr(T < t), depending on the nature of the null hypothesis.

If for some constant k, H₀: µ = k, then p = Pr(|T| > t).

If for some constant k, H₀: µ ≤ k, then p = Pr(T > t).

If for some constant k, H₀: µ ≥ k, then p = Pr(T < t).

The p-value of t can be found by examining the Table of Percentage Points of the t Distribution. Look in the row with the appropriate number of degrees of freedom (n - 1) and then find the value that corresponds to the observed test statistic t. Then look at the heading of the column to see which tail probabilities t corresponds to. If you seek to find Pr(|T| > t), then your desired p-value is the probability associated with two tails of the distribution. If you seek to find Pr(T > t), then your desired p-value is the probability associated with one tail of the distribution.

Source: Broverman, Sam. Actuarial Exam Solutions - CAS Exam 3 - Fall 2006.

Larsen, Richard J. and Morris L. Marx. An Introduction to Mathematical Statistics and Its Applications. Fourth Edition. Pearson Prentice Hall: 2006. pp. 436-437, 447-449, 476-478.

Original Problems and Solutions from The Actuary's Free Study Guide

Problem S3L56-1. Similar to Question 6 from the Casualty Actuarial Society's Fall 2006 Exam 3. You are using a 7% level of significance for a hypothesis test where H₀ states that the random variable X is uniformly distributed on the interval [5, 46], and H₁ states that the random variable X is uniformly distributed on the interval [36,46]. Your hypothesis test uses only a single observation. Find the probability of a Type II error associated with this test.

Solution S3L56-1. We are given the level of significance, which is the same as Pr(Type I error) = Pr(reject H₀│ H₀ is true) = 0.07. If H₀ is true, then X is uniformly distributed on the interval

[5, 46]. We note that H₁ states that X can assume values within the upper tail of the distribution of X according to H₀. If we are to reject H₀, should do so if the values of x we get are within the highest 7% of the values in the interval [5, 46]. The minimum value of x for which we reject H₀ is (46-5)(1-0.07) + 5 = 43.13.

Now we can calculate 1- β = Pr(Reject H₀│ H₁ is true). If H₁ is true, then X is uniformly distributed on [36,46]. Thus, 1- β = Pr(x ≥ 43.13│X is uniformly distributed on [36,46]) =

(46-43.13)/(46-36) = 0.287. Thus, our desired probability is β = 1 - 0.287 = β = 0.713.

Problem S3L56-2. Similar to part of Question 7 from the Casualty Actuarial Society's Fall 2006 Exam 3. You have collected a sample of 37 values from a normal distribution. The sample mean is µ_X = 5. You also know that _i=1³⁷∑(X_i - µ_X)/36 = 12.98644507.

You are testing the following two hypotheses.

H₀: µ = 4

H₁: µ ≠ 4

What is the test statistic t for this test?

Solution S3L56-2. Since we have a random sample drawn from a normal distribution, we need to use the test statistic t, which can be found via the following formula:

t = (µ_actual - µ_hypothesized)/(S/√(n)). We are given µ_actual = 5, µ_hypothesized = 4, and n = 37. The expression _i=1³⁷∑(X_i - µ_X)/36 is equivalent to the sample variance, or S². Thus, we know that S² = 12.98644507 and so S = 3.603532305.

Now we can find t = (5 - 4)/(3.603532305/√(37)) = t = 1.688.

Problem S3L56-3. Similar to part of Question 7 from the Casualty Actuarial Society's Fall 2006 Exam 3. You have collected a sample of 37 values from a normal distribution. The sample mean is µ_X = 5. You also know that _i=1³⁷∑(X_i - µ_X)/36 = 12.98644507.

You are testing the following two hypotheses.

H₀: µ = 4

H₁: µ ≠ 4

What is the p-value for this test? Use the Table of Percentage Points of the t Distribution.

Solution S3L56-3. In Solution S3L56-2, we found our test statistic t to be 1.688. The probability we seek to find as our p-value is Pr(|T| > t). Why do we use the absolute value of T rather than just T? We do this because the null hypothesis is a statement of equality with respect to the mean. If the actual mean gets too far from 4 in either direction, we will reject the null hypothesis. Thus, in our t-Distribution table, we will need to consider the two-tail probabilities.

We look in the row corresponding to 37-1 = 36 degrees of freedom. Conveniently enough (you knew this was going to happen), we find our t value of 1.688 in the column corresponding to a two-tail probability of 0.10. Thus, for this test, p = 0.10.

Problem S3L56-4. You have collected a sample of 20 values from a normal distribution. The sample mean is µ_X = 19. You also know that _i=1²⁰∑(X_i - µ_X)/19 = 18.26213422.

You are testing the following two hypotheses.

H₀: µ ≤ 17

H₁: µ > 17

What is the test statistic t for this test?

Solution S3L56-4. Since we have a random sample drawn from a normal distribution, we need to use the test statistic t, which can be found via the following formula:

t = (µ_actual - µ_hypothesized)/(S/√(n)). We are given µ_actual = 19, µ_hypothesized = 17, and n = 20. The expression _i=1²⁰∑(X_i - µ_X)/19 is equivalent to the sample variance, or S². Thus, we know that S² = 18.26213422 and so S = 4.273421839.

Now we can find t = (19 - 17)/(4.273421839/√(20)) = t = 2.093.

Problem S3L56-5. You have collected a sample of 20 values from a normal distribution. The sample mean is µ_X = 19. You also know that _i=1²⁰∑(X_i - µ_X)/19 = 18.26213422.

You are testing the following two hypotheses.

H₀: µ ≤ 17

H₁: µ > 17

What is the p-value for this test? Use the Table of Percentage Points of the t Distribution.

Solution S3L56-5. In Solution S3L56-4, we found our test statistic t to be 2.093. The probability we seek to find as our p-value is Pr(T > t). Why do we use just T instead of the absolute value of T? We do this because the null hypothesis is a statement of inequality with respect to the mean. We only reject the null hypothesis if our observed sample mean becomes too large.

Thus, in our t-Distribution table, we will need to consider the one-tail probabilities.

We look in the row corresponding to 20-1 = 19 degrees of freedom. Conveniently enough (you knew this was going to happen), we find our t value of 2.093 in the column corresponding to a one-tail probability of 0.025. Thus, for this test, p = 0.025.

See other sections of The Actuary's Free Study Guide for Exam 3L.