Least-Squares Regression and Assorted Exam-Style Questions

Here, we will discuss the method of least-squares regression. According to Larsen and Marx, "Given n points (x₁, y₁), (x₂, y₂), ..., (x_n, y_n), the straight line y = a + bx minimizing

L = _i=1ⁿ∑[y_i - (a + bx_i)]² has slope

b = [n*_i=1ⁿ∑(x_iy_i) - _i=1ⁿ∑(x_i) _i=1ⁿ∑(y_i)]/[n*_i=1ⁿ∑(x_i²) - _i=1ⁿ∑(x_i)²]

and y-intercept

a = [_i=1ⁿ∑(y_i) - b*_i=1ⁿ∑(x_i)]/n." (647-648)

Alternatively, b = _i=1ⁿ∑[(x_i - µ_x)(y_i - µ_y)]/_i=1ⁿ∑(x_i - µ_x)² and a = µ_y - b*µ_x, where µ_y is the mean of the values y₁ through y_n and µ_x is the mean of the values x₁ through x_n.

Source: Broverman, Sam. Actuarial Exam Solutions - CAS Exam 3 - Fall 2006.

Larsen, Richard J. and Morris L. Marx. An Introduction to Mathematical Statistics and Its Applications. Fourth Edition. Pearson Prentice Hall: 2006. pp. 647-648.

Original Problems and Solutions from The Actuary's Free Study Guide

Problem S3L57-1. Similar to Question 8 from the Casualty Actuarial Society's Fall 2006 Exam 3. The number y_i of complimentary ordinary widgets that you get with the purchase of x_i superwidgets, follows the relationship y_i = a + bx_i, where a and b are found using the method of least-squares regression. Today, you purchase 54 superwidgets and are given the following additional information. _i=1¹⁷∑[(x_i - µ_x)(y_i - µ_y)] = 450 and _i=1¹⁷∑(x_i - µ_x)² = 50. Moreover, the mean µ_x is equal to 60 and the mean µ_y is equal to 194. Find how many complimentary ordinary widgets you will be getting. Non-whole numbers of superwidgets are acceptable.

Solution S3L57-1. We first find b using the formula b = _i=1ⁿ∑[(x_i - µ_x)(y_i - µ_y)]/_i=1ⁿ∑(x_i - µ_x)² = 450/50 = b = 9. Now we find a = µ_y - b*µ_x = 194 - 9*60 = -346.

Then, for x_i = 54, we have y_i = a + bx_i = -346 + 9*54 = 140 ordinary widgets.

Problem S3L57-2. You are given the following points (4, 5), (6, 7), (10, 23). Using the method of least-squares regression, where y = a + bx, find b.

Solution S3L57-2. We use the formula

b = [n*_i=1ⁿ∑(x_iy_i) - _i=1ⁿ∑(x_i) _i=1ⁿ∑(y_i)]/[n*_i=1ⁿ∑(x_i²) - _i=1ⁿ∑(x_i)²], where n = 3.

We have _i=1ⁿ∑(x_iy_i) = 4*5 + 6*7 + 10*23 = 292.

Moreover, _i=1ⁿ∑(x_i) = 4 + 6 + 10 = 20 and _i=1ⁿ∑(y_i) = 5 + 7 + 23 = 35.

Moreover, _i=1ⁿ∑(x_i²) = 4² + 6² + 10² = 152 and _i=1ⁿ∑(x_i)² = 20² = 400.

Thus, b = (3*292 - 20*35)/(3*400 - 152) = b = about 0.1679389313.

Problem S3L57-3. You are given the following points of the form (x, y): (4, 5), (6, 7), (10, 23). Using the method of least-squares regression, where y = a + bx, find a.

Solution S3L57-3. We use the formula a = [_i=1ⁿ∑(y_i) - b*_i=1ⁿ∑(x_i)]/n, where n = 3 and b = 0.1679389313 (from Solution S3L57-2). We also know that

_i=1ⁿ∑(x_i) = 4 + 6 + 10 = 20 and _i=1ⁿ∑(y_i) = 5 + 7 + 23 = 35.

Thus, a = (35 - 20*0.1679389313)/3 = a = about 10.54707379.

Problem S3L57-4. Similar to Question 9 from the Casualty Actuarial Society's Fall 2006 Exam 3. This question is also an excellent review of the concepts in Section 53.

You are taking a sample of 30 random values from an exponential distribution whose mean θ is equal to 100.

The ith order statistic of Y has the following probability density function (p. d. f.):
f_{Y'_i}(y) = (n!/[(i -1)!(n - i)!])*F(y)^i-1*(1-F(y))^n-i*f(y), where f(y) is the p. d. f. of Y and F(y) is the cumulative distribution function (c. d. f.) of Y.

Find the probability that the second order statistic of Y is between 40 and 80. The answer will be very small!

Solution S3L57-4. We are given that Y follows an exponential distribution with mean 100. Thus, f(y) = 0.01e^-0.01y, F(y) = 1 - e^-0.01y, and 1 - F(y) = e^-0.01y. Here, n = 30 and i = 2, so

f_{Y'_2}(y) = (30!/[(1)!(28)!])*(1 - e^-0.01y)*( e^-0.01y)²⁸*0.01e^-0.01y =

f_{Y'_2}(y) = 8.7(1 - e^-0.01y)( e^-0.29y)

f_{Y'_2}(y) = 8.7(e^-0.29y - e^-0.3y)

Thus, Pr(40 < Y₂< 80) = ₄₀⁸⁰∫8.7(e^-0.29y - e^-0.3y)dy =

(8.7/-0.29)(e^-0.29y) - (8.7/-0.3)(e^-0.3y)│₄₀⁸⁰ =

[(8.7/-0.29)(e^-0.29*80) - (8.7/-0.3)(e^-0.3*80)] - [(8.7/-0.29)(e^-0.29*40) - (8.7/-0.3)(e^-0.3*40)] =

Pr(40 < Y₂< 80) = about 0.00009679904811.

Problem S3L57-5. Similar to Question 5 from the Casualty Actuarial Society's Spring 2008 Exam 3L. This question is also an excellent review of the concepts in Section 56.

X is a normally distributed random variable, whose standard deviation σ is 2 and whose mean µ is either 50 or 55. You perform a hypothesis test between these two hypotheses:
H₀: σ = 2, µ = 50.

H₁: σ = 2, µ = 55.

You observe one random value x of X and will reject the null hypothesis if x > k, for some specific k. The probability of a Type I error is 0.1469. What is the probability of Type II error?

Use theTable of Values for the Standard Normal Distribution.

Solution S3L57-5. Pr(Type I error) = Pr(reject H₀│ H₀ is true) = Pr(x > k │ µ = 50) = 0.1469.

For the standard normal random variable Z and observations z of Z, we know that

Pr(z > k │ µ = 50) = 0.1469, so 1 - Φ(z) = 0.1469. so Φ(z) = 0.8531.

Using the Table of Values for the Standard Normal Distribution, we find that z = 1.05, which means that x = 50 + 1.05*2 = x = 52.1.

Thus, if x > 52.1, we accept hypothesis H₁.

Now we can find Pr(Type II error) = Pr(Do not reject H₀│ H₀ is false) = Pr(x < 52.1│µ = 55).

We find the z value corresponding to 52.1, if σ = 2 and µ = 55.

This value is z = (52.1 - 55)/2 = -1.45.

Thus, Pr(x < 52.1│µ = 55) = Φ(-1.45) = 1 - Φ(1.45) = 1 - 0.9265 = Pr(Type II error) =0.0735.

See other sections of The Actuary's Free Study Guide for Exam 3L.