Residuals in Linear Regression and Assorted Exam-Style Questions

In least-squares linear regression, the ith residual is the difference between an observed value y_i when x = x_i and the value of y_i predicted by the least squares line. The greater the ith residual, the greater the failure of the least squares line to accurately model the location of the point

(x_i, y_i). The ith residuals for all values of i can be graphed to produce a residual plot.

Some of the problems in this section were designed to be similar to problems from past versions of the Casualty Actuarial Society's Exam 3L and the Society of Actuaries' Exam MLC. They use original exam questions as their inspiration - and the specific inspiration for each problem is cited so as to give students a chance to see the original. All of the original problems are publicly available, and students are encouraged to refer to them. But all of the values, names, conditions, and calculations in the problems here are the original work of Mr. Stolyarov.

Sources: Broverman, Sam. Actuarial Exam Solutions - CAS Exam 3L - Spring 2008.

Larsen, Richard J. and Morris L. Marx. An Introduction to Mathematical Statistics and Its Applications. Fourth Edition. Pearson Prentice Hall: 2006. pp. 650.

Original Problems and Solutions from The Actuary's Free Study Guide

Problem S3L61-1. Similar to Question 8 from the Casualty Actuarial Society's Spring 2008 Exam 3L. This question is also an excellent review of the concepts in Section 53.

You are taking a sample of 2 random values from an exponential distribution whose mean θ is equal to 700.

The ith order statistic of Y has the following probability density function (p. d. f.):
f_{Y'_i}(y) = (n!/((i -1)!(n - i)!))*F(y)^i-1*(1-F(y))^n-i*f(y), where f(y) is the p. d. f. of Y and F(y) is the cumulative distribution function (c. d. f.) of Y.

Find the mathematical expectation of the larger of the two values drawn from the sample.

Solution S3L61-1. We are asked to find E(Y'₂). We are given that Y follows an exponential distribution with mean 700. Thus, f(y) = (1/700)e^-y/700, F(y) = 1 - e^-y/700, and 1 - F(y) = e^-y/700. Here, 2 = 30 and i = 2, so
f_{Y'_2}(y) = (2!/[(1)!(0)!])*(1 - e^-y/700)¹*(e^-y/700)⁰*(1/700)e^-y/700 =

f_{Y'_2}(y) = (2/700)(1 - e^-y/700)e^-y/700

f_{Y'_2}(y) = (2/700)(e^-y/700 - e^-y/350)

f_{Y'_2}(y) = 2*(1/700)(e^-y/700) - (1/350)e^-y/350

We find E(Y'₂) = ₀^∞∫y(2*(1/700)(e^-y/700) - (1/350)e^-y/350)dy =

2*₀^∞∫(y/700)(e^-y/700)dy - ₀^∞∫(y/350)e^-y/350)dy. But ₀^∞∫(y/700)(e^-y/700)dy is 700 - the expected value for an exponential distribution with mean 700. Likewise, ₀^∞∫(y/350)e^-y/350)dy is 350 - the expected value for an exponential distribution with mean 350. So E(Y'₂) = 2*700 - 350 = E(Y'₂) = 1050.

Problem S3L61-2. Similar to part of Question 9 from the Casualty Actuarial Society's Spring 2008 Exam 3L. This question is also an excellent review of the concepts in Section 57.

You are given the following six points in a data sample.

(a): (6, 7)

(b): (18, 25)

(c): (34, 49)

(d): (45, 56)

(e): (67, 99)

(f): (100, 150)

You model this data with a line of the form y_i = a + bx_i, where a and b are found using the method of least-squares regression. Find the value of b.

Solution S3L61-2.

We use the formula b = (n*_i=1ⁿ∑(x_iy_i) - _i=1ⁿ∑(x_i) _i=1ⁿ∑(y_i))/(n*_i=1ⁿ∑(x_i²) - _i=1ⁿ∑(x_i)²).

Here, n = 6.

_i=1⁶∑(x_iy_i) = 6*7 + 18*25 + 34*49 + 45*56 + 67*99 + 100*150 = _i=1⁶∑(x_iy_i) = 26311.

_i=1ⁿ∑(x_i) = 6 + 18 + 34 + 45 + 67 + 100 = _i=1ⁿ∑(x_i) = 270.

_i=1ⁿ∑(y_i) = 7 + 25 + 49 + 56 + 99 + 150 = _i=1ⁿ∑(y_i) = 386

_i=1ⁿ∑(x_i²) = 6² + 18² + 34² + 45² + 67² + 100² = 18030

Thus, b = (6*26311 - 270*386)/(6*18030 - 270²) = b = about 1.520578231.

Problem S3L61-3. Similar to part of Question 9 from the Casualty Actuarial Society's Spring 2008 Exam 3L. This question is also an excellent review of the concepts in Section 57.

You are given the following six points in a data sample.

(a): (6, 7)

(b): (18, 25)

(c): (34, 49)

(d): (45, 56)

(e): (67, 99)

(f): (100, 150)

You model this data with a line of the form y_i = a + bx_i, where a and b are found using the method of least-squares regression. Find the value of a.

Solution S3L61-3. We use the formula a = (_i=1ⁿ∑(y_i) - b*_i=1ⁿ∑(x_i))/n, where, from Solution S3L61-2, we know that n = 6, b = 1.520578231, _i=1ⁿ∑(y_i) = 386, and _i=1ⁿ∑(x_i) = 270. Thus,

a = (386 - 1.520578231*270)/6 = a = about -4.092687075.

Problem S3L61-4. You are given the following six points in a data sample.

(a): (6, 7)

(b): (18, 25)

(c): (34, 49)

(d): (45, 56)

(e): (67, 99)

(f): (100, 150)

You model this data with a line of the form y_i = a + bx_i, where a and b are found using the method of least-squares regression. Find the residual at the point (67, 99).

Solution S3L61-4. The residual at point (67, 99) is the actual value of y minus the estimated value of y. The actual value of y is 99. The estimated value of y is a + b*67, where, from Solutions S3L61-2 and S3L61-3, we know that b = 1.520578231 and a = -4.092687075.

Thus, the estimated value of y is -4.092687075 + 1.520578231*67 = 97.78605442, and our desired residual is 99 - 97.78605442 = about 1.213945578.

Problem S3L61-5. Similar to Question 10 from the Casualty Actuarial Society's Spring 2008 Exam 3L. This question is also an excellent review of the concepts in Section 50. The number of rabid wild ostrich attacks near a remote desert village follows a Poisson distribution with a mean of 9 per day on Mondays, Tuesdays, and Thursdays, a mean of 5 per day on Wednesdays and Saturdays, and a mean of 7 per day on Fridays and Sundays. Find the probability that on a given week, there will be exactly 53 rabid wild ostrich attacks.

Solution S3L61-5. The mean of the sums of homogeneous Poisson random variables is the sum of their means. Thus, the mean for the number of rabid wild ostrich attacks in a week is 9*3 + 5*2 + 7*2 = 51. We recall the formula Pr[M = k] = e^-ΛΛ^k/k!. Here, k = 53 and Λ = 51. Thus, Pr[M = 53] = e^-5151⁵³/53! = about 0.0526352263.

See other sections of The Actuary's Free Study Guide for Exam 3L.