Order Statistics, Likelihood Functions, and Maximum Likelihood Estimates: Practice Problems and Solutions

Following Larsen and Marx 2006, we let Y be a continuous random variable for which y'₁, y'₂, ..., y'_n are values of a random sample of size n such that y'₁< y'₂< ... < y'_n.

We can define the variable Y'_i, the ith order statistic of Y, as Y'_i = y'_i, where 1 ≤ i ≤ n.

The ith order statistic of Y has the following probability density function (p. d. f.):

f_{Y'_i}(y) = (n!/[(i -1)!(n - i)!])*F(y)^i-1*(1-F(y))^n-i*f(y), where f(y) is the p. d. f. of Y and F(y) is the cumulative distribution function (c. d. f.) of Y.

We can also find E[Y'_i] = ₀^∞∫y*f_{Y'_i}(y)*dy = ₀^∞∫s_{Y'_i}(y)*dy.

Then E[Y'_i] can be used to estimate some particular value Y_k of Y.

If this is the case, then the bias of the ith order statistic can be determined as follows:

Bias(Y'_i) = E[Y'_i] - Y_k.

The following integration rule offers a useful shortcut for problems involving integration by parts:

₀^∞∫tⁿe^-at*dt = n!/aⁿ⁺¹.

The following definition of the likelihood function is offered by Larsen and Marx.

"Let k₁, k₂,..., k_n be a random sample of size n from the discrete p. d. f. p_X(k; θ), where θ is an unknown parameter. The likelihood function, L(θ), is the product of the p. d. f. evaluated at the n k_is. That is, L(θ) = _i=1ⁿΠ p_X(k _i; θ)."

If the p. d. f. in question is equal to f_Y(y _i; θ) and is continuous for random sample y₁, y₂,..., y_n of size n, then L(θ) = _i=1ⁿΠ f_Y(y _i; θ).

Now we can define the maximum likelihood estimate, following Larsen and Marx.

"Let L(θ) = _i=1ⁿΠ p_X(k _i; θ)and L(θ) = _i=1ⁿΠ f_Y(y _i; θ) be the likelihood functions corresponding to random samples k₁, k₂,..., k_n and y₁, y₂,..., y_n drawn from the discrete p.d.f. p_X(k _i; θ) and the continuous p.d.f. f_Y(y _i; θ), respectively, where θ is an unknown parameter. In each case, let θ_e be a value of the parameter such that L(θ_e) ≥ L(θ) for all possible values of θ. Then θ_e is called a maximum likelihood estimate for θ."

Some of the problems in this section were designed to be similar to problems from past versions of the Casualty Actuarial Society's Exam 3L and the Society of Actuaries' Exam MLC. They use original exam questions as their inspiration - and the specific inspiration for each problem is cited so as to give students a chance to see the original. All of the original problems are publicly available, and students are encouraged to refer to them. But all of the values, names, conditions, and calculations in the problems here are the original work of Mr. Stolyarov.

Sources: Broverman, Sam. Actuarial Exam Solutions - CAS Exam 3 - Fall 2006.

Larsen, Richard J. and Morris L. Marx. An Introduction to Mathematical Statistics and Its Applications. Fourth Edition. Pearson Prentice Hall: 2006. pp. 241-243, 347.

Original Problems and Solutions from The Actuary's Free Study Guide

Problem S3L53-1.

Similar to part of Question 1 from the Casualty Actuarial Society's Fall 2006 Exam 3.

The 2^nd order statistic in a sample of 4 is used to estimate the 40^th percentile of an exponential distribution with mean θ. What is the actual 40^th percentile of this distribution, in terms of θ?

Solution S3L53-1. The 40^th percentile of an exponential distribution is the value y for which F(y) = 0.4 and s(y) = 1 - F(y) = 0.6. For an exponential distribution, s(y) = e^-y/θ.

Thus, for the 40^th percentile, 0.6 = e^-y/θ and ln(0.6) = -y/θ, so y = -ln(0.6)*θ = ln(5/3)θ.

So the actual 40^th percentile is ln(5/3)θ.

Problem S3L53-2.

Similar to part of Question 1 from the Casualty Actuarial Society's Fall 2006 Exam 3.

The 2^nd order statistic in a sample of 4 is used to estimate the 40^th percentile of an exponential distribution with mean θ. What is the estimated 40^th percentile of this distribution, in terms of θ, given by using the 2^nd order statistic?

Solution S3L53-2. We need to find E[Y'₂] = ₀^∞∫y*f_{Y'_2}(y)*dy.

We find f_{Y'_2}(y) using the formula f_{Y'_i}(y) = (n!/[(i -1)!(n - i)!])*F(y)^i-1*(1-F(y))^n-i*f(y).

Here, n = 4, i = 2, f(y) = (1/θ)e^-y/θ, and F(y) = 1 - e^-y/θ.

Thus,

f_{Y'_2}(y) = [4!/(1!*2!)](1 - e^-y/θ)(e^-y/θ)²*(1/θ)e^-y/θ

f_{Y'_2}(y) = 12*(1/θ)(1 - e^-y/θ)(e^-y/θ)³

f_{Y'_2}(y) = 12*(1/θ)(e^-3y/θ - e^-4y/θ)

Thus, E[Y'₂] = ₀^∞∫y*12*(1/θ)(e^-3y/θ - e^-4y/θ)dy

E[Y'₂] = (12/θ)₀^∞∫y*(e^-3y/θ - e^-4y/θ)dy

E[Y'₂] = (12/θ)(₀^∞∫y*e^-3y/θdy - ₀^∞∫y*e^-4y/θdy)

Now we can use the integration shortcut ₀^∞∫tⁿe^-at*dt = n!/aⁿ⁺¹to find

₀^∞∫y*e^-3y/θdy = 1/(3/θ)² and ₀^∞∫y*e^-4y/θdy = 1/(4/θ)².

Thus, E[Y'₂] = (12/θ)(1/(3/θ)² - 1/(4/θ)²)

E[Y'₂] = (12/θ)(θ²/9 - θ²/16) = 12θ(1/9 - 1/16) = E[Y'₂] = 7θ/12.

Problem S3L53-3.

Similar to part of Question 1 from the Casualty Actuarial Society's Fall 2006 Exam 3.

The 2^nd order statistic in a sample of 4 is used to estimate the 40^th percentile of an exponential distribution with mean θ. What is the bias resulting from estimating the 40^th percentile of this distribution by using the 2^nd order statistic? Give your answer in terms of θ.

Solution S3L53-3. We use the formula Bias(Y'_i) = E[Y'_i] - Y_k.

From Solutions S3L53-1 and S3L53-2, we know that E[Y'₂] = 7θ/12 and Y_k = ln(5/3)θ.

Thus, Bias(Y'₂) = 7θ/12 - ln(5/3)θ = about 0.0725077096θ.

Problem S3L53-4.

Similar to part of Question 2 from the Casualty Actuarial Society's Fall 2006 Exam 3.

The amount of time in hours it takes Hernando to write a property deed follows the cumulative distribution function F(x) = x^θ+5, where 0 ≤ x ≤ 1 and θ > -5 is an unknown parameter. You are aware of the following sample of six times during each of which Hernando completed one property deed:

0.24, 0.643, 0.46, 0.45, 0.34, 0.36

Find the likelihood function L(θ) based on this sample.

Solution S3L53-4. We use the formula L(θ) = _i=1ⁿΠ f_Y(y _i; θ).

We find f_X(x) = F'(x) = (θ +5)x^θ+5.

Thus,

L(θ) = (θ +5)0.24^θ+5*(θ +5)0.643^θ+5*(θ +5)0.46^θ+5*(θ +5)0.45^θ+5*(θ +5)0.34^θ+5* (θ +5)0.36^θ+5

L(θ) = (θ +5)⁶(0.24*0.643*0.46*0.45*0.34*0.36)^θ+5

L(θ) = (θ +5)⁶(0.003909975)^θ+5.

Problem S3L53-5. Similar to part of Question 2 from the Casualty Actuarial Society's Fall 2006 Exam 3.

The amount of time in hours it takes Hernando to write a property deed follows the cumulative distribution function F(x) = x^θ+5, where 0 ≤ x ≤ 1 and θ > -5 is an unknown parameter. You are aware of the following sample of six times during each of which Hernando completed one property deed:

0.24, 0.643, 0.46, 0.45, 0.34, 0.36

Find the maximum likelihood estimate for θ using this sample.

Solution S3L53-5. We know from Solution S3L53-4 that L(θ) = (θ +5)⁶(0.003909975)^θ+5.

The maximum of L(θ) will occur where L'(θ) = 0.

We find L'(θ) = 6(θ +5)⁵(0.003909975)^θ+5 + (θ +5)⁶*ln(0.003909975)(0.003909975)^θ+5

We set 0 = 6(θ +5)⁵(0.003909975)^θ+5 + (θ +5)⁶*ln(0.003909975)(0.003909975)^θ+5 and find θ.

6(θ +5)⁵(0.003909975)^θ+5 = -(θ +5)⁶*ln(0.003909975)(0.003909975)^θ+5

6(θ +5)⁵= -(θ +5)⁶*ln(0.003909975)

6 = -(θ +5)*ln(0.003909975)

θ +5 = 6/-ln(0.003909975)

θ = 6/-ln(0.003909975) - 5 = θ = about -3.917792703.

See other sections of The Actuary's Free Study Guide for Exam 3L.