Exam-Style Questions on Maximum Likelihood Estimation

When the data is grouped into intervals of the form (c_j-1, c_j], where c₀< c₁< ...< c_k are the boundaries of the intervals, and there are n_j observations in each interval (c_j-1, c_j], then the likelihood function is L(θ) = _j=1^kΠ(F(c_j│θ) - F(c_j-1│θ))^n_j. The corresponding loglikelihood function is l(θ) = _j=1^kΣ(n_j*ln(F(c_j│θ) - F(c_j-1│θ))).

If the data are truncated, they can be shifted by subtracting the point of truncation from each observation. Alternatively, one can simplify the data by assuming a lack of information for values below the truncation point. In that case, the denominator of each factor of the likelihood function becomes 1-F(d), where d is the truncation point for the observation in question.

The following properties are useful:

Theorem 57.1.

For any exponential distribution and any sample of any size n, both the maximum likelihood estimate and the estimate using the method of moments will be equal to the sample mean.

Theorem 57.2.

The maximum likelihood estimator is asymptotically unbiased: its bias approaches zero as the sample size increases without bound.

The variance of the maximum likelihood estimator is extremely difficult to calculate directly, but it can in some cases be approximated. The approximation is itself a difficult process, and so we will approach it in a piecewise fashion here.

The concept of information associated with a maximum likelihood estimator of a parameter θ is important in the approximation.

The following is the relationship between information and asymptotic variance of a maximum likelihood estimator. This relationship holds when there is a single parameter θ and when information is a function of the sample size.

Information = 1/Asymptotic variance = 1/Var(θ^)

Some of the problems in this section were designed to be similar to problems from past versions of Exam 4/C, offered jointly by the Casualty Actuarial Society and the Society of Actuaries. They use original exam questions as their inspiration - and the specific inspiration is cited to give students an opportunity to see the original. All of the original problems are publicly available, and students are encouraged to refer to them. But all of the values, names, conditions, and calculations in the problems here are the original work of Mr. Stolyarov.

Source:
Exam C Sample Questions and Solutions from the Society of Actuaries.

"Maximum Likelihood." Wikipedia, the Free Encyclopedia.

Loss Models: From Data to Decisions, (Third Edition), 2008, by Klugman, S.A., Panjer, H.H. and Willmot, G.E., Chapter 15, pp. 384-388.

Original Problems and Solutions from The Actuary's Free Study Guide

Problem S4C57-1. Similar to Question 14 of the Exam C Sample Questions from the Society of Actuaries. The informationassociated with a maximum likelihood estimator of a parameter θ is 14n, where n is the number of observations. Find the asymptotic variance of the maximum likelihood estimator of 95θ.

Solution S4C57-1. We use the formula Information = 1/Var(θ^) → Var(θ^) = 1/Information → Var(θ^) = 1/(14n). We want to find Var(95*θ^) = 95²*Var(θ^) = 9025/(14n) =

Var(95*θ^) = 644.6428571/n.

Problem S4C57-2. Similar to Question 26 of the Exam C Sample Questions from the Society of Actuaries. There are three types of slugs: green, purple, and striped. You are aware that the lifetimes of green slugs follow an exponential distribution with parameter θ. The lifetimes of purple slugs follow an exponential distribution with parameter 4θ. The lifetimes of striped slugs follow an exponential distribution with parameter 16θ. You observe the following sample:

1 green slug with a lifetime of 39.

2 purple slugs with lifetimes of 45 and 165.

1 striped slug with a lifetime of 900.

Use the method of maximum likelihood estimation to find θ.

Solution S4C57-2. We use Theorem 57.1: For any exponential distribution and any sample of any size n, both the maximum likelihood estimate and the estimate using the method of moments will be equal to the sample mean.

However, we must be careful here, because we are trying to find the sample mean as pertains to θ. For purple slugs, the parameter is 4θ, so we must divide each observation by 4 to find the observation relevant to calculating the sample mean for θ. For striped slugs, the parameter is 16θ, so we must divide each observation by 16 to find the observation relevant to calculating the sample mean for θ.

Thus, the sample of which we will be taking the mean is 39, 45/4 = 11.25, 165/4 = 41.25, and 900/16 = 56.25. Hence, θ = (39+11.25+41.25+56.25)/4 = θ = 36.9375.

Problem S4C57-3. Similar to Question 34 of the Exam C Sample Questions from the Society of Actuaries. A negative binomial distribution has a value of r = 3. For a certain sample size n, the sample mean is 15. Find the value of β for this distribution using maximum likelihood estimation.

Relevant properties of negative binomial distributions: p(k) = (r(r-1)...(r-k+1)/k!)β^k/(1+β)^r+k.

Solution S4C57-3. We first find L(θ) = _j=1ⁿΠ((r(r-1)...(r-k_j+1)/k_j!)β^k_j/(1+β)^k_j) =

_j=1ⁿΠ((3(3-1)...(3-k_j+1)/k_j!)β^k_j/(1+β)^3+k_j).

We notice that this function is proportional to another function, which we call M(θ) = _j=1ⁿΠ(β^k_j/(1+β)^3+k_j).

We can take the logarithm of M(θ):

_{j =1}ⁿΣ(ln(β^k_j/(1+β)^3+k_j)) =

_{j =1}ⁿΣ(ln(β^k_j) - ln((1+β)^3+k_j)) =

_{j =1}ⁿΣ(k_j*ln(β) - (3+k_j)ln(1+β)).

To maximize this function, we can take its derivative and set it equal to zero:
(ln(M(θ))' = _{j =1}ⁿΣ(k_j/β- (3+k_j)/(1+β)) = 0 →

_{j =1}ⁿΣ(k_j/β) = _{j =1}ⁿΣ((3+k_j)/(1+β)) →

_{j =1}ⁿΣ(k_j*(1+β)) = _{j =1}ⁿΣ((3+k_j)* β) →

_{j =1}ⁿΣ(k_j*(1+β) - (3+k_j)* β) = 0 →

_{j =1}ⁿΣ(k_j* - 3* β) = 0 →

_{j =1}ⁿΣ(k_j) - 3nβ = 0 →

_{j =1}ⁿΣ(k_j) = 3nβ →

_{j =1}ⁿΣ(k_j)/n = 3β.

However, _{j =1}ⁿΣ(k_j)/n is the sample mean or 15, so 15 = 3β → β = 5.

Problem S4C57-4. Similar to Question 37 of the Exam C Sample Questions from the Society of Actuaries. For a Pareto distribution, it is known that the parameter θ = 30. The parameter α is estimated via the method of maximum likelihood by analyzing data from the following sample: 13, 25, 36, 40. Find the maximum likelihood estimate of α.

Relevant properties of Pareto distributions: f(x) = αθ^α/(x+θ)^α+1

Solution S4C57-4. We first find L(α) = (αθ^α/(13+θ)^α+1)(αθ^α/(25+θ)^α+1)(αθ^α/(36+θ)^α+1)(αθ^α/(40+θ)^α+1) =

α⁴*(30⁴)^α/(43*55*66*70)^α+1 = α⁴*(30⁴)^α/(10926300)^α+1 =

α⁴*0.0741330551^α/10926300.
Now we find L'(α) and set it equal to zero:

L'(α) = 4α³*0.0741330551^α/10926300 + ln(0.0741330551)*α⁴*0.0741330551^α/10926300 = 0

-ln(0.0741330551)*α⁴*0.0741330551^α/10926300 = 4α³*0.0741330551^α/10926300 →

-ln(0.0741330551)*α = 4 → α = 4/-ln(0.0741330551) = α = 1.537341787.

Problem S4C57-5. Similar to Question 44 of the Exam C Sample Questions from the Society of Actuaries. You are trying to estimate the parameter θ of an exponential distribution. You know the following: 10 values fall within the interval from 0 to 25, inclusive. 34 values fall within the interval (25,50]. 9 values are greater than 50. Find the maximum likelihood estimate of θ.

Solution S4C57-5. This is a case where data are grouped into intervals, so L(θ) = _j=1^kΠ(F(c_j│θ) - F(c_j-1│θ))^n_j. Therefore, we consider the cumulative distribution function at each endpoint of each interval:
F(0│θ) = 0; F(25│θ) = 1 - e^-25/θ; F(50│θ) = 1 - e^-50/θ; F(∞│θ) = 1.

Thus, our likelihood function will have 10 factors of (F(25│θ)- F(0│θ)) = (1 - e^-25/θ).

Our likelihood function will have 34 factors of (F(50│θ)- F(25│θ)) = (e^-25/θ- e^-50/θ).

Our likelihood function will have 9 factors of (F(∞│θ)- F(50│θ)) = (e^-50/θ).

Hence, L(θ) = (1 - e^-25/θ)¹⁰(e^-25/θ- e^-50/θ)³⁴(e^-50/θ)⁹.

It is cumbersome to directly maximize this function in terms of θ. However, maximization will be easier if we let e^-25/θ = p. Then, L(p) = (1-p)¹⁰(p-p²)³⁴(p²)⁹ = (p⁵²)(1-p)⁴⁴.

Then ln(L(p)) = l(p) = ln((p⁵²)(1-p)⁴⁴) = ln(p⁵²) + ln((1-p)⁴⁴) = 52ln(p) + 44ln(1-p).

Thus, l'(p) = 52/p - 44/(1-p) = 0 at its maximum, so 52/p = 44/(1-p) and 52(1-p) = 44p → 52 = 96p and p = 52/96. Thus, e^-25/θ = 52/96 → -25/ln(52/96) = θ = 40.77608484.

See other sections of The Actuary's Free Study Guide for Exam 4 / Exam C.