4
$\begingroup$

Assume the data to be i.i.d. random variables $X_1, X_2, \dots, X_n$ with density $$f(x,p) = p e^{-x} + 2(1-p)e^{-2x}, \quad x \ge 0.$$ We need to find the procedure of determining MLE of $p$ based on given data $x_1, \dots, x_n$.

The log-likelihood $$\ell(p) = \sum_{i=1}^n \ln \{p e^{-x_i} + 2(1-p)e^{-2x_i}\}.$$ Then for MLE, $$0 = \ell'(p) = \sum_{i=1}^n \frac{e^{-x_i} - 2e^{-2x_i}}{pe^{-x_i} + 2(1-p)e^{-2x_i}}.$$

From that, can we find MLE $\hat p$ without using numerical methods? I guess we can find it using bisection method on $p \in [0,1],$ but is it really not possible to find it by means of analyzing the data $X_1, \ldots, X_n,$ for instance? Since we are not given what is $p$ in the original problem, I guess we also need to determine the possible domain of $p$ so that MLE $\hat p$ exists.

Improve this question
$\endgroup$
7
  • $\begingroup$ Are you asking if there's a 'closed form' solution? (for some definition of what counts as being in closed form for your purposes) $\endgroup$ Commented 9 hours ago
  • $\begingroup$ What domain of p are you using here? $\endgroup$ Commented 9 hours ago
  • $\begingroup$ @Glen_b I guess there's no closed form. I actually wanted to do it without depending on some long iterative process. For instance, often using some properties of the data like minimum/maximum of $x_i$ helps. For the domain of $p,$ as I said, it's not given. I guess the domain of $p$ needs to be such that so that $\ell(p)$ is differentiable (or it's a singleton, which was not meant by the problem I guess). $\endgroup$ Commented 8 hours ago
  • $\begingroup$ I would hope that it's $ 0\leq p \leq 1$ but it's best to be clear. I would necessarily assume that differentiation (or even logs) is ideal. Maybe it is here but don't automatically assume it for every mle problem. $\endgroup$ Commented 8 hours ago
  • $\begingroup$ @Glen_b what would be your response if $p$ lies on the closed interval $[0, 1]$? $\endgroup$ Commented 8 hours ago

1 Answer 1

Reset to default
2
$\begingroup$

I dont think there is any trick that will give you a shortcut to the MLE in general for this problem. I am happy to be corrected on that, though.

For $n=2$ you can get a nice formula, and that should extend to formulas for $n=3$ or $4$ (perhaps with some checking of cases) but the approach I see to do these would leave us trying to solve a quintic at $n=5$ so unless the problem has special structure I don't see, that wouldn't work for larger $n$ than $4$.

In general with mixtures (this is a one-parameter mixture) you don't have some fancy shortcut, and it's often ill-conditioned $-$ at least in parts of the parameter space $-$ to boot, often making numerical optimization tricky, particularly if youre also estimating parameters of the components (indeed, for some parts of the space the model is degenerate).

In this case, we have mixtures of exponentials, which are sometimes difficult to fit. You may occasionally find the likelihood function is not that nice (e.g. pretty flat near the peak) even with plenty of data.

Fortunately in this case, with two components and just the mixing parameter to estimate, its relatively nice and numerical approaches should work in most situations. You can even plot $\log L$ vs $p$ for any given data, if you like (and I'd encourage that if you have data to work with).

Here's the log-likelihood function for 3 different (made up) small data sets, which have $\hat{p}=$ $0$, $1$, and $\sim 0.18$ respectively.

plots of three log-likelihood functions vs the parameter, p. All are concave over (0,1). The first is monotonic decreasing, with argmax at 0, the second is monotonic increasing, with argmax at 1, and the third is approximately quadratic with peak near 0.18

If you use R, the function optimize (built in, it's in stats ... thanks Thomas for correcting my mistake) should be adequate for optimizing the log-likelihood for most data sets on this particular problem. Even so, I'd still suggest plotting it.

A warning: take care with trying to apply the standard asymptotic inference if the parameter estimate hits the bounds.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I'd use stats::optimise for 1-d optimisation rather than stats::uniroot, since it will handle easily the two examples, where there isn't a solution to the likelihood equations. Also, while the MLE doesn't seem to have closed form there are consistent estimators that do, such as one based on $E[X]=p+(1-p)/2$ $\endgroup$ Commented 3 hours ago
  • $\begingroup$ Ah, yes, the optimizer is the one, not the rootfinder. Only half awake there $\endgroup$ Commented 2 hours ago

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.