# Exercises in EM, Expectation Maximization Algorithm

## Flury, Bernard and Zoppe

I am going to explain how to obtain equation (2) of the paper “Exercises in EM” by:

Flury, Bernard and Zoppe, Alice. Exercises in EM. The American Statistician, Vol. 54, №3, August 2000.

In equation (2) of the paper, we see the **Expected value** for two cases (two categories of lamps (light bulbs) ) are obtained as:

## A brief explanation about the notation of exponential distribution:

We know exponential distribution as the :

*exp*(-λ*t*)

Where

`λ`

is the rate or the probability of “failing/expiring” lamps over the unit of time, and

*exp*(- λ*t*)

is the probability of a lamp to be alive until time *t**.*

So we can say the probability of expiring a lamp at time** t**

*,*when exactly expiration happens at

*t**,*is the multiplication of two probabilities, I mean:

`λ `*exp*(-λ*t*)

We can replace λ with **1/θ** and reach this new notation:

Let’s go back to the paper:

# First part of Eq. 2:

The first part of equation (2) says the **expectation** **value (lifetime) **of those lamps that are still alive after the moment ** t** becomes:

*t+**θ*

It is easy to guess. It is due to memoryless property of exponential distributions.

# Second part of Eq. 2:

I have explained this part using two methods.

# Method I:

For the case when the lamps are failed/expired before time ** t, **we have:

X_i is the time of the *i*th lamps. So the expectation of the X_i for this case becomes:

or:

Where the integrand itself is a conditional probability:

and we can write the integral in this way:

## Explaining the denominator:

We know the probability of a lamp to be alive until time ** t** is

*p *= *exp*(-*t*/θ)

Since these kinds of lamps are expired/failed before time ** t,** the probability becomes

`(1-`*p*)

## Explaining the Integrand:

The integrand is :

Where we can write for each part:

and therefore we obtain:

Now we have this new integral:

and finally we reach:

# Method II:

We consider total cases, I mean *E_i=0* and *E_i=1*, in other words I consider **both **categories of lamps, those which are expired before **and** those which are still alive after time *t*.

So the expectation value of ** X_i ,** over the whole range of time, becomes:

According to the law of total expectation, the left side of the above equation can be written as:

So we have:

We can substitute the values we already know, as:

and finally obtain the expectation of those lamps that are expired before time ** t **:

## Acknowledgement:

I would like to thank Prof. Anatoly Yambartsev for his valuable corrections and comments on this blog.