2.8 Machine learning lecture 8 course notes  (Page 2/3)

The last step of this derivation used Jensen's inequality. Specifically, $f\left(x\right)=logx$ is a concave function, since $f^{\text{'}\text{'}}\left(x\right)=-1/x^2<0$ over its domain $x\in {\mathbb{R}}^{+}$ . Also, the term

in the summation is just an expectation of the quantity $\left[p,(x^{\left(i\right)},z^{\left(i\right)};\theta ),/,Q_i,\left(z^{\left(i\right)}\right)\right]$ with respect to $z^{\left(i\right)}$ drawn according to the distribution given by $Q_i$ . By Jensen's inequality, we have

where the “ $z^{\left(i\right)}\sim Q_i$ ” subscripts above indicate that the expectations are with respect to $z^{\left(i\right)}$ drawn from $Q_i$ . This allowed us to go from  [link] to [link] .

Now, for any set of distributions $Q_i$ , the formula  [link] gives a lower-bound on $\ell \left(\theta \right)$ . There're many possible choices for the $Q_i$ 's. Which should we choose? Well, if we have some current guess $\theta$ of the parameters, it seems natural to try to make the lower-bound tight at that value of $\theta$ . I.e., we'll make the inequality above hold with equality at ourparticular value of $\theta$ . (We'll see later how this enables us to prove that $\ell \left(\theta \right)$ increases monotonically with successsive iterations of EM.)

To make the bound tight for a particular value of $\theta$ , we need for the step involving Jensen's inequality inour derivation above to hold with equality. For this to be true, we know it is sufficient that that the expectationbe taken over a “constant”-valued random variable. I.e., we require that

for some constant $c$ that does not depend on $z^{\left(i\right)}$ . This is easily accomplished by choosing

Actually, since we know ${\sum }_z{Q}_i\left(z^{\left(i\right)}\right)=1$ (because it is a distribution), this further tells us that

Thus, we simply set the $Q_i$ 's to be the posterior distribution of the $z^{\left(i\right)}$ 's given $x^{\left(i\right)}$ and the setting of the parameters $\theta$ .

Now, for this choice of the $Q_i$ 's, Equation  [link] gives a lower-bound on the loglikelihood $\ell$ that we're trying to maximize. This is the E-step. In the M-step of the algorithm, we then maximizeour formula in Equation  [link] with respect to the parameters to obtain a new setting of the $\theta$ 's. Repeatedly carrying out these two steps gives us the EM algorithm, which is as follows:

• Repeat until convergence $\{$
  • (E-step) For each $i$ , set
    $$Q_i\left(z^{\left(i\right)}\right):=p\left(z^{\left(i\right)}\right|x^{\left(i\right)};\theta ).$$
  • (M-step) Set
    $$\theta :=arg\underset{\theta }{max}\sum _i\sum _{z^{\left(i\right)}}{Q}_i\left(z^{\left(i\right)}\right)log\frac{p(x^{\left(i\right)},z^{\left(i\right)};\theta )}{Q_i\left(z^{\left(i\right)}\right)}.$$
• $\}$

How will we know if this algorithm will converge? Well, suppose ${\theta }^{\left(t\right)}$ and ${\theta }^{(t+1)}$ are the parameters from two successive iterations of EM. We will now prove that $\ell \left({\theta }^{\left(t\right)}\right)\le \ell \left({\theta }^{(t+1)}\right)$ , which shows EM always monotonically improves the log-likelihood.The key to showing this result lies in our choice of the $Q_i$ 's. Specifically, on the iteration of EM in which the parametershad started out as ${\theta }^{\left(t\right)}$ , we would have chosen $Q_i^{\left(t\right)}\left(z^{\left(i\right)}\right):=p\left(z^{\left(i\right)}\right|x^{\left(i\right)};{\theta }^{\left(t\right)})$ . We saw earlier that this choice ensures that Jensen's inequality,as applied to get  [link] , holds with equality, and hence

The parameters ${\theta }^{(t+1)}$ are then obtained by maximizing the right hand side of the equation above. Thus,