1.13 Machine learning lecture 14  (Page 13/13)

Instructor (Andrew Ng) :And so what we’ll do is we’ll use a low dimensional subspace on which the data lies, and in practice a PCA dimension of 50 would be fairly typical. So when you think of – there’s some axes that really measure the shape or the appearance of the face, and there are some other axes that we’re not interested in that are maybe just random noise.

So what we’ll do is for face recognition I might give you a picture of a face and ask you what face looks the most similar to this. I’ll give you a picture of someone and ask you can you find other pictures of the same person. And so the key step to do that is to look at two faces and to compare how similar these two faces are. So here’s how we’ll use PCA. Given a face there and a different face there, the way I’ll measure the difference between these two faces is I won’t just measure the Euclidian distance similarity. Instead, I’ll take the space and project it onto my 50-dimensional subspace – take this and project it onto my 50-dimensional subspace and measure the similarity between those two points on the subspace. And so when I do that, I may end up with a face here and a face here that look very far upon the original space, but when I project them onto the subspace, they may end up looking much more similar.

So what I want to show you a second on the laptop is given each of these training examples is a 10,000-dimensional vector, I can plot that as a grayscale image and I’d get like a picture of some person’s face. What I’ll also show you on the laptop will be plots of my eigenvectors. And so the eigenvectors also live in a 10,000-dimensional subspace. And I plot that.

You get some image that’s – and these images are called eigenfaces. And what PCA is doing is essentially using linear combinations of these eigenface images to approximate these XIs, and it’s the eigenfaces that span – so that these bases U1, U2 and so on that span hopefully the subspace along which my faces live. [Crosstalk]

Instructor (Andrew Ng) :So here’s a training set comprising aligned images like these, a much larger set than is shown here, and so when you run PCA, these are some of the plots of the UIs. Remember when I said plot the eigenvectors UI in the same way that it’s plotting the training examples, and so you end up with the eigenvectors being grayscale images like these, and what approximate images of faces with linear combinations of these. So I said it’s dangerous to look at individual eigenvectors and you really shouldn’t do that, but let me just do that a little bit anyway. So if you look at the first eigenvector, this corresponds roughly to whether the face is illuminated from the left or the right. So depending on how heavy the weight is on this image, that roughly captures variation in illumination. The second image is hard to tell. It seems to be capturing variation in overall brightness of the face. The third eigenvector capturing roughly how much of a shadow or maybe a beard or something a person has and so forth. It’s dangerous to look at individual eigenvectors, but it sort of slightly informs the look of it.

Here’s an example of a specific application of eigenfaces. This is from Sandy Pentland – Alex Pentland’s lab at MIT. In this display, this upper leftmost image is the input to the eigenface’s algorithm. The algorithm is then asked to go through a database and find the most similar faces. This second image here is what it thinks is the most similar face of the input. The next one over is the second most similar. The next one over is the third most similar and so on. And so using eigenfaces, and by measuring differences between faces in that lower dimensional subspace, it is able to do a reasonable job identifying pictures of the same person even with faces of the person removed. And the next row shows the next most similar faces and so on, so this one is the fourth most similar face and so on. This is a usual application of eigenfaces.

The last thing I wanna say is just when people tell me about machine learning problems, I do often recommend they try PCA for different things, so PCA is a useful thing to know to use for compression, visualization, and so on. But in industry, I just tend to see PCA used slightly more often. There are also some times where you see people using PCA when they really shouldn’t. So just one final piece of advice for use of PCA is before you use it, also think about whether you could just do it with the original training data XI without compressing it, since I’ve also definitely seen people compress the data when they really didn’t need to. But having said that, I also often advise people to use PCA for various problems and it often works. Okay. Sorry about running late. So let’s wrap up for today.

[End of Audio]

Duration: 82 minutes