1.5 Machine learning lecture 6  (Page 10/15)

[Begin Video] You’re going to hear first what the network sounds like at the very beginning of the training, and it won’t sound like words, but it’ll sound like attempts that will get better and better with time. [Computer’s voice]The network takes the letters, say the phrase, “grandmother’s house,” and makes a random attempt at pronouncing it. [Computer’s voice] Grandmother’s house. The phonetic difference between the guess and the right pronunciation is sent back through the network. [Computer’s voice]Grandmother’s house. By adjusting the connection strengths after each attempt, the net slowly improves.

And, finally, after letting it train overnight, the next morning it sounds like this: Grandmother’s house, I’d like to go to my grandmother’s house. Well, because she gives us candy. Well, and we – NETtalk understands nothing about the language. It is simply associating letters with sounds. [End Video]

All right. So at the time this was done, I mean, this is an amazing piece of work. I should say today there are other text to speech systems that work better than what you just saw, and you’ll also appreciate getting candy from your grandmother’s house is a little bit less impressive than talking about the Dow Jones falling 15 points, and profit taking, whatever. So but I wanted to show that just because that was another cool, major landmark in the history of neural networks. Okay. So let’s switch back to the chalkboard, and what I want to do next is tell you about Support Vector Machines, okay?

That, sort of, wraps up our discussion on neural networks. So I started off talking about neural networks by motivating it as a way to get us to output non-linear classifiers, right? I don’t really approve of it. It turns out that you’d be able to come up with non-linear division boundaries using a neural network like what I drew on the chalkboard earlier.

Support Vector Machines will be another learning algorithm that will give us a way to come up with non-linear classifiers. There’s a very effective, off-the-shelf learning algorithm, but it turns out that in the discussion I’m gonna – in the progression and development I’m gonna pursue, I’m actually going to start off by describing yet another class of linear classifiers with linear division boundaries, and only be later, sort of, in probably the next lecture or the one after that, that we’ll then take the support vector machine idea and, sort of, do some clever things to it to make it work very well to generate non-linear division boundaries as well, okay? But we’ll actually start by talking about linear classifiers a little bit more.

And to do that, I want to convey two intuitions about classification. One is you think about logistic regression; we have this logistic function that was outputting the probability that Y equals one, and it crosses this line at zero. So when you run logistic regression, I want you to think of it as an algorithm that computes theta transpose X, and then it predicts one, right, if and only if, theta transpose X is greater than zero, right? IFF stands for if and only if. It means the same thing as a double implication, and it predicts zero, if and only if, theta transpose X is less than zero, okay?