Monday, November 14, 2011

Linear Regression

This post will be another short Machine Learning lesson (or a set of materials to be more precise). Particularly, it will be about Linear Regression, which is a method of supervised machine learning (when there is a training set).

First of all, please watch these videos explaining the theoretical material: video 1, video 2 and video 3

Basically, it is a way to "predict" (or diagnose) a result from an input, after training the model with the training set.

The mathematics behind this is explained here and it is also known as the best fitting curve.

In order to consolidate this material (at least for myself :)), here is an easy exercise. Given the training set:

x:01234
y:367811
Find the hypothesis function ƒ(x)=w1⋅x+w0.

Using the formulas from the video material:


and the fact that M=5, we have:

w1=[5⋅(0⋅3+1⋅6+2⋅7+3⋅8+4⋅11)–(0+1+2+3+4)⋅(3+6+7+8+11)][5⋅(0+1+4+9+16)-( 0+1+2+3+4)2]=[5⋅88-350] ⁄ [5⋅30-100]=9 ⁄ 5

w0=(1 ⁄ 5)⋅(3+6+7+8+11)–(9 ⁄ 5)⋅(1 ⁄ 5)⋅(0+1+2+3+4) = 7 – 18 ⁄ 5 = 17 ⁄ 5

So ƒ(x)=1.8⋅x+3.4