Learning From Data – A Short Course: Exercise 1.3
Page 8:
The weight update rule in (1.3) has the nice interpretation that it moves in the direction of classifying
correctly.
(a) Show that
. [Hint: x(t) is misclassified by
.]
Because is misclassified by
so:
:
. Hence:
.
:
. Hence:
.
So: .
(b) Show that
. [Hint: Use (1.3).]
My solution for (b) is wrong. Please see: http://problemania.org/wiki/index.php?title=%E3%80%8ALearning_from_Data_-_A_Short_Course%E3%80%8B_Exercise_1.3:_Iterative_perceptron_learning_algorithm (thanks Coleman).
is misclassified by
so
.
correcly classifies
so
(the argument is much alike above).
Hence: .
(c) As far as classifying
is concerned, argue that the move from
to
is a move “in the right direction”.
Check the slide 13 of Lecture 01. Remind yourself the formula: where
is the angle between
and
.






I think your argument for (b) is wrong. In details, “w(t + 1) is correctly classified” cannot be guaranteed.
http://problemania.org/wiki/index.php?title=%E3%80%8ALearning_from_Data_-_A_Short_Course%E3%80%8B_Exercise_1.3:_Iterative_perceptron_learning_algorithm
I think the above website lists the correct argument.
Thank you. You are correct. I misinterpreted the learning algorithm.