## [Notes] Learning From Data – A Short Course: e-Chapter 9

Page 30: Why ? My understanding:   as the events and are independent, the same goes for the event . Note that both data points and may exist in dataset so if is a deterministic hypothesis then it is obviously  as . If is a non-deterministic hypothesis,  is still independent from the random target function (even “ are [...]

## Learning From Data – A Short Course: Exercise 9.6

Try to build some intuition for what the rotation is doing by using the illustrations in Figure 9.1 to qualitatively answer these questions. (a) If there is a large offset (or bias) in both measured variables, how will this affect the ‘natural axes’, the ones to which the data will be rotated? Should you perform [...]

## Learning From Data – A Short Course: Exercise 9.4

Let and  be independent with zero mean and unit variance. You measure inputs and . (a) What are variance (), variance () and covariance ? First, we have:                 Now, we consider and get: Expected values:             Variance:           [...]

## Learning From Data – A Short Course: Exercise 8.15

Consider two finite-dimensional feature transforms and and their corresponding kernels and . (a) Define . Express the corresponding kernel of in terms of and .     (b) Consider the matrix and let be the vector representation of the matrix (say, by concatenating all the rows). Express the corresponding kernel of in terms of and [...]

## Learning From Data – A Short Course: Exercise 8.14

Suppose that we removed a data point with . (a) Show that the previous optimal solution remains feasible for the new dual problem (8.21) (after removing ). is the old dual problem while is the new dual problem.     Because  appears when appears and reverse. So when we replace the value of into (8.21), there is [...]

## [Notes] Learning From Data – A Short Course: e-Chapter 8

Page 29 “Then, at least one of the will be strictly positive.”. Please refer to the constraint .

## Learning From Data – A Short Course: Exercise 8.11

(a) Show that the problem in (8.21) is a standard QP-problem: where and ( for dual) are given by: It is easy to show this, what should be noted here is that:     (b) The matrix of quadratic coefficients is . Show that , where is the ‘signed data matrix’, Hence, show that is [...]

## Learning From Data – A Short Course: Exercise 8.9

Let be optimal for (8.10), and let be optimal for (8.11). (a) Show that . If is the optimal of (8.10) then it must satisfy the constraint , hence . If then it doesn’t matter what is, . If  then we can always choose , so . (b) Show that is feasible for (8.10). To show this, [...]

## Learning From Data – A Short Course: Problem 8.6

Show that is minimized at .             Now we need to check what type of the found critical point is:     So the found critical point is the global minimum.

## Learning From Data – A Short Course: Problem 8.7

For any with and even, show that there exists a balanced dichotomy that satisfies , and (This is the geometric lemma that is need to bound the VC-dimension of -fat hyperplanes by .)  The following steps are a guide for the proof. Suppose you randomly select of the labels  to be , the others being [...]