It looks like the equality is not true in general except the case where or . Try the case and then you will see the contradiction.
Wikipedia: “However, the logistic loss function does not assign zero penalty to any points. Instead, functions that correctly classify points with high confidence (i.e., with high values of ) are penalized less.”
“but also due to examples that have near zero”: To be more specific, if and has the same sign and is near zero then the loss will be also very large. Sep 23, 2016: I wonder if this note is correct?
Page 462:
“i.e., the example is close to the decision boundary”: Notice that we have: (same goes for ). Also remember that a point is on the boundary if and only if .
Hinge Lost. Solver.com: “A non-convex optimization problem is any problem where the objective or any of the constraints are non-convex, as pictured below. Such a problem may have multiple feasible regions and multiple locally optimal points within each region. It can take time exponential in the number of variables and constraints to determine that a non-convex problem is infeasible, that the objective function is unbounded, or that an optimal solution is the “global optimum” across all feasible regions.” Bookmark: NIPS 2015 workshop on non-convex optimization, ResearchGate.