The Power of Selective Memory. Shai ShalevShwartz Joint work with Ofer Dekel, Yoram Singer Hebrew University, Jerusalem. Outline. Online learning, loss bounds etc. Hypotheses space – PST Margin of prediction and hingeloss An online learning algorithm Trading margin for depth of the PST
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Shai ShalevShwartz
Joint work with
Ofer Dekel, Yoram Singer
Hebrew University, Jerusalem
For any fixed hypothesis h :
Where does the lower bound come from?
y = +  +  +  + 
y = +  +  +  + 
0
+

1.41
1.41
Problem: The tree we learned is much more deeper !
Lets force gt+1 to be sparse by “canceling” the new coordinate
Now we can show that:
More specifically:
y = +  +  +  +  …
Only 3 mistakes
The last PST is of depth 5
The margin is 0.61 (after normalization)
The margin of the max margin tree (of infinite depth) is 0.7071
Example revisited0

+
.55
.55
+

. 22
.39

+
.07
.07
+

.05
.05

.03
Future work