The Power of Selective Memory. Shai ShalevShwartz Joint work with Ofer Dekel, Yoram Singer Hebrew University, Jerusalem. Outline. Online learning, loss bounds etc. Hypotheses space – PST Margin of prediction and hingeloss An online learning algorithm Trading margin for depth of the PST
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
The Power of Selective Memory
Shai ShalevShwartz
Joint work with
Ofer Dekel, Yoram Singer
Hebrew University, Jerusalem
For any fixed hypothesis h :
Each hypothesis is parameterized by a triplet:
context function
0
3
1
1
4
2
7
y = ?
y =
0
y = ?
y = +
0
y = ??
y = +
0
y = ??
y = +
0
+
.23
y = ???
y = +
0
+
.23
y = ???
y = ++
0
+

.23
.23
+
.16
y = ???
y = ++
0
+

.23
.23
+
.16
y = ???
y = ++
0
+

.23
.42
+

.16
.14
+
.09
y = ???+
y = ++
0
+

.23
.42
+

.16
.14
+
.09
y = ???+
y = +++
0
+

.41
.42
+

.29
.14

+
.09
.09
+
.06
Where does the lower bound come from?
y = ++++
y = ++++
0
+

1.41
1.41
Problem: The tree we learned is much more deeper !
Lets force gt+1 to be sparse by “canceling” the new coordinate
Now we can show that:
More specifically:
Performance of Algo. II
y = +  +  +  +  …
Only 3 mistakes
The last PST is of depth 5
The margin is 0.61 (after normalization)
The margin of the max margin tree (of infinite depth) is 0.7071
0

+
.55
.55
+

. 22
.39

+
.07
.07
+

.05
.05

.03
Future work