MILESTONE RESULTS Mar. 1 st , 2007. Agnostic Learning vs. Prior Knowledge challenge Isabelle Guyon, Amir Saffari, Gideon Dror, Gavin Cawley, Olivier Guyon, and many other volunteers, see http://www.agnostic.inf.ethz.ch/credits.php. Thanks.
MILESTONE RESULTS Mar. 1st, 2007
Isabelle Guyon, Amir Saffari, Gideon Dror,
Gavin Cawley, Olivier Guyon,
and many other volunteers, see http://www.agnostic.inf.ethz.ch/credits.php
When everything else fails,
ask for additional domain knowledge…
ADA is the marketing database
GINA is the digit database
HIVA is the HIV database
Subject: Re: Goalie masksLines: 21Tom Barrasso wore a great mask, one time, last season. He unveiled it at a game in Boston. It was all black, with Pgh city scenes on it. The "Golden Triangle" graced the top, alongwith a steel mill on one side and the Civic Arena on the other. On the back of the helmet was the old Pens' logo the current (at the time) Penslogo, and a space for the "new" logo.A great mask done in by a goalie's superstition.Lori
NOVA is the text classification database
SYLVA is the ecology database
PROTOCOL and SCORING
dBER = abs(testBER – guessedBER)
Time (days)Learning curves
Time (months)Learning curves
The black vertical line indicates the best ranked entry (only the 5 last entry of each participant were ranked). Beware of overfitting!
Agnostic learning best ranked entries as of August 1st, 2007
Best ave. BER still held by Reference (Gavin Cawley) with “the bad”. Note that the best entry for each dataset is not necessarily the best entry overall.
Agnostic track no significant improvement so far
Gavin Cawley, July 2006
Roman Lutz, July 2006
Prior knowledge best ranked entries as of August 1st, 2007
Best ave. BER held by Reference (Gavin Cawley) with “interim all prior”.
Louis Duclos-Gosselin is second on ADA with Neural Network13, and S. Joshua Swamidass second on HIVA, but they are not entered in the table because he did not submit a complete entry.
The overall entry ranking is performed with the overall score (average rank over all datasets). The best performing complete entry may not contain all the best performing entries on the individual datasets.
We indicate the ranks of the “prior” entries only for individual datasets.
We compare the best results of the ranked entries for entrants who entered both tracks. If the Agnostic Learning BER is larger than the Prior Knowledge BER, “1” is shown in the table. The sign test is not powerful enough to reveals a significant advantage of PK or AL.