Review of EARL 1.0 -- Evaluation and Report Language

1. Review of EARL 1.0-- Evaluation and Report Language Reference: Evaluation and Report Language 1.0 Schema Evaluation and Report Language 1.0 Guide Evaluation and Repair Tools Working Group Home page Prepared by: Qi Li

2. What is EARL • To express test results • To facilitate the exchange of test results • Between Web accessibility evaluation tools and other generic quality assurance • In a vendor neutral and platform independent format

3. Structure of EARL • Test Subject: what is to evaluate ( can be web content, like web pages, videos, or applets; it also can be software like user agents; or something else), • Test Criterion: the standard chose to evaluate your objects (it can be a specification, guidelines, test suite or other testable statements) • Test Result: the outcome of the test. • Assertor: to identify who or what ran the test.

4. Purpose of EARL • initially for Web accessibility evaluation tools, Web quality assurance and validation tools and authoring tools • Now for more general topics such as software • E.g.: web site evaluation, combining results, comparing results from different evaluation tools, or exchanging data with search engines

5. Status of EARL • working draft and not yet operational (In W3C organization, technical draft divides into working draft, candidate recommendation;proposed recommendation; recommendation ) • may be changed dramatically • Call for feedback on • about the sufficiency of the vocabulary the language provide to describe the test subjects, • about the compatibility or consistency of the vocabulary, • and other additional questions.

6. Other relevant document • Rather new topic • Apply to our group project: • to figure out the correct passage (or snippet) the users’ selection from the document according to our ground truth annotation by every subject