Evaluating electronic textbooks: a methodology Ruth Wilson and Monica Landoni ECDL 2001 5 th September 2001 The Visual Book The WEB Book EBONI’s methodology for evaluating electronic textbooks Outline
Ruth Wilson and Monica Landoni
5th September 2001
The WEB Book
EBONI’s methodology for evaluating electronic textbooksOutline
Studied the application of the paper book metaphor to the design and production of electronic books
Found that users were able to rely on their experience with paper books to interact with the electronic book.The Visual Book
Focused on the impact of appearance on the usability of textbooks on the Web.
Comparison of two versions of a textbook chapter:
Original version: very plain, one “page” per chapter, basic use of hyperlinks.
Revised, “scannable” version: chapter divided between 10 “pages”, heavier use of hyperlinks, extra headings, indented lists, coloured keywords.
Revised version was 92% more usable.The WEB Book
EBONI aims to: textbooks on the Web.
Evaluate the different approaches to the design of electronic textbooks
Identify and report on the individual requirements of students and academics in learning and teaching using electronic textbooks
Compile and promote a set of best practice guidelines for the publication of electronic textbooksEBONI: aims
Peter K textbooks on the Web.aiser. The Joy of Visual Perception. http://www.yorku.ca/eye/thejoy.htm
Eric Chudler. Neuroscience for Kids. http://faculty.washington.edu/chudler/neurok.html
C. Robin Timmons and Leonard W. Hamilton. Drugs, Brains and Behaviour. http://www.rci.rutgers.edu/~lwh/drugs/Experiments:psychology Web books
Cliff McKnight, Andrew Dillon and John Richardson. textbooks on the Web.Hypertext in Context.
Original electronic version
Revised electronic versionExperiments:Hypertext in Context
Rosa Ainley (ed.). textbooks on the Web.New Frontiers of Space…
Adobe Acrobat eBook Reader
Britannica textbooks on the Web.
HP Jornada textbooks on the Web.
Selection of material textbooks on the Web.
Selection of participants
Selection of tasks
Scavenger hunt, memory tasks, high cognitive skill tasks
Selection of evaluation techniques
Questionnaires, observation, think-aloud, interviewsEbook evaluation model
Electronic books offer a diverse array of textbooks on the Web.
material for evaluation:
Hardware devices, e.g. REBs, eBookMan, goReader
Ebook reader software, e.g. Microsoft Reader, Adobe Acrobat Ebook Reader
“Web books”Selection of material (1)
EBONI aims to identify design techniques which maximise information intake by users, in order to provide guidance to creators of digitised educational material.
Texts can be compared according to three parameters:
MediumSelection of material (2)
Four main roles can be distinguished: information intake by users, in order to provide guidance to creators of digitised educational material.
Task assessorSelection of actors
Tasks are a way of bringing together participants and the test material in a structured manner. Three types are suggested:
“Scavenger Hunts”, or retrieval tasks
High cognitive skill tasksSelection of tasks
Qualitative feedback can be obtained via: test material in a structured manner. Three types are suggested:
Subjective satisfaction questionnaires
InterviewsSelection of evaluation techniques
Technique complexity test material in a structured manner. Three types are suggested:
High cognitive skill tasks
Sophisticated experiments can be “mapped” to simple experiments.
Results of all evaluations will be comparable at some level.
Results will feed into a set of best practice guidelines for producing electronic textbooks.Guidelines