quick search it is not testing response times of traditional and nextgen catalogs n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
PowerPoint Presentation
Download Presentation

Loading in 2 Seconds...

play fullscreen
1 / 39

- PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on

“Quick Search” It Is Not: Testing Response Times of Traditional and NextGen Catalogs. Nina McHale Margaret Brown- Sica LITA Forum 2010. Esteemed Researchers. Our Research. Forthcoming:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about '' - cana


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
quick search it is not testing response times of traditional and nextgen catalogs

“Quick Search” It Is Not: Testing Response Times of Traditional and NextGen Catalogs

Nina McHale

Margaret Brown-Sica

LITA Forum 2010

our research
Our Research

Forthcoming:

Margaret Brown-Sica, Jeffrey Beall, and Nina McHale, “Next-Generation Library Catalogs and the Problem of Slow Response Time,” Information Technology and Libraries, Volume 29/4, December 2010, 207-216.

our research questions
Our Research Questions

Are NextGen catalogs—or traditional catalogs that add NextGen content—too slow?

Do 2.0/NextGen features slow them down too much?

our conclusions
Our Conclusions

Yup.

Features such as cover art, reviews, tagging, etc., can significantly increase the amount of data, and therefore time, required to return a catalog record page.

Performance factors, particularly speed, should be required criteria for librarians and vendors evaluating and designing products.

speed standards
Speed Standards?
  • W3C does not set forth standards
  • Jakob Nielsen
    • 0.1 of a second: feels “instantaneous”
    • 1.0 second: feels “uninterrupted”
    • 10 seconds:
      • “About the limit for keeping the user’s attention focused on the dialogue.”
      • Give the user “time-remaining” feedback

Jakob Nielsen, Usability Engineering (San Francisco: Morgan Kaufmann, 1994) 135.

our method
Our Method

During a busy time during the semester, we recorded response times in seconds of permalinks for three catalog records

Tested our classic/NextGen catalogs and three others

3 books, 5 catalogs, 3 times per day for 13 days=585 data points

Collecting several data points in this way using www.websitepulse.com ensured that data was consistent

additional catalogs tested
Additional Catalogs Tested
  • Library of Congress Catalog
    • Voyager
    • Traditional catalog
  • University of Texas at Austin
    • Innovative Interfaces
    • Traditional catalog with added NextGen elements
  • University of Southern California
    • Sirsi/Dynix
    • Traditional catalog with added NextGen elements
books used
Books Used

Hard Lessons: The Iraq Reconstruction Experience. Washington, DC: Special Inspector General, Iraq Reconstruction, 2009. (OCLC number 302189848)

Ehrenreich, Barbara. Nickel and Dimed: On (Not) Getting by in America. 1st ed. New York: Metropolitan Books, 2001. (OCLC number 256770509)  

Langley, Lester D. Simón Bolívar: Venezuelan Rebel, American Revolutionary. Lanham: Rowman & Littlefield Publishers, c2009. (OCLC number 256770509)  

permalink examples
Permalink Examples

http://aurarialibrary.worldcat.org/oclc/302189848

http://skyline.cudenver.edu/record=b2433301~S0

http://lccn.loc.gov/2009366172

http://catalog.lib.utexas.edu/record=b7195737~S29

http://library.usc.edu/uhtbin/cgisirsi/x/0/0/5?searchdata1=2770895{CKEY}

testing tools
Testing Tools
  • WebSite Pulse™
    • www.websitepulse.com
    • Allows testing on any web page/site; does not require server installation
  • Similar services:
    • websiteoptimization.com
    • browsermob.com
    • tools.pingdom.com
    • your favorite?
websitepulse results
WebSitePulse™ Results
  • Horizontal bar:
    • Gives visual representation of load time for each item (image files, javascript files, style sheets, etc.)
    • Provides quick indication of “sticking points”
  • Table:
    • provides specifics about file size and delivery time for each
    • DNS, Connect, Redirect, First Byte, Last Byte, Error
numbers crunched average response time in seconds
Numbers Crunched: Average Response Time in Seconds

Auraria’s Skyline: 1.2930

Auraria’s WCL: 11.5734

Library of Congress: 2.1530

University of Texas at Austin: 3.4997

University of Southern California: 4.1085

individual catalog test results
Individual Catalog Test Results

After data was analyzed, we took a closer look at each individual catalog, using the Hard Lessons catalog record

WebSitePulse™ allowed us to take a glimpse at the inner workings of each catalog

Findings confirmed that extra data and load times were from 2.0/NextGen content

skyline findings
Skyline Findings
  • Missing favicon (item 4)
  • 0.9172 seconds “uninterrupted” per Nielsen
  • 14 items, for a total of 84.64 K:
    • 9 GIFs
    • 2 CSS
    • 1 JavaScript
  • Good performance, but an interface that only a librarian could love
worldcat@auraria findings
WorldCat@Auraria Findings
  • Reference & Instruction librarians’ observations corroborated
  • 10.3615 seconds
  • 31 items, for a total of 633.09 K, to load:
    • 10 CSS files
    • 10 JavaScript files
    • 8 GIFs/PNGs
  • No single NextGen feature slowed down load time, but multitude of files created unacceptable delay
library of congress catalog findings
Library of Congress Catalog Findings
  • Overall, second fastest of all five catalogs tested
  • 1.2900 seconds
  • Only six items and 19.27 K to load:
    • 2 CSS files
    • 3 GIFs
  • Like Skyline, fast, but has that “legacy look”
university of texas at austin findings
University of Texas at Austin Findings
  • Added NextGen features:
    • Cover art
    • LibraryThing’s Catalog Enhancement
      • Supports recommendations, tag browsing, alternate editions/translations
  • 2.4674 seconds: user experience interrupted
  • 19 items, 138.84 K
  • Cover art nearly doubles response time
    • Item 14: script on ILS that queries Amazon for art
usc findings
USC Findings

Slowest among traditional catalogs; Sirsi/Dynix takes longer to make initial connection (Item 1 on graph)

8.7295 seconds (though average was 4.1085 seconds)

16 items, 148.47 K

While attractive and well-integrated, Syndetic Solutions content (cover art, summary, author biography, and table of contents) adds 1.2 seconds to load time

is the content worth the wait
Is the Content Worth the Wait?

“The new database seems based on Amazon.com. I don’t need suggestions, and poor ones at that, of related books when I use the library. I don’t need to see what other borrowers thought of the book. The information I need is poorly displayed. It is hard to cut and paste. It takes several screens to scan through, instead of the much quicker scroll in the traditional format…. It supplies distracting, if not useless information (a picture of the cover, the distance to other libraries—as if I need to know how far Provo is).”

-Auraria Campus Faculty Member

our conclusions1
Our Conclusions

Make performance testing part of evaluation process for vendor products

Adhere to industry standards for acceptable response times when testing

Optimize delivery of 2.0/NextGen features as much as possible

Conduct user testing to ensure that the content is “worth the wait” to their minds

questions comments
Questions? Comments?

Nina McHale

@ninermac

milehighbrarian.net