1 / 12

Instrumenting the Learning Commons: Enhancing User Experience through Behavioral Metrics

This project aims to evaluate the effectiveness of search/discovery tools in Learning Commons by utilizing behavioral metrics and conducting aggregate, longitudinal studies. The project includes tools for usability studies, scalability for hundreds/thousands of participants, realistic tasks, and on-demand playback of sessions. The framework allows unified analysis and query capabilities for internal/external resources, with enforced privacy measures. The approach involves client-side instrumentation using LibX Toolbar for tracking user behavior on web search pages. Detailed events such as clicks, mouse movements, scrolling, and text entries are recorded for analysis and improvement. The system also offers an example application for classifying search intent using machine learning techniques. Emory Libraries' User Behavior Analysis System (EUBA) includes client-side instrumentation and data mining to enhance user experience.

vidor
Download Presentation

Instrumenting the Learning Commons: Enhancing User Experience through Behavioral Metrics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Instrumenting the Learning Commons Eugene Agichtein, Qi Guo and Ryan Kelly Intelligent Information Access Lab, Math & CS Department Arthur Murphy, Selden Deemer, Kyle Fenton Emory Libraries

  2. Goals/Motivation • Evaluate effectiveness of search/discovery with behavioral metrics (task-specific) • Perform aggregate, longitudinal studies • Tools for usability studies “in the wild” • Scale (hundreds/thousands of “participants”) • Realistic behavior and tasks • On-demand playback of “interesting” sessions • Unified analysis/query framework for internal and external resource access and usage statistics • Web-based query and statistics interface • Access auditing, privacy, anonymity enforced Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  3. Approach: Client-side instrumentation • Implementation within the Emory Installation of the LibX Toolbar: (http://www.libx.org) • Extended LibX to track UI events: JavaScript patch to sample the mouse movements and other events on pre-specified web search pages. Events are encoded into a string and buffered, and periodically sent to the server (on internal library network). Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  4. How it works • On login, firefox is started with • http://irlib.library.emory.edu/consent.cgi?user=USERID • If user has previously opted in (or out) • Redirect to Euclid homepage • If new user, show consent form • Store choice in database; if opted in, also store salted hash string for user log in • Can track user behavior over “lifetime” • No way to recover login id by dictionary attack • Can be removed at any time by deleting mapping • LibX sends http requests to server with encoded event strings. Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  5. User Opt-in (new Learning Commons Users) http://irlib.library.emory.edu/ Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  6. Events captured (v0.4, deployed) • Button/link clicks/Url changes • Name of the button, link, other meta-info • Mouse movements • (x,y) coordinates sampled ~every 10ms • Scrolling • Start, stop position, ~ every 10ms • Text entry • Query text, options changes • Keypress events • Menu item events • Print, bookmark, save (all of them) Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  7. Example: Mouse Movement 5 segments: initial, early, middle, late, and end. • “Cheap” proxy for eye-tracking: • Capture physiological characteristics of the mouse trajectories • Segment Properties: • Speed • Acceleration • Rotation • Drift • Hover For each: avg. speed, avg. acceleration, rotation etc. Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  8. Example applications: Classifying Search Intent • Initial exploration: • Standard supervised machine learning classification techniques • WEKA implementation of SVM and decision trees. Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  9. Summary of Features Used Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  10. Example Informational query: “spanish wine” Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  11. Emory User Behavior Analysis System • EUBA: • Client-side instrumentation, Data mining/machine learning (Qi Guo) • Log DB parsing, indexing, web-based interface for querying, playback, annotation (Ryan Kelly) • Plan: to release the system to research/library community Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

  12. Demo Prototype: http://ir.mathcs.emory.edu/library/private/index.pl user: test password: notsafe Intelligent Information Access Labhttp://ir.mathcs.emory.edu/

More Related