1 / 18

Usability and Utility of TopicLens

Usability and Utility of TopicLens. a Visualization System for the Exploration of Topic Models. SUNIL GAHLAWAT LU GAN MIYA SYLVESTER YIRAN WANG. Topic Models. Topic models provide a simple way to analyze large volumes of unlabeled text. Topic Models.

aviv
Download Presentation

Usability and Utility of TopicLens

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Usability and Utility of TopicLens a Visualization System for the Exploration of Topic Models SUNIL GAHLAWAT LU GAN MIYA SYLVESTER YIRAN WANG

  2. Topic Models • Topic models provide a simple way to analyze large volumes of unlabeled text.

  3. Topic Models • A "topic" consists of a cluster of words that frequently occur together.

  4. Topic Models • An example of Topic Modeling : Twitter tweets search for "Hollywood"

  5. Why develop these kinds of Models? • Organizing and finding patterns in data has become important in the sciences, humanities, industry, and culture. • Topic models can be built for images, social networks, music, purchase histories, computer code, genetic data, click-through-data...

  6. TopicLens Interface

  7. TopicLens Interface

  8. TopicLens Interface

  9. Goals • Get a good understanding whether the user editing function in TopicLens is useful or necessary. • Identify and classify any usability defects. • Study whether the design elements and aesthetics enhance the Usablity of the program.   • Evaluate User satisfaction/User experience in terms of ease of learning, retention, efficiency, and reliability.

  10. A Detailed Summary of Usability Study • Primary focus on User Testing • Initial Heuristic Analysis • Short Survey and Interview with Participants • 6~10 Participants • Two sub-groups to evaluate different usage behavior • 3~5 Students, 3~5 Non-students • Recruited from grad student housing, university center, etc. • Recording: Audio and Screen Video Capture • Time: 30~40 minutes • Location: Donald Bren Hall

  11. A Detailed Summary of Usability Study • Part1: First Impression on Visual Presentation • We will briefly explain about TopicLens to the participant  • Participants will be asked to think aloud as they experiment with the interface. • Editing Mode: OFF 

  12. A Detailed Summary of Usability Study • Part2: Details on User Editing Functionality • We will give participants a few scenarios to explore all the functions available on TopicLens (merging/filtering/hiding) and answer any questions the participant may have about these functions. • Bill is Professor at a research university who interested in software/hardware and security. Bill will be using the merging function of TopicLens to visually organize his topics. • Hazel is a movie buff and she wants to see topics related to all movies. She’ll be filtering function to find the topics related to movies. • Hazel has filtered her movie topics but there are still too many for her to sort through. She will use the hiding function to better suit her needs. • Participants will be asked to think aloud as they complete the tasks. • Editing Mode: ON

  13. A Detailed Summary of Usability Study • Part3: Survey and Interview • Survey: • Participants will be asked to complete a short survey (adapted from System Usability Scale) on the functionality of TopicLens. The following are examples participants would rate on a scale of 1(strongly disagree) to 5(strongly agree): • I thought the system was easy to use. • I found the various functions in this system were well integrated. • I found the system unnecessarily complex. 

  14. A Detailed Summary of Usability Study • Interview: • We will also ask Participants open-ended questions that focus on editing: • Was it easy to understand the purpose of merging/hiding/filtering the topics or not? • After the editing, does the visual layout of the topics make more or less sense to you? • Right now, once you merge two topics, the nodes associated are either related to one or both topics. Do you prefer this, or do you prefer to see topics that are only related to both topics you merged? (For Participants who like the editing function, we will ask detailed questions about the AND/OR Boolean for two or more topics.) • Would you use the editing function or not? • Would you recommend the editing function to a friend or not?

  15. Plans to analyze usability test data • Qualitative "think aloud" data • We will play back the audio recording of each participant during their period of experimenting the system and transcribe those that we find most useful. • We will discuss any emergent patterns of user intentions, frustrations, and opinions that were expressed by Participants as a group. • Survey results • We will use Excel to perform basic analysis of the rating scales • We will also compare and contrast the ratings and their open-ended comments. • Open-ended interview question • We will transcribe the audio-taped open comments of each participant. • We will use content analysis on the open comments.

  16. Challenges • Environment configuration • Finding participants • Striving for consistency • Instructions • Interview questions • Testing situation • “Hard” participants • Transforming data into findings and practical recommendations

  17. Timeline

  18.  Drawbacks of TopicLens Observed • Font size is not adjustable • The background colors of program boxes sometimes make the label unreadable • Range slider design could be better • Titles of topics overlap when dragged together • Reset/return button missing

More Related