1 / 16

The Analyst’s Tool

Explore the challenges in developing groupware systems and the use of the analyst's tool to analyze and improve coordination. Learn about iterative design, coordination representations, and building coordinating representations.

pchapman
Download Presentation

The Analyst’s Tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Analyst’s Tool and its use in the construction of Groupware systems Alex Feinman afeinman@cs.brandeis.edu

  2. Developing Groupware • Developing Groupware systems is HARD • Users don’t know what they need • Developers don’t know what the users need • Solutions are of mixed usage • Forcing the users to use what you come up with • Design Guidelines • E.g., GUI designs; Patterns • Iterative Design • Design -> Implement -> Evaluate -> Redesign …

  3. Iterative Design • Slow, Expensive • Requires many cycles for useful software • Initial design may be very far from acceptable product • Often users are not patient enough for a polished product to be produced • Same problems are solved over and over again by different developers • Exponentially more difficult for multi-user system (# subgroups = 2n)

  4. Iterative Design: Evaluation • Much work has been done on Design (e.g., GOMS) and Implementation (e.g., ADAPTIVE) • Less work on Evaluation • More precise evaluation should allow for fewer cycles, leading to faster development • Multi-user evaluation has its own extra complications

  5. Analyst’s Tool • Part of a larger system for implementing and analyzing Groupware Analyst’s Tool More Groupware Systems VesselWorld SAGE (replay tool) Basic Groupware Tools (chat room, shared whiteboard, etc.) ADAPTIVE

  6. Methodology • Implement a base groupware system, providing only basic tools such as a chat room and shared whiteboard • Collect significant amount of data of real workgroups doing their task using the base system • Analyze data using analyst’s tool to isolate and evaluate coordination issues • Implement Coordination Representations to solve recurring problems of coordination

  7. Guidelines for Building Coordinating Representations • Library of patterns in coordinating representations is under construction • list, checklist, set, table, flowchart • media attributes: Clark’s list [reviewable, editable, synchronous, etc.]; verification needed, approval needed, vetoable; voluntary vs. automatic updates; etc. • interface design issues: screen real estate, effort to create/modify/remove items, • Guidelines for matching coordination problems to attributes also under construction • e.g., if a known of actions needs to be performed in order, a checklist may be indicated • First, need gather and analyze data to determine which coordinating representations are needed

  8. Gathering Data • ADAPTIVE automatically collects data • Data collected consists of: • Chat transcripts: • GUI actions performed by users • mouse clicks, window movement, keystrokes, etc. • Domain-specific actions of users • in VesselWorld, lifting and moving waste, moving barges, navigating the harbor, etc. crane2: let’s get swaste1 crane1: no way dude. We need to wait for teh tug crane2: tug, isn’t swaste1 no eq? crane2: no, wait, it’s unknown tug1: haven’t id’d it yet crane2: so wait crane1: we could do the xl int he corner

  9. Analyzing the Data • Primarily analyzing the chat transcript, supported by investigation of domain actions • Chat can be analyzed using augmentation of Conversation Analysis techniques • Four basic areas of interest • Segmenting • Turn construction • Turn-taking • Group dynamics

  10. Segmenting • Segment tool allows analyst to easily delineate interesting sections of data • Allows alignment of corresponding sections of similar segments and extract the shared structure • Allows construction of segment hierarchy, from which a hierarchy of actions can be inferred • Once a case-base of possible user actions has been constructed, user choices in turn-construction are made more visible

  11. Segmenting Tool Segment Comparison View Segment A: jerry segment Segment B: are you sure that’s the net crane2: so I says to him, jerry, I says, I don’t want to go crane1: quite and submit something tug1: waste at 300/125 needs net crane1: what size? crane2: and he says, shut up and be quick about it. crane1: [tug1] What size is the waste at 300, 125? tug1: waste at 320/98 needs nothing tug1: net crane1: [tug1] not equipment, what size is it? crane2: doesn’t anyone want to hear my storiy? tug1: small waste at 300/125 crane1: whatst hew aste at 300 190? crane1: oops 190 300 crane2: ln@300,190!l crane1: thanks tug1: Are you sure that’s net? crane2: I thought so tug1: I had ‘no equipment’. Let me check before you get it. crane1: ok. FRAME: WASTE INFORMATION QUERY ------------------------------------ SLOT 1: NAME: Request for information COMPLICATIONS: no response (busy, ignoring?) (A:1b) self-correction (B:1b) correction after next (no examples) STRUCTURES: none recorded yet ------------------------------------ SLOT 2: NAME: Information returned COMPLICATIONS: mismatch of CG between users (B:2c-2e) wrong information returned (A:2a-2c) STRUCTURES: jargon (B:2a) * 1a * 1b 2a * 2b 1a 2c * 2d 1b 2e * * 2a 2b * 2c

  12. Turn construction • Analyst can tag individual utterances as desired to manually extract patterns • Two sets of tags • Domain-independent list of tags, generally dealing with coordination issues (e.g., Commitment, Reference Error) • Domain-specific list of tags (e.g., Waste Dropped, Joint Lift) • Tag lists are customizable by analyst

  13. Turn-taking • Quantitative analysis tools allow examination of trends in turn-taking • Turn-taking clusters may point to subgroup formation • Patterns in turn-taking may indicate negotiation of roles

  14. Group Dynamics • Structured notepad allows analyst to record impressions of group dynamics • Overview/timeline tool presents impressions in situ, allowing comparison of dynamics across problem-solving sessions and groups.

  15. Statistical Analysis Tool • Allows rapid investigation into statistical trends in the data. • Currently under development by Michael Alex

  16. Integration of Tools • Tools not meant to be used in isolation • e.g., segmenting and comparing segments allows easier extraction of case-base of (conversational) actions • Tool data is persistent and shareable, allowing collaboration by multiple analysts either concurrently or in sequence • Tool could perform automatic comparison of analysis by multiple reviewers, strengthening confidence in results

More Related