1 / 24

User Engagement in the eSciences

User Engagement in the eSciences. Dr Marina Jirotka Workshop on Requirements Capture for Humanities ICT projects The Classics Centre Oxford 12th October 2006. Overview. Importance of User Engagement Different Forms of User Participation The Problems of Analysis

drea
Download Presentation

User Engagement in the eSciences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. User Engagement in the eSciences Dr Marina Jirotka Workshop on Requirements Capture for Humanities ICT projects The Classics Centre Oxford 12th October 2006

  2. Overview • Importance of User Engagement • Different Forms of User Participation • The Problems of Analysis • Critical Issues from Related Research (CSCW) • Embedding User Requirements into System Design • Challenges for Future Research

  3. Used as delivered Used but after changes Used but extensively re- worked or abandoned Delivered but never used Paid for but not delivered (Davis 1993) Importance of User Engagement Only 2% of software used as delivered USA Government Accounting Office 53% of projects investigated failed 31% partially successful planned time overrun 122% average budget overrun 89% Standish Group (1995-99) Study of 500 IT managers in US & UK 76% experienced complete project failure Sequent Computer Systems Inc (1997)

  4. Reasons for Project Failure Study of 500IT managers (76% complete project failures) changing user requirements Sequent Computer Systems Inc (1997) incomplete requirements 13.1% lack of user involvement 12.4% unrealistic user expectations 9.9% requirements kept changing 8.7% product no longer needed 7.5% inadequate resources 10.6% lack of management support 9.3% inadequate planning 8.1% Standish Report 1995-99

  5. Mapping Out Different Forms of User Participation • None • Users as Sources of Information/Opinion • Participatory Design • Users in the Workplace • Contextual Design • Co-Design

  6. Users as Sources of Information/Opinion - Elicitation (1) • Traditional techniques - surveys, questionnaires, interviews • Often quantitative in nature • Advantages • Large population can be surveyed • Costs lowered, pre-coding and computerisation speed up analysis • Can get information quite quickly • Disadvantages • Response rate of questionnaires low (less than 50%) • Answers may be incomplete, illegible and incomprehensible • Need clear idea of what questions will elicit answers to research problems and skill in interviewing • Relationship between what we say we do and what we actually do is problematic

  7. Users as Sources of Information/Opinion - Elicitation (2) • Group techniques - brainstorming, focus groups, SSM, RAD/JAD workshops, creative workshops • Often more in depth qualitative interviews with small number of people • Advantages • Can get stakeholders together to discuss issues • Guided discussion - more interactional • Share different viewpoints and generate data for analyst and each other • Possible to converge more easily on agreement of issues and prioritisation • Disadvantages • Skill in managing and moderating • Eliciting all views not only loudest and most confident • Difficult to compare results in quantitative sense

  8. Participatory Design • Scandinavian approach (Greenbaum and Kyng 1991) • Original goal of increasing worker involvement in technical change and innovation • To improve IT system design by encouraging shared practice between users and designers • Advantages • Can use ‘low tech’ mock ups to elicit requirements • Promotes opportunities to build up shared understandings • Envision practices for how new system will be used in practice - scenarios • Disadvantages • Often focus on early design and prototype not into development and deployment • Not just involving users but how and when that is important • Tacit knowledge

  9. Scenarios Scenarios are a powerful antidote to the complexity of systems and analysis. Telling stories about systems helps to ensure that people – stakeholders – share a sufficiently wide view to avoid missing vital aspects of problems. Scenarios vary from brief stories to richly structured analyses, but are almost Always based on the idea of a sequence of actions carried out by intelligent agents. People are good at reasoning from even quite terse stories, for example, detecting inconsistencies, omissions, and threats with little effort. These innate human capabilities give scenarios their power. Scenarios are applicable to systems of all types, and my be used at any stage of the development lifecycle for different purposes. Ian Alexander, 2004 p3, Scenarios, Stories, Use Cases, I. Alexander & N. Maiden (eds.) Wiley

  10. Scenarios • A linear sequence of steps taken by independently acting agents playing roles • Analysis is about: decomposition, abstraction,moves toward the technical • • Problems • Engineers’ enthusiasm for early analysis and technical language • How to maintain the engagement of users throughout the development process? • Scenarios are simple way to help bridge this gap

  11. Scenarios - Suzanne Robertson • Elicitation, Analysis, Design, Testing, Agile methods • Normal, Alternative, Exception and What if • Get rough outline of business case • Identify right users and stakeholders • Sketch normal case scenario • Ask questions and refine scenarios • Identify alternative case scenarios • Identify exception case scenarios • Identify what if scenarios • Iterate • Derive requirements

  12. Users in the Workplace • Context - observation, ethnographic techniques, workplace studies • Rich qualitative fieldwork to gain understanding of users work in organisational settings • Advantages • Can make visible ‘real world’ naturalistic workplace activities vs operationalised accounts • Use of video • Focus on contingencies of workplace • Focus on details of everyday activities and use of artefacts in interaction • Resource throughout design and development process- transportability of experience • Resource for: informing requirements, design and generation of scenarios, prototype evaluation and deployment • Disadvantages • Data not immediately amenable to formal representation for design • Cannot predict future work practice from analysis of how things are now • Need some skill in analysis and communication

  13. Contextual Design • Understanding context indispensable to design (Beyer and Holzblatt 1998) • Contextual enquiry - talking to people as they do their work • Interpretations and modelling with cross functional teams • Consolidation of information gained through previous steps • Visioning about work practices and development of storyboards • User environment design - using storyboards to develop software floor plan that drives user interface design

  14. Co-Design • Cooperative design (Trigg et al 1999) Co-realisation (Hartswood et al 2002 ) • Focus on user led innovation and design-in-use • Co-development of technologies • Re-specification of IT design and development • Tightly coupled ‘lightweight’ design,, construction and evaluation techniques • Attending to evaluation of technologies • Appreciation of active user participation • Adapting to particular organisational setting • Commitment to long term engagement - putting users in control • Shift focus of technical work of design and development into users workplace • Role of broker / facilitator

  15. Problems for Analysis • Many different analytic techniques • Task analysis, walkthroughs, naturalistic analysis • Providing a warrant for analysis • Contingencies of project - time , resources, money • Recording - practical issues for audio-visual recording • Iterative development of analysis involving users, designer in data sessions, design sessions etc

  16. The The Five Foci of Interface Development after Grudin 1990 Mainframe used by operator Mainframe used by programmer Workstation used by skilled user Personal computer used by individual Networked computers used by groups

  17. Groupware and CSCW • Groupware - systems and applications to support groups • CSCW - Computer Supported Co-operative Work (the study, the field) PLACE Co-present Distributed synchronous face-to-face telephone TIME asynchronous Post It sticker letter

  18. Example collaborative applications PLACE Co-present Distributed synchronous GDSS shared drawing tools media spaces VRE TIME asynchronous - electronic mail bulletin boards shared writing tools workflow systems VRE

  19. Issues from CSCW Research • Synchronous or asynchronous • Distributed or co-located • Level of awareness of others required e.g. does the collaboration require face to face contact or is voice contact sufficient • Artefacts used e.g. phone, whiteboard, email, video conferencing • How information disseminated across setting • Issues or barriers to collaboration • Public and private activities - what resources publicly available • Tacit practices - overseeing, overhearing, practices of writing and reading etc

  20. Embedding User Requirements into System Design • Conventional system design - requirements to design to development • Flexible cross functional teams • Iterative prototype development • Evaluate prototypes to feed into design and development (eDiaMoND) • Technological interventions - cultural probes - Gaver 1999 (IBVRE) • Involving users and designers in data sessions, design meetings, preliminary reports and presentations • Managing user expectations • Plan for conflict and conflict resolution • Formal requirements inspections

  21. Evaluation for Requirements -eDiaMoND • Evaluation of Screening Workstation Prototype • Feed into requirements for next iteration • Based on workplace analysis

  22. Challenges • Securing user participation • Benefit to users • Time constraints • Incentives • Embed resources and commitment at project proposal stage • Commitment from project management • Ethical legal and institutional dynamics - global • Analysis still problematic • How fits into organisational practices • **System development culture **

  23. References The Standish Group (1995) The Chaos Report. www.projectsmart.co.uk/docs/chaos_report.pdf Beyer, H. and Holtzblatt, K. (1998) Contextual Design. Defining Customer-Centred Systems. Morgan Kaufmann. August, J. (1991) Joint Application Design. The Group Session Approach to System Design. Yourdon Press. Greenbaum, J. and Kyng, M. (1991) (Eds) Design at Work: Cooperative Design of Computer Systems. Lawrence Erlbaum Associates. Heath, C.C. and Luff, P. (2000) Technology in Action. Cambridge University Press. Checkland, P. (1981). Systems Thinking, Systems, Practice. John Wiley. Hartswood, M., Procter, R., Slack, R., Voss, A., Buscher, M., Rouncefield, M., Rouchy, P. (2002) Co-realisation. Towards a Principled Synthesis of Ethnomethodology and Participatory Design. In The Scandinavian Journal of Information Systems. Grudin, J. (1990) The computer reaches out: The historical continuity of interface design. CHI ‘90, April Seattle. Working IT out in e-Science: Experiences of Requirements Capture in a HealthGrid project (2005) inn Proceedings of HealthGrid 2005, Oxford UK. Jirotka, M and Wallen, L. (2000) Analysing the Workplace and User Requirements: Challenges for the Development of Methods for Requirements Engineering. In Workplace Studies: Recovering Work Practice and Informing System Design (Eds) Luff, P., Hindmarsh, J., and Heath, C.C. Cambridge University Press. Robertson, S. (2004) in (Eds) Alexander, I. and Maiden, N. Scenarios, Stories, Use Cases. Wiley

More Related