Embedding e-Science Applications: Designing and Managing for Usability

1. Embedding e-Science Applications: Designing and Managing for Usability Marina Jirotka and Sharon Lloyd Oxford University NeSC The Changing Landscape 16th April 2009

2. Embedding e-Science Applications: Designing and Managing for Usability • A 3-year project funded by the EPSRC • Project staff: Marina Jirotka (PI) Anne Trefethen (CI) Sharon Lloyd (Advisor) Dimitrina Spencer (Researcher) Ralph Schroeder (Researcher) and Grace de la Flor (DPhil student) (Andrew Warr -researcher Year 1) • Working closely with the Oxford e-Social Science (OeSS) project, the e-Horizons Institute and the UK e-Science Usability Task Force (UTF)

3. Objectives • To develop an online toolkit defining processes and practices for managing collaboration to facilitate usability in e-Science projects • Through engagement with project collaborators, to investigate approaches, tools and techniques that may enable the development of shared understanding of e-Science project expectations, management and implementation • To develop recommendations, guidelines and procedures that facilitate the effective integration of e-Science technologies with existing work practices whilst also allowing potentially new ways of working • To consider usability of nationally provided services through a broker, and to draw upon case studies to provide insights into the use of the NGS and large-scale resources like it • To consider specific tools and technologies that allow user engagement with projects such as personal Access Grids. • To develop a set of case studies leading to recommendations for managing usability on e-Science projects

4. Embedding Use of large scale infrastructure Use of services/ applications Use of collaborative tools and approaches Task Forces STC, ETF, UTF OGF NGS HPCx CSAR Research Projects: dealing with specific problem Dame, e-DiaMoND, Integrative Biology, NeuroGrid, Carmen… Access Grids, PIGS Commercial Solutions OMII - Product development

5. What is Usability? • As defined by the ISO standard ISO 9241 Part 11, usability can be measured only by taking into account the context of use of the system • who is using the system, what they are using it for, and the environment in which they are using it • Furthermore, measurements of usability have several different aspects • effectiveness (can users successfully achieve their objectives) • efficiency (how much effort and resource is expended in achieving those objectives) • satisfaction (was the experience satisfactory) • Who is the system built for? Management? End users?

6. Who are the users? Typically… Applications Portals Middleware Developers Middleware/Core infrastructure development Users of Standards, Products and Trends

7. Infrastructure/Services - challenges • to 'productise' the outputs from e-science projects/initiatives and to ensure outputs were developed in a scalable and robust fashion. • how to ensure that what is developed is usable for everyone? • What do users expect of an infrastructure? - robust and sustainable egs HPCx vs CSAR - user forums very technical • Less uptake than expected • Inadequate understanding of kinds of services • Insufficient resources to make it happen • National vs local solution • Whose responsibility is to ensure usability?

8. Infrastructure - lessons learned • Mode 1 Provision of national services and infrastructure. Seamless and sustainable provision - different mode of engagement with users: training - handholding - documentation.. • Mode 2 Use of standards and services to develop own infrastructure for a specific scientific problem - need visible/transparent infrastructure where users can see what it is doing and modify it • Applications and infrastructure co-evolve • Gap exercises with users • Localisation - local staff, system administrators, groups of users • But many projects involve cross-institutional work • No large scale data sets to work across institutions • No one solution as an institution to investigate • Researchers have to work out own mechanisms for long-term collaboration

9. Users and Applications - challenges identified • In depth qualitative studies of several key e-Science projects reveal lack of impact - applications not being used beyond lifetime of the project • User requirements not clearly understood - little expertise in elicitation or how they fit into the development cycle • Different types of ‘users’ - middleware developers, end users.. • Stakeholder requirements often poorly conceptualised - who is a stakeholder? • Embedding of applications seen as an additional requirement once system developed

10. Users - lessons learned • Early strong user/stakeholder buy in and feedback - engaging people who are v busy - communication across different groups translation exercise • Engage in project vision - recalibrate throughout project lifetime • Showcase technical possibilities - milestones in project plan • Understanding current activities tools and techniques • Develop stakeholder analysis to ensure right partners and people in the community • Ensure enough time and resources for engaging users efficiently • Focus on exploitation and impact - is this research? and who funds?

11. Collaboration and Communication - Challenges • Instantaneous methods for communicating are important in dynamic teams e.g. use of video conferencing requiring a weeks notice is problematic - or limited skill set • Partners may have favourite audio/video services - either initiate change, or ensure interoperability of services - applies to visualisation tools also • Methods for recording and recollecting information are vitally important for wider dissemination of project knowledge • Translation of information to communities requires consideration of target audience constantly and often may require external development - good for public engagement • Support for training people and institutional buy in to provide tools such as AG

12. Collaboration and Communication - lessons learned • Methods and tools need to be inclusive - so select an approach that interoperates with all platforms - compliant technologies - Neurogrid - Cancergrid commercial solution (coffee room - open activities could have been intrusive) • Using shared repositories and Wikis to collate decision making processes - who does this and maintains these repositories even after the project is over • Certain tools such as Crewe and Memetic potentially interesting for recording - but seemingly no active embedding

13. Project Management - challenges • Scientific vs operational management • Imperative that there is clear role definition and identification of who takes the scientific lead and who takes the operational lead – or one and the same person? Neutral PM beneficial. • Project initiation – key to ensuring project progresses with clear understanding by all project members of what is expected of them. • Communication methods • Developing and maintaining shared visions and objectives • Closedown and Sustainability • How do you assess the delivery of a project and what legacy do they leave behind?

14. Project Management - lessons learned (1) • Taught methods not wholly applicable • Process skills needs to be coupled with social /management skills (e.g. NLP) • ‘Executing plans’ , but if cannot motivate the team … ? • Strategic vs operational management • Scientific vs operational - is PI both? • Different skill sets - may not be for entire duration of project • Project initiation • Defining roles and activities, key to maintaining the project teams • Stakeholder questionnaires, ‘cheap’ means of monitoring project team health • Communication methods - for different purposes • Teleconferences, wikis, maillist, websites, showcases, newsletters • Development and exploitation plan • Can be used to communicate how individuals fit into the bigger picture • Consider sustainable routes - users involved to sustain activity • For every deliverable what can be done to push beyond project

15. Project Management - lessons learned (2) • Management Styles - Use of language important ‘We’ ‘Our’ - Empowerment drives problem ownership - Embarrassment drives delivery! • Project closedown • Needs time… • Must indicate what will happen to output • Delivering exploitation plan • Consider what to do with website, wikis, reports, documents etc and whether others can benefit from them • Lessons Learned exercises - useful for all contributors • Blueprint documents useful to document what could not be achieved - content often results in new proposals for follow on projects and enables projects to publish knowledge that is not research.

16. Consider • Build it and they will come • Technical decision making in isolation from users • Users determining requirements • Disciplinary silos • PM as requirements engineer • Rigid inflexible technical vision • No stakeholder analysis at project inception and/or throughout project lifetime • Fixed waterfall development • No management of user expectations • Scientific and operational concerns • Involving users from project conception to closedown and beyond - strategies • Initiation activities • Showcasing technical potential • Closedown activities • Exploitation plans • Lessons learned activities • Blueprint • Open, modifiable and transparent infrastructure (not only by SA) • Ongoing agreements between users, users and developers, and other stakeholders

17. Suggestions • Provide training and support in operational management of large scale multi-disciplinary projects? • Can we learn from the EU Network of Excellence approach bringing together communities of interest? • Perhaps OMII + open tools + knowledge base + training = ‘toolkit’ for communities of interest?