1 / 19

WG3 Interplay between Design and Evaluation, Quality Models and Standards

WG3 Interplay between Design and Evaluation, Quality Models and Standards. Twintide Coimbra, Portugal March, 2013 Ebba Þóra Hvannberg. WG3a Qualities in use.

hidi
Download Presentation

WG3 Interplay between Design and Evaluation, Quality Models and Standards

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. WG3 Interplay between Design and Evaluation, Quality Models and Standards Twintide Coimbra, Portugal March, 2013 Ebba Þóra Hvannberg

  2. WG3a Qualities in use • Identify which software qualities in use (e.g., trust, creativity, automaticity, security, safety, sociability, usability, reliability, evolvability - (related to standards, e.g., ISO 9241, 27477)) • are differentiated by sector-dependent or sector-independent criteria and describe connections • vary over lifecycle • are traded-off against quality attributes • are prioritized and realized in the process of systems development in different sectors/disciplines

  3. WG3c design-evaluationfeedback • Understand how iterative design-evaluation-redesign feedback cycles operate for • computing systems in different sectors • relative to specific evaluation criteria: social, economic, technical etc. • traceability through cycle

  4. Past activities • Qualityattributemaps of relationsbetweendomains • Created initially(Bertinoro 2011) • Refined ( London, 2012) • Consolidated (Skopje, 2012)

  5. Analysis of Transfer of Quality Attributes • The goal was to strengthen the quality attribute map • Papers selected with the following criteria • Quality attribute(s), Sector/Domain, Method • Had to have empirical data • Papers analysed • 9 memberscontributedtothe analysis of 17 papers

  6. Stories of transfer described • Storieson transfer described • As a <role> I wantto <task> so that I can do <need> • Storytellerswereaskedtothinkaboutmethod transfer whendescribing a task

  7. Prescribed format (from facilitator) • 20 storieswerecollected in London, March 2012 • Thestorieshavebeenclassified and cleaned • A set of 12 storiesremainthatarerelatedtosome kind of transfer of methodfromone „methodapplication“ toanother

  8. Forms of transfer • Transfer between domains • E-Learning to Games • Transfer from generalization to specialization • Heuristics evaluation for Virtual Environments • Adoption of a technique from another field followed by adaptation • E.g. Laddering from marketing

  9. Storiesanalysed and grouped • Generalizationtospecialisation • Specialisation • Transfer betweendomains • Transfer betweensimilardomains • Adaptation

  10. Openquestions • Howcanthese results beverified or expanded • In theliterature, look for similar patterns • Casestudies • Knowledge on transfer • Collectmore transfer storiesfromliterature • In practice • Designers and developerschangejobsoften • Theymaybeabletotellus transfer stories, i.e. howthey transfer knowledge/methodsbetweensectors

  11. Copenhagen 2012 results • Tell a story of how you have carried out an evaluation, e.g. in a project and contrast it to another instance of an evaluation. • 18 cases of transfer stories were collected from Twintide members • Results are here

  12. Overview of form of transfer • Overview

  13. Revising stories and further analysis • Read and amend story • Typos • Specify domain of story • Review stories • Review transfer between stories • Add publication if available

  14. Identify method resources used in the stories * • Participant-recruitment • Task selection • Reporting format • Problem identification • Problem classification • Analysis • Heuristics • Thinking aloud protocol • Please add more as needed *Law, E.L.-C., Hvannberg, E., Vermeeren, A.P.O.S., Cockton, G., Jokela, T.: Made for Sharing: HCI Stories of Transfer, Triumph and Tragedy. CHI 2013 Extended Abstracts. ACM, Paris, France (2013)**Woolrych, A., Hornbæk, K., Frøkjær, E., Cockton, G.: Ingredients and Meals Rather Than Recipes: A Proposal for Research That Does Not Treat Usability Evaluation Methods as Indivisible Wholes. International Journal of Human-Computer Interaction 27 (2011) 940-970

  15. Identify contextual factors* • Business goal • Design purpose • Development context • Team skills • Corporate culture/values • Please add more as needed *Law, E.L.-C., Hvannberg, E., Vermeeren, A.P.O.S., Cockton, G., Jokela, T.: Made for Sharing: HCI Stories of Transfer, Triumph and Tragedy. CHI 2013 Extended Abstracts. ACM, Paris, France (2013)**Woolrych, A., Hornbæk, K., Frøkjær, E., Cockton, G.: Ingredients and Meals Rather Than Recipes: A Proposal for Research That Does Not Treat Usability Evaluation Methods as Indivisible Wholes. International Journal of Human-Computer Interaction 27 (2011) 940-970

  16. Contrast Method resources

  17. Contrast Contextual Factors

  18. Schedule

More Related