1 / 29

Seminar Software Quality and Safety

Seminar Software Quality and Safety. Topic: “Comparing design models resultant from two processes: with and without Architecture – an empirical study“ Author: Lukas Banach Tutors: Nazim Madhavji Remo Ferrari Thursday 17.07.2008. Motivation Task Context Variables Instrument Results.

henrik
Download Presentation

Seminar Software Quality and Safety

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Seminar Software Quality and Safety Topic: “Comparing design models resultant from two processes: with and without Architecture – an empirical study“ Author: Lukas Banach Tutors: Nazim Madhavji Remo Ferrari Thursday 17.07.2008 Motivation Task Context Variables Instrument Results

  2. Overview Motivation Task description Study context Static / dynamic attributes Instrument to measure attributes Preliminary results Future Work Motivation Task Context Variables Instrument Results

  3. Motivation Someone wants to create lower-level models Does architecture help? Do we produce higher quality (faster / more precise, ...) with architecture? What are the costs and the benefits? There is a huge gap between research and the industry Useful to say: use architecture because of ... Motivation Task Context Variables Instrument Results

  4. Motivation Answer research question: “How do lower-level design models constructed with a software architecture compare to lower-level design models that are not based on an architecture?“ Motivation Task Context Variables Instrument Results

  5. My task description My focus is to establish the methology of study to answer this question Dependent variables, instument, statistical analysis Prepare an empirical study No study – beyond the scope of this seminar Motivation Task Context Variables Instrument Results

  6. Basic idea of an empirical study Dependent variable (292) Dependent variable (38) Dependent variable (35) Dependent variable (436) Dependent variable (007) Dependent variable (112) Dependent variable (82) Dependent variable (2) Dependent variable (93) Dependent variable (4) influence Independent variable (1) Independent variable (0) Binary variable (0/1) Motivation Task ContextVariables Instrument Results

  7. Where are the models from? Participants: ungraduate students Randomly assigned by the staff Their task: model the behaviour of a garage door Two types of groups With and without architecture Motivation Task ContextVariables Instrument Results

  8. Where are the models from? 6 groups ouf of 19 XML data (Models) Task description All created documents Motivation Task ContextVariables Instrument Results

  9. Uppaal design models Templates States Transitions Xml – Files Simulator Verifier Motivation Task ContextVariables Instrument Results

  10. Study variables Independent variable (binary) Architecture or no architecture Dependent variables Static attributes Dynamic attributes Motivation Task Context VariablesInstrument Results

  11. Static attributes Number of templates Number of states Number of transitions Size of the xml file Number of available features Detail level of the available features Complexity Level of the templates Motivation Task Context VariablesInstrument Results

  12. Dynamic attributes Percentage of accessible states Percentage of correct scenarios Deadlocks Infinite loops Motivation Task Context VariablesInstrument Results

  13. Instrument to measure attributes Xml parser Counting states, transitions, templates & complexity level Creates queries to check accessible states (E <>) Checklist available features and detail level Motivation Task Context VariablesInstrument Results

  14. Instrument to measure attributes - Xml parser Motivation Task Context VariablesInstrument Results

  15. Instrument to measure attributes Verifier Accessible states (E <>) Deadlocks (E<> deadlock) Scenarios Available features Simulator Motivation Task Context VariablesInstrument Results

  16. Instrument to measure attributes - verifier Motivation Task Context VariablesInstrument Results

  17. Instrument to measure attributes - simulator Motivation Task Context VariablesInstrument Results

  18. Collected data Motivation Task Context VariablesInstrument Results

  19. Collected data - features 1a: Opening the garage door by user pressing the button. 1e: All operations should involve displaying messages on the control unit which would be located in the garage. Motivation Task Context VariablesInstrument Results

  20. Collected data - scenarios 1: User presses the garage door button to open the garage. While the door is opening, the user presses the button. The user presses the button again; however, there is an obstacle in the way of the door going down. Motivation Task Context VariablesInstrument Results

  21. Statistical analysis - total Motivation Task Context VariablesInstrument Results

  22. Statistical analysis - percentage Motivation Task Context VariablesInstrument Results

  23. Statistical analysis - complexity Motivation Task Context VariablesInstrument Results

  24. Statistical analysis - correlation Motivation Task Context VariablesInstrument Results

  25. Statistical analysis - significance • χ2-Test (Chi-Square-Test) • H1: The correlation between the attribute “Architecture“ and the number of correct Scenarios is statistically significant. • H0: The attribute “Architecture“ and the number of correct Scenarios are statistically not correlated • χ2 = 0.1893 Motivation Task Context VariablesInstrument Results

  26. Summary Attributes Created instrument Collect preliminary data Statistical analysis Motivation Task Context VariablesInstrument Results

  27. Future work Improve scenarios (yes/no -> stopped at ) Create more queries with the xml parser For the study: Use professional people instead of students Motivation Task Context VariablesInstrument Results

  28. The end Questions ?

  29. χ2 χ2 = (3.5 – 4)² / 4 + (3.5 – 3)² / 3 + (11.5 – 11)² / 11 + (11.5 – 12)² / 12 χ2 = 0.1893 χ2 df = 1:5% = 3.84 χ2 df = 1:1% = 6.68 χ2 df = 1:0.1% = 10.8

More Related