1 / 22

R&D SDM 1 Metrics How to measure and assess software engineering?

R&D SDM 1 Metrics How to measure and assess software engineering?. 2009 Theo Schouten. Contents. What are “software metrics” Why is measurement important? Software Quality Qualitative measures Quantitative measures Final remarks Book chapter 15. Metrics. What is a metric?

marlow
Download Presentation

R&D SDM 1 Metrics How to measure and assess software engineering?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. R&D SDM 1MetricsHow to measure and assess software engineering? 2009 Theo Schouten

  2. Contents • What are “software metrics” • Why is measurement important? • Software Quality • Qualitative measures • Quantitative measures • Final remarks Book chapter 15

  3. Metrics • What is a metric? • “A quantitative measure of the degree to which a system, component or process pocesses a given attribute” (IEEE Software Engineering Standards 1993) : Software Quality • Different from • Measure (size of a system, component e.d) • Measurement (act of determining a measure) • Metrics • Qualitative Metrics • Quantitative Metrics

  4. Why important, difficult • Why is measurement important • to characterize • to evaluate • to predict • to improve • Why is measurement difficult? • No “exact” measure (‘measure the unmeasurable’, subjective factors) • Dependent on technical environment • Dependent on organizational environment • Dependent on application and ‘fitness for use’

  5. Quality definition • (degree of) conformance to: • explicitly stated functional and performance requirements • explicitly documented developments standards • implicit characteristics that are expected of all professional developed software. Requirements are the foundation to measure software quality Standards define the development criteria for software engineering Software quality should conform to explicit and implicit requirements

  6. Software quality attributes Product Revision Product Transition Software Quality Factors (McCall, 1977): Maintainability Flexibility Testability Portability Reusability Interoperability Product Operation Correctness Usability Efficiency Reliability Integrity

  7. Software Quality: Qualitative Measures McCall: Metrics that affect cq influence software quality factors: • Software Quality Factors are the dependent, metrics are the independent variable • Metrics: audibility, accuracy, communication commonality, completeness, consistency, data commonality, error tolerance, execution efficiency, expandability, generality, hardware independence, instrumentation, modularity, operability, security, self-documentation, simplicity, software system independence, traceability, training. • Software quality factor = c1 m1 + c2 m2 + … + cn mn • Cn is a regression coefficient based on empirical data • Software quality factor gives an indication of the quality of the software

  8. McCall Matrix

  9. FURPS, Quality Factors • Developed at Hewlett-Packard (Grady, Caswell, 1987) • Functionality: • Feature set and capability of the system • Generality of the functions - Security of the overall system • Usability: • Human factors (aesthetics, consistency and documentation) • Reliability: • Frequency and severity of failure • Accuracy of output • MTTF - Failure recovery and predictability • Performance: • Speed, response time, resource consumption, throughput and efficiency • Supportability: • Extensibility, Maintainability, Configurability, Etc.

  10. ISO 9126 Quality Factors 6 key quality attributes, each with several sub-attributes • Functionality • Reliability • Usability • Efficiency • Maintainability • Portability Also often not direct measurable, but gives ideas for indirect measures and checklists. defect : The nonfulfilment of intended usage requirementsnonconformity : The nonfulfilment of specified requirements superseded by the new project SQuaRE, ISO 25000:200

  11. Quantitative Metrics Desired attributes of Metrics (Ejiogu, 1991) • Simple and computable • Empirical and intuitively persuasive • Consistent and objective • Consistent in the use of units and dimensions • Independent of programming language, so directed at models (analysis, design, test, etc.) or structure of program • Effective mechanism for quality feedback Type of Metrics: • Size oriented • Focused on the size of the software (LinesOfCode, Errors, Defects, size of documentation, etc.) • independent of programming language? • Function oriented • Focused on the realization of a function of a system

  12. Product Metrics Landscape • See chapter 15.2.6 • Metrics for the analysis model • Metrics for the design model • Metrics for the source code • Metrics for testing

  13. Function Oriented Metrics Function Point Analysis (FPA) a method for the measurement of the final functions of an information system from the perspective of an end user • method to determine size of a system on basis of a functional specification • method is independent of programming language and operational environment • its empirical parameters are not. • Usable for • estimate cost or effort to design, code and test the software • predict number of components or Lines of Code • predict the number of errors encountered during testing • determining a ‘productivity measure’ after delivery

  14. Function Point Analysis Basis is the Function Punt Index (FPI) of a system to build: . Amount of work = Function Point Index * Resource factor Resource factor depends on: • development environment • experience of project team with environment and tools • size of the team Benchmarks and measurements of previous projects are used to estimate the resource factor.

  15. Function Point Index Determine System Attributes Determine ‘Value Adjustment” • How do you calculate the FPI of a system to be build? • Three steps are used: Determine Function Point Index

  16. FPA: System Attributes • Count the number of each ‘system attribute’ : • User (Human or other system) External Inputs (EI) • User External Outputs (EO) • User External Inquiries (EQ) • Internal Logical Master Files (MF) • Interfaces to other systems (IF) External User Transactions EI Interface EO EQ IF EQ MFs Transactions EO EI Other Systems System Environment

  17. FPA: System Atributes weighting • Determine per system attribute how difficult it is: • Low • Medium • High • Use the following matrix to determine the weighting factor: • Calculate the weighted sum of the system attributes: the ‘Unadjusted Function Point Index’ (UFPI)

  18. FPA: Value Adjustment • The UFPI needs to be adapted to the environment in which the system has to operate. The ‘degree of influence’ is determined for the 14 ‘values adjustment ‘ factors:. • Data Communications • Distributed Processing • Performance Objectives • Tight Configuration • Transaction Volume • On-line Data Entry • End User Efficiency • Logical File Updates • Complex Processing • Design for Re-usability • Conversion and Installation Ease • Multiple Site Implementation • Ease of Change and Use Value between 0 and 5

  19. FPA: final function point index • Total sum of the ‘degree of influence’ (DI) (0-70 • Determine Value Adjustment : VA= 0.65+ (0.01*DI) (0.65-1.35) • Function Point Index = VA*UF • Historical data can then be used, e.g. • 1 FP -> 60 lines of code in object oriented language • 12 FP's are produced in 1 person-month • 3 errors per FP during analysis, etc.

  20. Example Sensors Test Sensor Functie: Gebruiker Password Alarm Gebruiker’s Interactie Module Zone instellen Sensor opvragen Berichten Panic button Gebruiker Sensor Status Zone opvragen Aan/Uit Aan/Uit Password, sensors etc. Alarm Alert Externe Monitoring Systeem Configuratie Data

  21. Unadjusted Function Point Index • Determine the number of system attributes: • User External Inputs (EI) • User External Outputs (EO) • User External Inquiries (IQ) • User Logical Master Files (MF) • Interfaces to other systems (IF)

  22. End remarks • Metrics are usable for a relative view on a system, not an absolute view. • Metrics have qualitative and quantitative aspects. • Realize you try to measure the unmeasurable • Use metrics to determine the functionality of a system on basis of the wishes of the end user • Use metrics for: • To characterize • To evaluate • To predict • To improve

More Related