Software initiatives 4 Quality Standards

1. Software initiatives 4Quality Standards See Word 97 file “Software initiatives 4”

2. Quality (BURKS) • The totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needs. Not to be mistaken for "degree of excellence" or "fitness for use" which meet only part of the definition. • There is no single, unambiguous definition.

3. Quality Assurance (BURKS) • A planned and systematic pattern of all actions necessary to provide adequate confidence that the product optimally fulfils customer's expectations. • This term can be used generically to refer to all activities that affect quality in any way. • It also has a number of more specific definitions, such as the label assigned to quality assurance departments and to quality assurance personnel.

4. Quality Control (BURKS) • The assessment of product compliance. Independently finding deficiencies assures compliance of the product with stated requirements. • This term can be used generically for the same concepts as "quality assurance" or more specifically to specific combinations of inspections and test steps applied to software projects.

5. Quality Standards (BURKS) • "The nice thing about standards is that there are so many of them to choose from", • Standards are necessary for interworking, portability, and reusability. • They may be de facto standards for various communities, or officially recognised national or international standards.

6. Standards? • Every country has a standards organisation. • In addition, many industries and technologies also have standards bodies, such as the well known standards published by the IEEE (Institute of Electrical and Electronic Engineers) and the European Computer Manufacturers Association (ECMA). • Many large corporations have in-house standards. Indeed, for software quality the internal standards used at companies such as AT&T, IBM, Hewlett Packard, Motorola, and many others are among the most effective in the world. • In some cases, the in-house standards pioneered entirely new concepts such as Motorola's well known Six-Sigma quality standard.

7. Some standards-issuing organisations • Allied Quality Assurance Publications (AQAP) produced by NATO • American National Standards Institute (ANSI) • British Standards Institute (BSI) • Department of Defence (DoD) in the United States • Department of Defence (Def) in the United Kingdom • Deutsches Institut fur Normung (DIN) • European Computer Manufacturing (ECMA) • European Space Agency (ESA)

8. Some more standards-issuing organisations • Federal Information Process Standards (FIPS) • International Organisation for Standards (ISO) • Institution of Electrical Engineers (IEE) • Institute of Electrical and Electronic Engineering (IEEE) • Japanese Industrial Standards Committee (JISC) • Japanese Union of Scientists and Engineers (JUSA) • Software Publishers Association (SPA) • Various in-house standards and guidelines by specific companies

9. Oh Dear! • “On the whole, the software standards associated with quality have had only a marginal effect on quality.” • “Generally speaking, software standards are based on the subjective views of the standards committee, and seldom have even much solid empirical data behind them.”

10. European Software Initiatives • The combined software population of Western Europe and the U.K. is somewhat larger than the software population of the United States and very energetic in terms of quality initiatives, standards, conferences, and Internet communication channels. • Quality initiatives such as the ISO 9000-9004 standards, now have global impact.

11. The Good • The same industries tend to achieve the best quality results in every industrialised country in Europe, North America, South America, the Middle East, and the Pacific Rim. • These industries share a common characteristic that their software supports large and complex physical devices which need close to zero-defect software in order to operate successfully.

12. The Good • Aerospace manufacturers • Commercial systems software manufacturers • Computer manufacturers • Defence system manufacturers • Medical instrument manufacturers • Telecommunication manufacturers

13. The Bad • Several industries where software quality lags in the United States appear to be rather good in much of Europe: insurance and banking software, for example. • In the USA the insurance and banking software development practices tend to be somewhat less formal and more casual than the six industries with the best quality results.

14. The Bad • The financial and insurance industries have also been rushing to get into the • client/server world, • where quality is often marginal and even worse than equivalent mainframe applications. • By contrast, financial and insurance software in Europe has a higher probability of rigor during development, more care in quality control, and a greater use of non-test defect removal methods such as inspections.

15. The Ugly • The quality of government software seems somewhat better in several European countries than here in the United States. • Software produced by some of the civilian agencies of the Federal Government in the United States tends to lag the private sector in terms of defect prevention and defect removal operations, and hence gets deployed with substantially more errors than commercial norms.

16. The Ugly • By contrast, government-produced software in Belgium, Italy, France, Sweden, and the U.K. is sometimes equivalent to information systems software produced by public company or their outsourcing partners.

17. Differences • Requirements and design approaches are often more formal in Europe, and some of the European front-end approaches are technically very sophisticated • On the whole, formal methodologies are used more often than in the United States, and much more often in the information systems and client/server domains where U.S. practices tend to be rather careless. • The overall result is that European MIS software projects often run about 10% to 20% below equivalent U.S. projects in requirements and design errors.

18. Differences • The patterns of programming language usage tend to differ between the U.S. and Europe. • Logic-based languages such as ALGOL, Prolog, and Miranda have a higher frequency of use as do strongly-typed languages such as PASCAL and Modula-2. • The usage of Ada83 for civilian projects is far more common in Europe than in the United States. • Programming language research is very strong in much of Europe.

19. Differences • Usage of commercial software cost estimating tools and other software project management aids are less common in Europe than in the United States, although both sides of the Atlantic are now experiencing rapid growth in project management technologies. • Usage of function point metrics for data normalisation is roughly equal between the United States and Europe in terms of frequency of use.

20. Differences • Membership in various function point users' groups is growing at about 50% per year in every industrialised country, while usage of "1ines of code" is declining by about 10% per year. • In Europe, large function point associations have been formed in France, Italy, the Netherlands, the United Kingdom, and also a multi-country Scandinavian function point group. • Function point usage is growing but not yet formally organised in Austria, Belgium, Germany, Greece, Portugal, or Spain.

21. Differences • Although the factor is more important for productivity studies than for quality studies, the rather lengthy European vacation periods (running to more than 30 days per year in some cases) tend to favour countries with more working days per year, such as the United States, Japan, and India.

22. Differences • Although a difficult topic to explore since many tracking systems don't record it, the amount of unpaid overtime applied to software projects (i.e., unpaid work in the evenings, on weekends, and on public holidays) appears low in Europe and the United Kingdom compared to the United States and Japan. • Unpaid overtime is so often part of software projects that it can exert an overall productivity influence that approaches 20%.

23. Differences • One of the problems of doing direct international comparisons between the United States and Europe is that the metrics are sometimes different. • A survey of refereed software engineering journals carried out a few years ago noted that: • one-third of the articles using "lines of code" were based on physical lines; • one-third were based on logical statements; • the remaining third did not even state the basis of the lines of code count. • almost 20% of the articles did not even identify the languages from which the study was based.

24. Differences • Of course, there are function point variants too. • Between the United States and Europe, at least a dozen function point variations can now be enumerated. • On a global basis, however, standard IFPUG function points have about 90% penetration in terms of published books, articles, and data.

25. ISO 9000 (BURKS) • A set of international standards for both quality management and quality assurance that has been adopted by over 90 countries world-wide. • The ISO 9000 standards apply to all types and sizes of organisations. • Documentation is at the core of ISO 9000 • Say what you do. • Do what you say. • Write it down.

26. ISO 9000 • The standards require: • a standard language for documenting quality processes; • a system to manage evidence that these practices are instituted throughout an organisation; and • third-party auditing to review, certify, and maintain certification of organisations.

27. ISO 9000 • Say what you do. • Do what you say. • Write it down.

28. Used where? • The ISO standards are used intermittently in the United States, Japan and the Pacific Rim, South America, the Middle East, and Eastern Europe. • The ISO standards have had far and away their greatest deployment in Europe and the United Kingdom. • Indeed, they are now often required for marketing products within the European Community (EC).

29. Used where? • Not every company utilises the ISO standards in Europe, but the penetration is probably now in excess of 20% in Western Europe and the U.K. as opposed to perhaps 3% elsewhere based on interviews with software managers and QA personnel. • A few U.S. companies that produce only domestic software respond to questions about ISO standards with a question of their own: "What are they?"

30. ISO 9001 and ISO 9000.3 • ISO 9001 is the primary standard on how products are built, and a guideline • ISO 9000.3 explains the ISO approach in a software context.

31. But ... • “So far as can be determined, there is not yet any tangible or perceptible difference in the software quality levels of companies which have been certified when compared to similar companies in the same industries which have not been certified.”

32. More But ... • Currently the most tangible aspects remain • the high costs of the ISO certification process, • the large volumes of planning and control documents, and • the time required to achieve certification.

33. Motorola’s 2 Objections • ISO certification was used for marketing purposes with an implied assertion that achieving ISO certification meant high quality levels regardless of the lack of empirical data demonstrating that point. Motorola's Vice President of Quality, bluntly stated that was false advertising. • The ISO standards themselves lacked some of the key quality elements found in Motorola's own quality standards and could be viewed as a step back rather than a step forward in terms of quality methodology.

34. What? • Some ISO users have observed that it is not the place of the ISO 9000-9004 standards to actually improve quality. • Their purpose is simply to ensure that effective practices are followed, and are not in and of themselves a compilation of best current practices.

35. Does ISO 9001 ... • reduce software defect potentials below 5 per function point? • raise defect removal efficiency levels above 95% ? • reduce the number of had test cases below 15% ? • reduce the rate of "bad fix" injections below 7% ? • increase reliability under field conditions? • increase customer satisfaction levels? • have any affect on data quality? • reduce the number of cancelled or delayed projects? • reduce the incidence of litigation for breach of contract? • reduce the incidence of litigation for poor quality control?

36. Data? • Software standards groups should consider the methods through which medical and pharmaceutical standards are set. • Although medical standards are not perfect, their concept is that a therapy should be proven to be effective and its side effects understood before it becomes a standard. • It would be a strong statement that software is approaching the status of a real profession if software standards included information similar to medical and pharmaceutical standards on efficacy, counter-indications, known hazards, and potential side effects.

37. Data? • Software standards are normally totally published in a way that is totally devoid of empirical data regarding their effectiveness when the standards are first promulgated. • The fact that there may possibly be negative or harmful side effects from following the standards is usually not even discussed, although negative side effects are a major section of medical and pharmaceutical standards. • Standards organisations should collect data, and should encourage auditors and assessors to move toward quantification and empirical results rather than subjective opinions.

38. TickIT • TickIT is a method of evaluating software quality that originated in the late 1980s by the BCS. • The TickIT approach is based on ISO 9000.3 and is intended to serve as a consistency check on auditing ISO standards as applied to software. • The TickIT concept includes both training and certification of auditors, who are then qualified to perform ISO audits. • The TickIT approach includes on-site interviews and assessments and envisions the production of a number of quality-related plans and reports.

39. TickIT • TickIT is widely used throughout the United Kingdom, and especially so for projects involving the government. • Since the ISO standards did not originate for software, there is a strong need to customise or tailor the ISO standards and audits to the software world. The TickIT concept is to serve as a translator of ISO philosophy into the software domain. • Empirical data on the effectiveness of the TickIT approach is sparse. • The TickIT concept and the ISO standards ignore both functional metrics and defect removal efficiency measurement.

40. SPICE • "Software Process Improvement and Capability Evaluation." • The SPICE approach is a composite software assessment method based in part on • the Software Engineering Institute (SEI) capability maturity model (CMM); • the Bell Northern Research TRILLIUM assessment approach; and • the International Standards Organisation (ISO) 9000‑9004 standard series.

41. SPICE • The SPICE approach is moving toward an international standard for software process assessments that is intended to improve the relationship between software contractors and clients. • Many of the SPICE concepts are derived from the needs of large corporations and large government agencies whose software contracts might run to many millions of dollars for key contracts, and billions of dollars in aggregate.

42. SPICE • SPICE is a bold and ambitious program, and like many bold programs, also has some risks associated with it. • It is premature to judge the overall impact of SPICE since it is still evolving. • The direction is promising, but when so many organisations are involved in standards the results are often cumbersome and unsuitable for small projects and small companies.

43. Software development in 1984 • More than half of large software systems were over 12 months late. • The average costs of large software systems was more than twice the initial budget. • The cancellation rate of large software systems exceeded 35%. • The quality and reliability levels of delivered software of all sizes were poor. • Software personnel were increasing by more than 10% per year. • Software was the largest known business expense that could not be managed.

44. Software Engineering Institute • In order to explore software issues, and especially those topics associated with defence contracts, the Department of Defence in 1984 funded a new software research facility. • This was the essential origin of the Software Engineering Institute. • The SEI is located on the campus of Carnegie Mellon University in Pittsburgh. • It is a non-profit software research organisation funded primarily by the US government through the Defence Advanced Research Project agency (DARPA).

45. SEI Assessment Approach • The SEI assessment data is collected by means of on-site interviews using both a standard questionnaire and also observations and informal data. • Once collected, the assessment data is analysed, and used to place a software organisation on one of the five plateau of the now well-known SEI “Capability Maturity Model” • CMM

46. System Size? • The SEI assessment approach originally dealt primarily with the software processes and methodologies used by very large companies that produced very large systems. • It was derived from the best practices used by leading corporations such as IBM and ITT which employ from 5,000 to more than 25,000 software professionals, and which could safely build systems in excess of 1,000,000 lines of code or 10,000 function points.

47. SEI CMM

48. CMM • The Software Engineering Institute's model of software engineering that specifies five levels of maturity of the processes of a software organisation. • CMM offers a framework for evolutionary process improvement. • Originally applied to software development (SE-CMM), it has been expanded to cover other areas including Human Resources and Software Acquitition.

49. Focii - key process areas: Level - 1 Initial • Heroes • None.

50. Level 2 - Repeatable • Project Management – • Software Project Planning, Software Project Tracking and Oversight, Software Subcontract Management, Software Quality Assurance, Software Configuration Management, Requirements Management.