Unsustainable Software Development and its Causes

1. Unsustainable Software Development and its Causes Lecture 1CIS 6101 – Software Processes and Metrics

2. Outline • 1. Background of Unsustainable Development • 2. Causes of Unsustainable Development • 3. Project Controls • 4. Summary

3. 1. Background of Unsustainable Development

4. Definition • Unsustainable development is characterizdby constantly increasing the cost of change to the software. • Evidence of a high cost of change is a constantlyincreasing number of defects. • How does this happen? <Discuss> • Each change • adds complexity and • uncovers or causes of defects that require more changes. • This complexity leads to a declining ability to respond to customer requests.

5. Fixes versus New Features • Unsustainable development is all too-common today in software industry. • We place too much emphasis on • short term feature development (new) and • fixing defects (existing) and • NOT paying attention to the health of the underlying software. • Result: Cost of change rising with new changes coupled with the risk where each change can destabilize the product. • <Discuss: What does it mean to ‘destabilize’ the product?>

6. Fixes and Stability of Software • Here, teams spend more and more time fixing defects and in stabilizing software. • Features • Do get developed, but • Less time available due to time for stabilizing the software. • Development teams want to freezerequirements • But this reduces ability to respond to client requests. • Newer methodologies embrace customer involvement!! • So, much time spent on stabilization and not on new work.

7. Threshold Reached – When? • When team spends much time on defect backlog and not enough time developing new features. • When: occur within a few weeks, months,years. •  Finger pointing and blame game. • Consequences: • Morale down; • job layoffs; • outsourcing; • poor customer satisfaction. • Most teams, however, do not live with unsustainable software but get uncomfortably close. • <Discuss: CMDS; the methodology (process); reactions!>

8. TechnicalDebt • Underlying cause of inability to develop new features due to defect burden is attributable to technical debt. • Cause of Technical Debt: caused by build up of poor decisions over time • Cause of Technical Debt: decisions that seem right but often made for short-term fixes. • <Discuss: How does this happen?> • Rarely a single decision is the cause.

9. The Perils of Jumping in Place • Evidence of tech debt is clear. • When the number a products must be completely rebuilt to keep up with competitive products or • When products need to be rebuilt to respond to major technology changes. • Rewrites require • massiveinvestments in time and effort and • often results in loss of market share while the replacement product is being built. • Often impossible to recover market share where competitors get stronger and new competition enters marketplace.

10. 2. The Causes of Unsustainable Development

11. The Causes of Unsustainable Development • Why is unsustainable development so common? • A number of reasons called Product Stresses • There are things we cannot control (project stresses) and, • There are a number of things we can control (project controls). • Different for every project; stresses change • <Discuss Stresses without looking ahead…> • Let’s consider Project Stresses.

12. Causes of Unsustainable Development – External Dependencies • Dependencies on versions of the OS, compiler, IDEs, web browser components, etc. • Impossible to dictate at the end of a development environment what the dependencies might be. • Must often develop to support both old and new interfaces, such as two different versions of IE, netBeans, etc…

13. Causes of Unsustainable Development - Competition • Very easy to set up a software development company nowadays. • Need few bright people, a few computers, and Internet • Many do not have • a workable business plan and • a clear idea of their true market worth due to overestimating • do not know their marketsize. • Almost always suffer from inability to adequately sellor market their work. • Usually get initial funding and make some initial sales, • but struggle to stay in business.

14. Causes of Unsustainable Development - Disruptive Technologies • Constant advances in state of the technology can be disruptive. • Products need to meet users’ demands over the long term. • Cannot restrict thinking for ONLY what the customer wants, because customers may initially be interested in immediate needs. • But customers may be offered morefeatures on newertechnology for lessprice • The Internet is a disruptive technology • more and more support can easily be offered over the Internet. • Companies must learn how to deal with the Internet and its threatstoo (illicit software, hackers, a forum for malicious people to exploit security flaws, and more.)

15. Causes of Unsustainable Development - Disruptive Business Models • ConsiderDell: They have a disruptive business model. • Their model allows them to produce computers on demand by • clustering their suppliers together and • taking all orders over the Internet. • Other companies (HP, Compaq, IBM) have not been able to keep pace or duplicate Dell’s business. • Open Source software. A disruptive role! • Open source is ‘free,’ and usually usable by highly technical people. • No question that open source will play an increasingly disruptive role in the software industry

16. Causes of Unsustainable Development - Disruptive Business Models • Microsoft, while having a monopoly on desktop market, cannot compete against free software. • Linux: can its business model be sustained by keeping itself going almost exclusively through support and services as claimed.

17. Causes of Unsustainable Development - Cost Management • Software Projects are exceedingly expensive! • Difficult to control costs through greater efficiency due to mindset of the organization, its processes, training, deadlines, etc. • Best way: employ less expensive people. • This explains the increasing use of outsourcing. • But efficiency is at the heart of sustainable development.

18. 3. Project Controls There are things we can do and we can control.

19. Project Controls – Collaboration • Must work with other people within the organization • May involve others • with similar development projects or • With a history of similar projects • Must work with your customers!! • May involve end-user representatives on staff

20. Project Controls – Methodology • Development methodology – the principles (mindset) and practices of planning, design, testing, coding, etc.) has a dramaticimpact on the success of a project. • Methodology directly influences • efficiency, • change tolerance, • responsiveness, and • software quality. • <Discuss these bullets>

21. Methodology – Plan Driven Development - 1 • Some stick to a plan-driven methodology (which works well sometimes) because this is what is taught in school especially to engineering. • If this doesn’t work, they may use a variant such as code-then-fix approach. • Here, teams spend lots of time writing software and very little time on planning and analysis. • Software is brittle and will resist change. • Lack of sound practices leads to a great deal of dead code and thrashing.

22. Methodology – Plan Driven Development - 2 • Plan-driven development, derived from engineering, is best typified by the ISO 9000 set of standards. <Good research topic> • Downside of plan-driven approach • some software developers feel all the complexities of the software can be understood before any code is written. • This is rarely true and definitely false over the long term. • <Discuss: When might this be true?>

23. Plan Driven Development - 3 • While documentation of requirements, design, test plans, etc. have value as a means of communication, their realvalue comes not from the documentation but rather from the process one must go through to produce documentation. • Working though a design with your peers is a • valuable exercise; • fosters collaboration, understanding, and creativity; • The document is of less value and must be maintained. • <Discuss: is the document used?> • <Discuss: “must be maintained?”>

24. Plan Driven Development – 4Negatives on Documentation • Work gets done; but result is too often time spent producing and maintaining the documents detracts from amount of time spent on analysis. • On the other hand, if ‘maintained’ these represent a living, real set of specs, design, maintenance manuals… • Plan-driven software does not work for software development because it is impossible to put together a plan when the project starts to take into account all the changes in requirements and in the environment that will occur over the course of the project. <Discuss: Is this really true??>

25. Plan Driven Development - 5 • Plan-driven approach stiflesagility, because • They emphasize ceremony and • Assert documentationover having a workingproduct in the erroneous belief that somehow having a pile of documentation describing requirements and design is proof that • That the project cannot fail, • that a discipline is in use, and by extension, • That a project without the depth of documentation does not have discipline. • What is really needed is/are working projects in user’s hands such that feedback is produced, since this is the only real way to know if the desired system meets user needs.

26. Project Controls – Expertise • This will be a strongindicator of success. • Will be able to understand what is really important and what is not.

27. Project Controls – Decision Making • The ability to make decisions in • a complex and in constantly changing environment • often with incomplete information is crucial to success. • An important component of this is prioritization, where you chose what must be worked on and when.

28. Project Controls – Leadership • Helps to set up and communicate a vision, strategy, and tactics. • Keep projects on track.

29. Project Controls – Culture • Organizational culture shapes the attitude and practices of the people who work on a project. • Culture defines the requirements • for leaders, and • for how decisions will be made and by whom, and the methodology. • If new leaders or a decision-making style is required, then the culture must change.

30. Project Controls – Simplicity and Reliability - 1 • Most software today is • overly complex, • hard to use and learn, and • has too many features for most of its users. • Think about word processors: – have features that most users will never use. • Interesting: Word2007 had 265 functions; • only 12 were used by more than 75% of its users; • 42 were not used at all! • Ill-effects of unnecessary complexity on users? Hard to say. •  Complexity is behind at least part of the outsourcing trend because one way to manage complexity is to get someone else to do it for you.

31. Project Controls – Simplicity and Reliability – 2 • Huge disparity between what consumers want and what companies would liketo sell them. • Too many companies still compete on features and price alone. • Often encouraged • by the most technical users who are the early adopters of any technology and • by the media, who still publish reviews that contain feature comparison charts.

32. Project Controls – Simplicity and Reliability – 3 • Related to simplicity is the need for reliability. • Too many software organizations emphasize features over reliability, as many users refuse to buy the first version of any software, and • Users are frustrated by versionupgrades that breakorremovefeaturesrelied upon in previous version. • There’s not enough emphasis placed on simplicity and reliability. • Consider the iPod or the Palm PDA. Successful because they do few things very well. • There was a great deal of restraint in their development. • Easy to add features, but difficult to show restraint to keep them out.

33. 4. Summary - 1 • Unsustainable development is a development pace that is typified by stress, frustration, and a sense of not being in control. • Evidenced by continually increasing the cost of change and defect rate and corresponding decreasing ability to respond to changing conditions. • Unsustainability may not be apparent to the people on the project due to the varying pace of decline. But it is there! <Discuss: It is really there. No time to see it> • Some unsustainable development comes from continual change and stresses that are out of control of the organization.

34. 4. Summary - 2 • Let’s look at the principles and practices and outline some key principles that leads to sustainability. • We have the principles and these are used to frame the practices that reinforce them.