Advancing End-User Software Engineering: Challenges and Innovations

1. End User Software Engineering Mary Shaw Carnegie Mellon University

2. EUSE goes beyond EUP • EUP includes “programming-like” activities, such as programming, debugging, tailoring via parameters, simple scripting • EUSE goes beyond this … • Design and composition of systems from elements • Evolution, development, maintenance • Deliberate process for creating software • Expressiveness and understandability • Sufficient dependability for current need • Concern for system properties – usability, dependability, security, privacy • Note the process tension between “opportunisim” and systematic planning

3. EUs are not SEs • EUs do not have rich and robust mental models of their computing systems • they fail to do backups, misunderstand storage models, execute malware, and innocently engage in other risky behavior • EUSE is not “fixing the users” • EUSE must be about developing ways for EUs themselves to address traditional SE concerns

4. Papers that addressed (some of) these issues Brandt Brandt &al, Opportunistic Programming Costabile &al, End Users as Unwitting Developers Scaffidi &al, Topes for Validating and Reformatting Data Stoitsev &al, EUD for Enterprise Process / Info Mgt Wong &al, Patterns in Mashups Umarji &al, Teaching SE to End-users Coutaz, EUP and Intrinsic Complexity of … Shani &al, SW Design / UML / Domain Specific Lang Doerner &al, EUD: Challenges in SOA Sestoft, Function Spreadsheets Praehofer, EUP Environments in Automation Domain El-Gayyar, User-oriented Web Services Composition Constabile Scaffidi Stoitsev Wong Umarji Coutaz Shani Doerner Sestoft Praehofer El-Geyyar

5. Composition of systems from elements • Models for composition of elements • Use patterns to guide mashups [Wong] • Managing service compositions with many component proprietors [Coutaz] • Package new capability as plugins or extensions for existing systems [Scaffidi, Stoitsev, Sestoff] • Finding resources • Provide hierarchy of repositories from personal to local to global [Scaffidi] • Finding services [Doerner] Scaffidi Stoitsev Wong Coutaz Doerner Sestoft

6. Models for composition of elements • Wong, Patterns in Mashups • “Mashups” combine information from multiple web sites in new ways • Survey found several types: aggregation; alternate UI or specialized use; personalization; focused view of data; real-time monitoring • Large comparative table shows differences • Coutaz, EUP and Intrinsic Complexity of … • Model based on molecular chemistry shows richness of mixed physical-digital network artifacts • elements are digital or physical atoms • composition is based on, e.g., valence and affinity • Proposal: Base composition of services on this model Wong Coutaz

7. Packaging new capability as plug-ins, etc • Scaffidi&al,TopestoValidate,ReformatData • Newtechniqueformovingdataamongapplications • Integrates technique with existing applications by packaging it as plugins of various kinds • Stoitsev &al, EUD for Enterprise Proc / Info • New technique for EU tailoring of process models • Represented as hierarchical to-do lists, implemented as extensions to Outlook • Sestoft,ImplementingFunctionSpreadsheets • Added abstraction capability • Realized as a special kind of sheet Scaffidi Stoitsev Sestoft

8. Evolution, development, maintenance • Data sharing: interoperability across apps • Data from multiple applications must be combined [Stoitsev, Wong] • But data is often hard to share [Scaffidi, Doerner] • Evolution • Evolution includes dynamic changes [Doerner] • EU software evolves continuously [Costabile] • EUs need an evolutionary perspective [Umarji] • Tailoring • Individualized variants [Scaffidi, Stoitsev] • EUs tailor businesses processes [Stoitsev, Doerner] • Domain-specific tailoring for critical applications [Praehofer] • Personalizing workflows [el-Gayyar] Constabile Scaffidi Stoitsev Wong Umarji Doerner Praehofer El-Geyyar

9. Data Sharing Interoperability Problems • Scaffidi&al,TopestoValidate,ReformatData • Many kinds of data are carried in short strings with many formats: name, date, phone, address • Applications assume different format, and checking is binary, no “maybe” or special cases • Topes offer format conversion, validation in [0..1] • Doerner, EU Development: SOA Challenges • In SOA, problem for EU is gathering all the data for a decision from various sources, DBs • EUs want to export to Excel, but SAP can’t do this • Also can’t exchange data between applications Scaffidi Doerner

10. Evolution • Doerner, EU Development: SOA Challenges • EUs develop and evolve their business processes • They may not have enough info about process • Major challenge is adaptation at runtime • Costabile, EUs as Unwitting Developers • EUs often “program” without being aware of it • Their software and adaptations keep evolving • Solutions must extend through life cycle • Case study shows structured planning for this Constabile Doerner

11. Deliberate process for creating software Brandt • Shallow learning curve • People need immediate gratification [Brandt] • Gentle slope [Doerner, Stoitsev] • Opportunistic programming [Brandt] • Need rules and process, but not the same as for production SW [Brandt] • EU code is impermanent [Brandt] • Spectrum of users [Costabile] • EUs should know approaches to design and development, QA practices, evolutionary perspective, documentation, reuse [Umarji] • Some isolated Agile activities Constabile Stoitsev Umarji Doerner

12. Characteristics of EUs Brandt • Brandt: Opportunistic Programming • Opportunistic software: nontrivial software, little upfront planning, ease and speed of development prioritized over robustness and maintainability • Not just sloppy programming, but good process for prototyping, ideation, discovery • Build from scratch using high-level tools • Add new functionality via copy-and-paste • Iterate rapidly • Consider code impermanent • Face unique debugging challenges • Costabile, EUs as Unwitting Developers • EUs’ skills fall along a spectrum from EU to prof • Less-skilled often start with tailoring • Provide life cycle support, not just to release Constabile

13. Educating EUs • Umarji &al, Teaching SE to EUs • Surveyed bioinformatics curricula and recommended things that bioinformaticians should know about SE • Recommendations • Approaches to software design and development • Strong quality assurance (QA) practices • Evolutionary perspective • Documentation • Reuse Umarji

14. Expressiveness and understandability Brandt • Visual interfaces to tools • Define data formats with structured dialog and examples [Scaffidi] • Graphical composition [Stoitsev, El-Gayyar] • Visual models [Doerner] • Domain Oriented Visual Lang [Costabile] • Functional interface to visual tools to allow abstraction [Sestoft] • Chemical molecule metaphor for composition [Coutaz] • Use UML to create domain-specific languages [Shani] • People use informal cues to gauge quality of information [Brandt] Constabile Scaffidi Stoitsev Coutaz Shani Doerner Sestoft El-Geyyar

15. Questions for discussion • What are, and how can we resolve, the issues in the process tension between • “opportunism” / gentle learning curve / instant gratification / • systematic process / consistency / long-term usefulness • How can an EU manage evolution without a systematic process? Can we have a process with a shallow learning curve? • What are the potentials and problems for the technique of packaging a new capability variously as plugin, extension, macro in order to integrate it with various systems?