Inside an SAP business intelligence project, Part 2: Tips and tricks for execution and testing

2. Inside an SAP business intelligence project, Part 2: Tips and tricks for execution and testing Dr. Berg Comerit Inc.

3. In This Session … Tap into techniques for executing a successful SAP business intelligence project . Discover alternative project methodologies, such as Rapid Application Development (RAD), Joint Application Design (JAD), and Extreme Programming (XP), and learn why these methods are beneficial for deploying BI tools without having to write detailed functional and technical specifications. Delve into best-practice testing methods for queries, dashboards, cockpits, and ad hoc environments, such as unit, system, integration, and performance testing. Examine the execution strategies for three real-world SAP business intelligence projects and the critical lessons learned by the managers of those projects.

4. What We’ll Cover … • Introduction • Picking the Right methodology • Rapid Application Development (RAD) • Joint Application Design (JAD) and Extreme Programming (XP) • System Development Life-Cycle methodologies (SLDC) • Small, Medium and Large Implementations (real word examples) • The Value of Testing and a Formal Approach • Unit Test • System Test • Integration Test • The Support Organization • How to Write a Service Level Agreement (SLA) • Wrap-up

5. Rapid Application Development - A Short History Barry Boehm introduced the “Spiral Model” to reduce the risk of the projects. Each project is divided into critical parts and high risk areas was prototyped to drive the requirements and refine the deliverables with the users. In the late 1970s a method known as the Evolutionary Life Cycle (ELC) had been developed by Tom Gilb. There were no formal phases in a project. Each part of the project was developed with the users in sessions where feedback on prototypes were sought. In the 1980s Scott Shultz merged the Spiral Methodology and ELC and create Rapid Iterative Production Prototyping (RIPP) 1. RIPP = Spiral Model (SM), + Evolutionary Life Cycle (ELC) methodologies. 2. RAD = RIPP + SDLC methodologies.

6. What is RAD? In 1991, James Martin merged of best of RIPP and SDLC methods into a formal RAD methodology. What does RAD do? RAD compresses the phases of a SDLC and introduces an interactive approach for each of these phases. It also combines the design and construction stages into a single phase where the development process is interactive and not sequential. Each cycle of RAD is 1-21 days cycles and not phases

7. RAD as an Engine for NetWeaver BI Development The RAD idea: Traditional methodologies are too slow and rigid to meet the business demands of today’s economy. Developers chosen for RAD teams should be multi-talented "renaissance" people who are analysts, designers and programmers all rolled into one Teams should consist of about 6 people, including developers and users of the system, plus any stakeholder(Walter Maner) Small integrated teams - focus on small parts of the NetWeaver system (i.e. a set of InfoCubes, web reports or dashboard).

8. The First Session of RAD - How? RAD has a short blueprinting phase where meetings are executed in short succession to get requirements. The blueprinting & realization phase of the project are combined. The first meeting: 1-2 days uninterrupted time Who: Power users, casual users, people who today interact with the current system and managers who are stakeholders in the outcome. How many: A rapid pace is kept in these meetings and the number of attendees is kept at a manageable level, with typically no more than 20 people. Different needs: The coordinators and business analysts focus on shared information needs and conduct multiple sessions if needed. Why RAD? Increase involvement, less business disruption and opinions, more consensus, information sharing & an education event.

9. RAD - The Instruments and Approach The RAD sessions can use a "information request form" to gather the relevant information about reports, processes and information availability needed . Requirements and are consolidated, prioritized and used in follow-up discussions. Post all forms on the intranet and provide an easy way to communicate with the team. RAD is a spiral process that develops sets of prototypes Source: Dr T.H. Tse

10. RAD Approach For Smaller BI and Cockpit Projects In rapid Application Development (RAD), Keep the scope focused and use a simple approach: In- scope? Make enhancements Activate standard content Request for modifications Yes No Load InfoCube User acceptance session Test In-future scope? No Review data quality issues Deploy Create 2-3 sample queries Rejection In RAD, no functional or technical specs are used in this approach. Over 8-16 weeks multiple user acceptance session is used to refine requirements and multiple prototypes are built

11. RAD and Tools The key idea of RAD with NetWeaver is the rapid creation of reports and dashboards that users jointly develop. In the beginning these are often "mock-ups" with data from spreadsheet sin a sandbox environment. As the sessions progresses, tools such as query designer, WebI & dashboard builder are introduced and prototyping is done in each session with the users. RAD sessions are working sessions, not presentation sessions. Each session should therefore be 2-4 hours long (not at someone's cubicle). There should be at least 2-3 sessions each week to keep the work moving forward. Many critics of the SDLC methodologies have started to use development tools such as the Visual composer, Dashboard Builder (Xcelsius) and SAP web application designer to do interactive prototyping with the users.

12. Joint Application Design - JAD In the late 1970s, the first versions of Joint Application Design (JAD) were developed by Tony Crawford and Chuck Morris at IBM. JAD was first limited to user requirements gathering and sometimes designs of the applications. The JAD workshop was an alternative to the structured interviews that the SDLC methodologies advocated. The guiding principles of a JAD sessions are: 1. Keep it very focused 2. Conducted in a dedicated environment 3. Quickly drive major requirements and interface "look & feel" Remember: "A user sign off is a powerless piece of paper" when matched against the fury of top management"...

13. Who Participates? - JAD 1. Facilitator – Facilitates discussions, enforces rules 2. End users – 3 to 5, attend all sessions 3. Developers – 2 or 3, question for clarity 4. Tie Breaker – Senior manager. Breaks end user ties, don’t attend 5. Observers – 2 or 3, do not speak 6. SMEs – A few subject matter experts (SME) for understanding business & technology Keep it very focused and explore the interfaces. How do the user want to see the screen layouts and functionality? A study of 60 development projects and found that without JAD, 35% of the functionality was missed (Source: Caper Jones)

14. JAD - Benefits JAD helps to merge business knowledge with IT design, to provide useful interaction and improve the quality of the systems being developed. JRP, or Joint Requirements Planning, is an approach to focus on the whole plan and execution of getting the “right” requirements. JRP is often a supplement to the JAD sessions that is used for the specific program components and not a replacement for JAD. Initially JAD did not include the realization and the implementation phase. It was a set of workshops and not a comprehensive methodology.

15. JAD - Scalability Benefits The proponents of JAD claims that the advantages include a dramatic shortening of the time it takes to complete a project. It also improves the quality of the final product by focusing on the up-front portion of the development lifecycle, thus reducing the likelihood of errors that are expensive to correct later. The argument is that traditional methods have several built-in delay factors that get worse as more people become involved. Therefore, the traditional methods are not scalable to larger IT projects. JAD laid the framework for other more radical approaches such as Extreme Programming (XP).

16. Extreme Programming (XP) - Some background XP started in the 1990s as programmers decided that the requirements gathering sessions took too much time and often just verified what they knew. The XP argument: traditional methodologies were developed to build software for low levels of change and reasonably predictable outcomes. 1. The business world is no longer predictable, and software requirements change at extremely high rates. 2. Development can be completed faster with collaborative efforts of teamed programmers. 3. Phases exists, but development is interactive. Design work is subject to change during the construction. The core premise of XP is that you can only pick 3 out of these 4 dimensions: cost, quality, scope, time.

17. XP and The Release Plan and Refactoring the BI work XP's first piece of work is the planning. Now, user stories are written from a user interaction standpoint. The project development schedule is the result of the software release plan which consist of smaller go-lives. The momentum of the project, known as the “project velocity” is measured. The project is divided into smaller iterations. People on the project may be reassigned to different work in the various iterations. A stand-up meeting starts each day and re-planning occurs based on yesterday’s work progress

18. XP and the Blueprinting the SAP BI Solution The guiding principle of the blueprint is simplicity and meetings are short interactive design sessions. Stand-alone simple solutions are encouraged to reduce risk and only essential functionality is included. Any other functionality is added later, based on available time during the realization phase. The key to a successful design under XP is to refractor the work whenever and wherever possible The trick to stay on schedule is to focus on the bare minimum. The "nice-to-have" items are included in the realization phase if time allows.

19. XP - The Realization Phase Now the customer has to be present. Also, to assure quality, all BW work must meet agreed to standards and developers are responsible for unit testing. A unique feature of XP is that all development is done in pairs and only one pair of programmers can integrate their config into the production box at any given time. The system is collectively owned by the developers and final performance optimization is done after all the configuration is integrated. Whenever a bug is discovered, a test case is created (not before!) to verify the extent of the error and possible impact. The testing stage relies heavily on acceptance testing and multiple sessions are created for each iteration and for each software release.

20. XP for SAP BI - The Critics Many critics have raised concerns with the XP approaches: 1. Reliance on verbal communication which may lead to “finger pointing” when things go wrong. 2. It is based on intuition-based decision making and a high degree of dependency on individual skills. 3. Insufficient planning and does not contain any sound approach to system verification and validation. Some claim that XP leads to rework as a norm, error prone systems and non-maintainable systems that finally leads to frustrated staff, and few heroes.

21. XP - Limitations • XP does not work in “Dilbertesque” companies (companies with high degree of control for the sake of control). • XP does not work easily in groups of more than about 20 programmers. • It is hard to implement in environment where there is a high degree of commitment to existing data warehouses or reporting systems. • XP is hard to implement in outsourced development environment when programmers are separated by space XP is extremely popular among developers. It is often used to retain and motivate the IT staff

22. Source: Dr T.H. Tse The "Waterfall Methodologies"- SDLC The SDLC methodologies are known collectively as "waterfall methodologies". Some criticism: It gives a false sense of clear cut stages and does not address changes during development. It is hard to fix mistakes or missing functionality during integration test. What do you do when you find new critical requirements 2 weeks before go-live? The waterfall

23. System development lifecycle methodologies - SDLC SDLC are traditional methodologies from the 1970s, and widely used today. They are based on a step-by-step approach to system development. This forces users to “sign-off” after the completion of each specification before development starts. There is a long delay before the customer gets to see any results and the development can take so long that the customer’s business may fundamentally change before the system is ready. This has been the area of highest contention among the critics of the SDLC methodologies. SDLC methodologies have 5 phases: analysis, design, development, testing & implementation. SAP refers to these phases by different names.

24. Getting the NetWeaver BI requirements - SDLC Avoid creating a total inventory of all reports in the organization. The "top-5" (most used) sales, distribution, inventory etc. reports from each department will cover the vast majority of the reporting needs. Create structured interviews with individuals that have a stake in the outcome A single BW "report" may satisfy dozens of today's static reports. It is therefore impossible to map each individual legacy report to a single query. Avoid attempting to replicate each report based on what you might have in place today. Accept new ways of accessing data.

25. Examples for Accelerators: Project Plan, Estimating • Fill in the Blank Fill in the Blank Design Strategies, Scope Definition • vs. Versus Start from Scratch Documentation, Issues DB • Start from Scratch Workshop Agenda • Questionnaires • End - User Procedures • Test Plans • Technical Procedures • Made Easy guidebooks (printout, data transfer, system • administration…) What is ASAP? Source: SAP

26. The SAP NetWeaver Quality Workflow - SDLC / ASAP

27. Framework for picking your "poison"

28. The Gray areas of Methodologies There are in fact several dimensions when multiple methodologies can be employed. I.e. when time to delivery is moderate, or when the impact of failure is moderate. The framework is intended to illustrate the differences among the appropriateness of each methodology. This decision is clearer in the extreme. However, in reality there may be “gray zones” where more than one answer may be correct.

29. What We’ll Cover … • Introduction • Picking the Right methodology • Rapid Application Development (RAD) • Joined Application Design (JAD) and Extreme Programming (XP) • System Development Life-Cycle methodologies (SLDC) • Small, Medium and Large Implementations (real word examples) • The Value of Testing and a Formal Approach • Unit Test • System Test • Integration Test • The Support Organization • How to Write a Service Level Agreement (SLA) • Wrap-up

30. Top 25 global brands = ¾ of Unilever’s sales A Large Company Example - Unilever • Unilever has 160 million sales transactions each day and over 80 global brands. This creates very large volumes of data and (see Ram Cheeti's presentation from Unilever here at the conference)

31. Unilevers Scale and Geographic Reach Western Europe | $16.7bn | 30% The Americas | $17.8bn | 32% Asia Africa CEE | $20.8bn | 38% 2009 Unilever Turnover $55.3bn | 100%

32. Evolutionary Complexity - North America (Pre-2005) • Pre-EDW NA architecture • Complex landscape with many point-to-point interfaces • Existence of independent datamarts Target System1 Target System2 Target System3 Target System 4 Target System1 Target System2 Target System3 Target System4 DW Data Mart DW Source System1 Source System3 Source System n Source System1 Source System3 Source System n Source System2 Source System2 Division 1 (HPC) Division 2 (Foods) DW = Data Warehouse

33. System Rationalization (2007-Current) • NA architecture with SAP NetWeaver BW as EDW • Centralized • Data harmonized, standardized, and qualified • Data marts: Dependent and sourced from EDW • 2000+ users leverage BW reports and external BI applications Target System1 Target System2 Target System3 Target System4 Target System5 Target System6 Target System7 Target System n Sales Reporting SC Reporting EDW SAP BW BI Data Mart BI Data Mart Source System 1 Source System 2 Source System n Source System 1 Source System 2 ECC

34. The Large Organization - Lessons Learned • Limit the number of interfaces to legacy reporting systems. It is expensive to maintain multiple data warehouses. • Have a replacement strategy for a centralized DW as soon as possible. • "Rome was not built in a day", but taking more than 3-5 years to rationalize systems is too slow. • "Burn-the-Boats" is a great strategy. If a system is allowed to run in parallel, the new system will never be used.

35. Unit Logistics Material Billing information Currency Key Plant Unit of Measure Material number Shipping/receiving point Base unit of measure Material entered Billing document Sales unit of measure Material group Billing item Volume unit of measure Item category Billing type Weight unit of measure Product hierarchy Billing category EAN/UPC Billing Billing date Creation date Cancel indicator Number of billing documents Output medium Customer Number billing line items ~ Batch billing indicator Billed item quantity Debit/credit reason code Net weight Billing category Sold-to Subtotal 1 Reference document Ship-to Subtotal 2 Payment terms Bill-to Subtotal 3 Cancelled billing document Payer Subtotal 4 Davison for the order header Customer class Subtotal 5 Pricing procedure Customer group Subtotal 6 ~ Customer country Subtotal A Document details ~ Customer region Net value ~ Customer postal code Cost ~ Customer industry code 1 Sales order document type Tax amount End user Sales deal Volume Sales document Organization Company code Accounting Personnel Time Division Distribution channel Cost center Sales organization Sales rep number Calendar year Profit center Sales group Calendar month Controlling area Calendar week Account assignment group Calendar day LEGEND Delivered in standard extractors Delivered in LO extractor Not in delivered Content - but in R-3 Model the solution - real example 1. Create a model based on pre-delivered SAP NetWeaver BI content 2. Map your data requirements to the delivered content, and identify gaps 3. Identify where the data gaps are going to be sourced from Data Requirements + Standard Content Storage Objects Map functional requirements to the standard content before you make enhancements

36. Tracking Load Performance - A Real Example • A stabilization period after each go-live is normal, until the new process chain has been tuned in the production box • This is a time when active monitoring of process chains should occur Production Areas of BI Data Load Issues Performance Nov. 1st through Dec. 15th Demand 7 Planning Transaction - global 6 Source - Purchase 5 Orders Roughcut 4 Material Movements Number of Issues MD - Bev. 3 Packaging Master data 2 Hierarchies 1 Greycon 0 CO -line items 12/1/04 12/2/04 12/3/04 12/5/04 12/6/04 12/8/04 12/4/04 12/7/04 12/9/04 11/1/04 11/2/04 11/3/04 11/4/04 11/5/04 11/6/04 11/8/04 11/7/04 11/9/04 11/29/04 11/30/04 12/14/04 12/15/04 12/10/04 12/11/04 12/12/04 12/13/04 11/14/04 11/15/04 11/17/04 11/18/04 11/20/04 11/21/04 11/26/04 11/10/04 11/11/04 11/12/04 11/13/04 11/16/04 11/19/04 11/22/04 11/23/04 11/24/04 11/25/04 11/27/04 11/28/04

37. Data Warehousing and BI in Smaller Organizations It is hard for smaller organizations to be able to afford large BI and DW groups. Some examples: Smaller organizations often leverage out-of-house resources and keep a smaller team in-house.

38. What We’ll Cover … • Introduction • Picking the Right methodology • Rapid Application Development (RAD) • Joined Application Design (JAD) and Extreme Programming (XP) • System Development Life-Cycle methodologies (SLDC) • Small, Medium and Large Implementations (real word examples) • The Value of Testing and a Formal Approach • Unit Test • System Test • Integration Test and Performance Testing • The Support Organization • How to Write a Service Level Agreement (SLA) • Wrap-up

39. The SAP NetWeaver BI Test Methodology • Methodology used for system and integration tests Test Strategy Test Plan Test Execution Problem Resolution ERP and BI testing is not different from a methodology standpoint, but the global execution is different.

40. SAP NetWeaver BI Test:Planning Activities “There's no time to stop for gas, we're already late.” • Business analysts are responsible for planning, coordinating, and executing the system testing of queries.

41. SAP NetWeaver BI Test Scheduling: Example • Each team should have dedicated time in the test room • If needed, rent your own training/test room • Provide food and snacks • At least two testers (preferably three) should be assigned to test each functionality • All test results must be logged

42. SAP NetWeaver BI Test: Checklist • Preparations • Data source/cubes/ODS/queries prioritized for testing • Queries developed and available in the SAP NetWeaver BI test environment • Track specific test plans created using test template • Test cases written • People • Individuals (testers) perform the identified tests • Testers invited to complete SAP NetWeaver BI on-line training • Availability of testers confirmed • Security roles tested and user IDs for testers have been created • Logistics • Testers familiarized with test results recording tools • Identify test location and verify resources • Rooms, computers, SAPGUI, network connections, phone, etc. • Plan for problem resolution

43. Performance Testing • Performance test execution • Identify queries to be performance tuned, and determine cutoff load for load test (e.g., 40% of actual users — not named) • Schedule queries to run in background, and execute each query while load scripts are running to simulate “real” users • Monitor your system continuously, and attempt tuning at the query level • Perform analysis based on benchmarks, and build aggregates and/or indexes • Record findings in a formal tracking tool available to everyone • Meet with developers daily to discuss issues • Problem resolution Look at the BW Accelerator for improved performance ! Source: Alexander Peter, SAP AG

44. What We’ll Cover … • Introduction • Picking the Right methodology • Rapid Application Development (RAD) • Joined Application Design (JAD) and Extreme Programming (XP) • System Development Life-Cycle methodologies (SLDC) • Small, Medium and Large Implementations (real word examples) • The Value of Testing and a Formal Approach • Unit Test • System Test • Integration Test • The Support Organization • How to Write a Service Level Agreement (SLA) • Wrap-up

45. BI Support Organization — Big Picture You need to separate the operations of BI systems from the project work If there is no support organization, the BI system quickly becomes an orphan when the project ends Without a support org. there is a risk that future BI projects are delayed since the project team has to support previous projects

46. An Example of a Small Support Team Note: These are Support models and does not include any resources for new content development. This is assumed to be staffed separately. This small group is typically folded in under an existing manager, who devotes only part-time efforts to BI support The power user is normally also situated in a different organization Support leader Part time (50%) Power User Part time (25%) SAP BI Basis Full-time/part-time Helpdesk & data loads Full-time

47. An Example of a Medium Support Team This medium sized team is typically folded in under an existing manager, who devotes only part-time efforts to BI support The group sometimes also undertakes portal support, security, development standards, and feature enhancements such as broadcasting and cockpit consolidations; but is normally not extensively involved in new content development Support leader Part time (65%) Data loads & fixes Full-time SAP BI Basis Full-time Help desk, training, user support Full-time Query & Web Full-time

48. An Example of a Large Support Team This large team can support complex applications, cockpits, BI portals, and broadcasting while providing training and help desk support as well as on-going SAP NetWeaver BW production support. Note: Job areas are meant for illustrations, and will vary depending on the BI applications supported Support leader Full time Data loads & fixes Full-time SAP BI Basis Full-time Helpdesk, user support Full-time Query & cockpits i.e. SEM Full-time Query & cockpits i.e. APO, CRM Full-time Data loads & fixes Full-time Training, user support Full-time Data quality & data resource mgmt. Full-time Portal, collaboration, KM security Full-time

49. Hint: Break-Fix for Production Environment By Introducing a Break-Fix (BWB) environment, the support team can correct break-fixes and move code into the Testing environment (BWQ) and Production environment (BWP) without impacting the project team Transports can be captured in the buffer and moved to the Development environment (BWD) on a periodic basis Break fix and Production stack The Break-Fix and production stack as well as the training environment is owned by the support team. The project teams own the development and Sandbox environments (BWS and BWD). BWP BWB BWQ Project Stack Training BWD BWT BWS

50. Internal and External Rewards for the Support Team • While the compensation can vary by regions, and salaries have been revised downwards in 2009-20010, the typical support costs are: • Money is not the only compensation. Popular rewards include: • Extra week vacation for people in support roles • One week SAP training of choice each year • Clearly defined promotion path (given in writing) • Reduced work hours (7 hr workday) • Remote support from home 1-2 days per week