testing and go live best practices for a successful dashboard rollout l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Testing and go-live best practices for a successful dashboard rollout PowerPoint Presentation
Download Presentation
Testing and go-live best practices for a successful dashboard rollout

Loading in 2 Seconds...

play fullscreen
1 / 48

Testing and go-live best practices for a successful dashboard rollout - PowerPoint PPT Presentation


  • 181 Views
  • Uploaded on

Testing and go-live best practices for a successful dashboard rollout. Dr. Berg Comerit Inc. Background Planning for the Xcelsius Dashboard Initiative The JAD, RAD, Agile (XP) or ASAP methodologies Getting the Right Dashboard Requirements Upgrade and Tool Migrations

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Testing and go-live best practices for a successful dashboard rollout' - tarak


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
testing and go live best practices for a successful dashboard rollout

Testing and go-live best practices for a successful dashboard rollout

Dr. Berg

Comerit Inc.

what we ll cover
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
go live planning for the xcelsius dashboard initiative
During the planning phase you should also start planning your go-live strategy. This include answering the following questions:

Where are my users located?

What is the network capacity?

Do I need support people for extended time periods - different time zones?

Do I need multi-currency support on my dashboards?

Do I need multi-language support?

What type of users do I have in each region?

Go-Live Planning for the Xcelsius Dashboard Initiative

Create a user map as part of your project planning. This will help you understand your user base

user training options
There are four core options for the training strategy:

Classroom training

Best when users are similar and centrally located

On-line training

Best when users are dispersed, dashboards are simple or go-live over long time period

Train the trainer

Best when users resides in many locations, multiple languages are involved and when there is a very high number of users

One-on-one training

Best for executives and senior management. Should be

done at each user's office.

User Training Options

Source: dashbaord insights, 2009

Communicate and schedule training early in the project, so that everyone will be available

user training for complex dashbaords
If users need training you have often failed to create a good dashboards.

However, there are times when training are needed this include:

Interactive dashboards and dashboards with complex graphing

User Training for Complex Dashbaords

In this dashboard, users can budget for travel categories, for each month, and also save scenarios.

Some training is therefore required and should be planned early.

plan for an on line help system for your dashboard go live
On-line help should be created for each dashboardPlan for an On-Line Help System for Your Dashboard Go-Live

The on-line help system should explain

  • how number are calculated,
  • how to read graphs
  • what functionality is embedded
how to create on line help systems
Some ways to create a low-cost on-line help system Include:

Flash files

Create a simple help dashboard & embed this into the overall dashboard

Word

Create a word document with screenshots.

Save it as HTML and store the web pages on a web server.

You can then link the URL on your dashboard.

Custom Application

Use a tool like front-page or any web authoring tool and create a complex on-line help system with menus, search functionality, movies that shows demonstrations etc.

How to Create On-Line Help Systems

On-line help centers can include contact information, training schedules. They can also be used to communicate information on future projects

what we ll cover8
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
the jad rad agile xp or asap methodologies
There are several options for the Xcelsius Dashboard project

Joint Application Design (JAD)

Rapid Application Development (RAD)

Agile, or Extreme Programming (XP)

Accelerated SAP Methodology/ System Development Life-Cycle (SLDC)

Many of the methodologies are not appropriate for the dashboard development effort

The JAD, RAD, Agile (XP) or ASAP methodologies

Pick your Xcelsius methodology carefully.

Do not use ASAP unless you project is part of a budgeting, consolidation or planning effort.

the waterfall methodologies are not good for dashboards
The System Development Life Cycle (SDLC) methodologies, such as ASAP, are known collectively as "waterfall methodologies".

They give a false sense of clear cut stages and does not address substantial functionality changes during development. It is hard to fix missing functionality during integration test.

The "Waterfall Methodologies" are Not Good for Dashboards

The waterfall

The challenge with ASAP is that users don't know what they want until they see it...

joint application design jad who participates
1. Facilitator – Facilitates discussions, enforces rules

2. End users – 3 to 5, attend all sessions

3. Developers – 2 or 3, question for clarity

4. Tie Breaker – Senior manager. Breaks end user ties, don’t attend

5. Observers – 2 or 3, do not speak

6. SMEs – A few subject matter experts (SME) for understanding

business & technology

Keep it very focused and explore the interfaces. How do the user want to see the screen layouts and functionality?

Joint Application Design (JAD) - Who participates?

A study of 60 development projects and found that without JAD, 35% of the functionality was missed

(Source: Caper Jones)

rapid application development rad
RAD has an abbreviated blueprinting phase where meetings are executed in short succession to get the requirements. Most of the blueprinting and realization phase of the project are combined.

The first meeting: a one or two days work session with

uninterrupted time

Who: Power users, casual users, people who today interact with the current

system and managers who have a stake in the outcome of the dashboards

How many: A rapid pace is kept in these meetings and the number of attendees is

kept at no more than 7 people in attendance.

The coordinators should focus on shared information needs and conduct multiple sessions (typically once a week)

Rapid Application Development - RAD

Why RAD?.. Increase involvement, less business disruption, less opinions, more consensus, information sharing and an education event.

agile and extreme programming xp for xcelsius dashboards
The argument for XP is that other methodologies were

developed to build software for low levels of change and reasonably predictable outcomes.

But, the business world is no longer very predictable, and software requirements change at extremely high rates.

Development can be completed faster with collaborative efforts of

paired programmers with small 'sprint' timelines and many go-lives'.

Agile and Extreme Programming (XP) for Xcelsius Dashboards

XP was started by programmers who decided that the traditional requirements gathering sessions took too much time and often just verified what they already knew.

The core premise of XP is that you can only pick 3 out of these 4 dimensions: cost, quality, scope, time.

framework for picking your dashboard methodology
Framework for picking your Dashboard Methodology

Source: Dr. Berg,

DM Review, 2006

what we ll cover16
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
the xcelsius business requirements
Business requirements can be collected in a variety of ways based on the methodology that the company employs. It is a complex process and involves a period:

Discovery and Education,

Formal communication,

Prototypes and Reviews

Final approvals.

The Xcelsius Business Requirements

A dashboard implementation does not simply involve a series of black-and-white technical decisions; just because something is technically feasible does not mean it is wise or desirable from a business perspective.

Source: Gooy_GUI, 2007

slide18

Where to you start? - First Alternative

  • You can start with start with a blank template and fill-in the capabilities
  • Focus on graphs, layout, measures & navigation.
  • One method is to write story-boards from a user perspective and add needed functionality to support this
slide19

Where to you start? - Second Alternative

  • Get a group of 5-7 people for a brainstorming session.
  • Draw the solution, knowing that it may look somewhat different once developed.
  • Focus of the use of space, graphs, navigation, available data and the purpose of the dashboards.
  • Do not design fixed format 'reports'
building a mockup in excel
If you can make a 'mockup' in Excel, users can see what it may look like very quickly. You do not need to have any BOBJ tools installed. Building a Mockup in Excel

This can be done in 30-60 minutes

prototyping the dashboard requirements
Once the first day of brainstorming is completed, you can create data in Excel and prototype the solution in Xcelsisus. Focus on layout, space management, colors and basic formatting.Prototyping The Dashboard Requirements

Plan for multiple weekly prototypes before you get the solution that everyone can agree on.

flexible options to meet many requirements
There are often disagreement on how to present numbers and graphs.Flexible Options to Meet Many Requirements
  • Make your dashboard flexible and present data in many interactive ways.
  • Amount Vs. Percentages
  • Different graphs
  • Users can select what they want graphed
what we ll cover23
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
tool upgrades and deployment of new xcelsius functionality
When you upgrade your Xcelsisus dashboards consider a two step approach:

Conduct a developer training or workshop session to learn the new functionality and agree on the upgrade timeline

Do a technical upgrade over the weekend

Copy all of the dashboards

Implement new functionality on the copied dashboards

Have a formal feedback session with user involvement to see how they like the changes

Implement the new changes 2-6 weeks after the technical upgrade

Tool Upgrades and Deployment of New Xcelsius Functionality

Stability is the key to success in all end user systems. Even if the new functionality is better, users do not like a system that changes look and feel frequently.

tool migration strategy
If you are migrating users from a legacy reporting tool to your new dashboards you need a formal migration strategy. This could include:

Maintaining two systems (not recommended)

Running two systems in parallel for short time to reconcile results

Remove legacy reporting tool as part of go-live (recommended)

When Vikings settled new lands, they always burned the boats so that the settlers was 100% committed to the new situation. While it caused some anxiety, this is a great migration strategy that can be used for system migration efforts as well.

Tool Migration Strategy

A "burn the boats" reporting migration strategy assures high commitment to the new solution and possibly a high number of new requirements after go-live. Be prepared to support this...

what we ll cover26
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
the dashboard test methodology
Dashboards testing should follow a formal methodology The Dashboard Test Methodology

Test Strategy

  • The test strategy is written at the beginning of the project and should include:
    • What will be tested
    • Who will test it and approve it
    • Where will the test occur
    • When will the test be scheduled

Test Plan

Test Execution

Problem Resolution

The test plan is more detailed and should be written once the first prototype is built

the xcelsius user acceptance testing uat form
By requiring each of the 5-7 UAT members to complete a form for each dashboard, you get solid feedback that you can use in the next RAD development cycle

During each UAT test cycle, you should solicit detailed feedback on layout, graphs, theme, tables and navigation.

The Xcelsius User Acceptance Testing (UAT) Form

This form is also on your memory stick

the xcelsius load testing form
The load testing is intended to show any performance issues prior to the go-live.

While all areas cannot be tested, this will give you an idea on bottlenecks.

For large scale go-lives, you should consider having test PCs in multiple locations

The Xcelsius Load Testing Form

This form is also on your memory stick

the xcelsius stress testing form
Stress testing is very similar to load testing

The difference is that the number of concurrent users are expected to be doubled

Not all companies will use a stress test, nor pass it. This is due to the unrealistic concurrent load and the high hardware costs of meeting them.

However, it provides very useful information

The Xcelsius Stress Testing Form

This form is also on your memory stick

xcelsisus dashboard test planning
Key Activities:Xcelsisus Dashboard Test Planning
  • The business analysts are responsible for planning, coordinating, and executing the testing of dashboards
large scale xcelsisus test scheduling example
Each UAT team could have dedicated time in the test room

Provide food and snacks

At least two testers (preferably more) should be assigned to test each functionality

Large-Scale Xcelsisus Test Scheduling: Example

All test results should be logged so that fixes do not impact other dashboards and consistency is maintained

what we ll cover33
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
big bang vs gradual go lives
There are many ways to do a gradual rollout of the dashboards

The simplest way is:

Bring all content into the production box,

Setup the roles in production environment

Gradually release roles to the end users.

The benefit of doing this is that you can see how the system performs and solicit feedback from an increasingly large user group, before you release the dashboards to many users.

A gradual go-live can reduce the potentially negative impact on the organization.

Big-Bang Vs. Gradual Go-lives

Unless the dashboards are for a single department, or very few users, you should always plan for a gradual go-live

gradual go lives by region
A gradual go-live by region makes sense if:

Training is needed

Multi-language support is required

A high number of users are expected

Dashboards are tailored to local needs

When you use a regional go-live strategy, you can also roll out the dashboards to power users first (i.e. one month before go-live) to see how the system works in a real production setting

Gradual Go-lives by Region

Global dashboards require serious attention to language support, user support, 24/7 availability and attention to non-native English speakers.

xcelsius dashboard go lives by organization units
A gradual go-live by Organization Units makes sense if:

Organization is very large

Departments have very different needs

Dashboards are tailored to dept. needs

Data must be secured between organizations

When you use an organizational go-live strategy, you need to focus on the branding of each dashboard (i.e. logos of subsidiaries) and on integration of support functions in their organizations (i.e. helpdesk).

Xcelsius Dashboard Go-lives by Organization Units

Departmental dashboards should have first level support in their respective business units.

what we ll cover37
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
bi support organization big picture
You need to separate the operations of BI systems from the project work

If there is no support organization, the BI system quickly becomes an orphan when the project ends

Without a

support org. there

is a risk that

future BI projects

are delayed since

the project team

has to support

previous

projects

BI Support Organization — Big Picture
an example of a large bi support team
This large team can support complex applications, cockpits, BI portals, and broadcasting while providing training and help desk support as well as on-going data warehousing production support.An Example of a Large BI Support Team

Note: Job areas are meant for illustrations, and will vary depending on the BI applications supported

Support leader

Full time

Data loads & fixes

Full-time

Dashboards

i.e. Xcelsius

Full-time

SAP BI Basis

Full-time

Helpdesk,

user support

Full-time

BI Query

i.e. WebI, BEx

Full-time

Data loads & fixes

Full-time

Training,

user support

Full-time

Data quality &

data resource mgmt.

Full-time

Portal, collaboration,

KM, security

Full-time

what to include in a bi dashboard service level agreement sla
What to Include in a BI Dashboard Service Level Agreement (SLA)

When must data stores be loaded by (time)

What will happened if a persistent problem occurs (“swat” teams)?

Who is responsible for fixing process chains and who pays?

Do you get a discount for each DataStore that is not loaded in time?

How should Service Packs and Fix Packs be applied

When will SP, FP and SAP Notes be applied?

Who pays for it?

Who is responsible for testing them?

When will the BOBJ system be upgraded

When will upgrades occur, how is the pricing determined?

Who pays for it and who is responsible for testing?

How long can the system be off-line?

Minimum uptime and target uptime

What is uptime defined as (data store loaded vs. queries available vs. security fixes applied vs. portal uptime vs. third-party reporting tool uptime vs. network uptime vs. Xcelsius issues, etc.)?

What are the penalties (money) for missing the dashboard uptime requirements?

what to include in a dashboard bi sla cont
Issues log

What issues must be logged?

Who owns the log? Do you have access?

Can entries be updated, or must an audit trail be preserved?

Backup and disaster recovery

What is included in the backup and when is it taken?

When will restore abilities be tested?

How fast must restore occur, and what data stores and users will first have access (priority list)?

Who owns the data

If you switch vendors, who owns the data?

How will you get access to the data? Do you get full insights to all?

Who, of the vendor’s employees, gets access to your data? Can they share it with your competitor?

Service tickets

When will service tickets be monitored?

What are the categories and who will resolve them?

What are the resolution process and timelines?

How are customer and support satisfaction measured?

What to Include in a Dashboard BI SLA (cont.)
what to include in a bi dashboard sla cont
Escalation process

What will happened if an issue cannot be resolved by the internal IT department/ vendor and your business SLA manager?

What are the steps needed to terminate the SLA contract and are there any payments/fault payments or budget recourse (i.e., move money from cost centers)?

What to Include in a BI Dashboard SLA (cont.)

The more details you put into the dashboard SLA up- front, the easier it will be to measure and the more likely you are to have a successful relationship

reasonable xcelsius dashboard sla performance
Examples of reasonable performance targets:

90% of all dashboards run under 20 seconds

System is available 98% of the time

Data loads are available at 8am — 99% of the time

User support tickets are answered within 30 minutes (first response)

User support tickets are closed within 48 hours — 95% of the time.

System is never unavailable for more than 72 hrs — including upgrades, fix packs, service packs, and disaster recovery

Delta backups are done each 24 hrs cycle and system backups are done every weekend

Reasonable Xcelsius Dashboard SLA Performance
what we ll cover44
Background

Planning for the Xcelsius Dashboard Initiative

The JAD, RAD, Agile (XP) or ASAP methodologies

Getting the Right Dashboard Requirements

Upgrade and Tool Migrations

The Xcelsius Testing Approach

Big-Bang Vs. Gradual Go-lives

The BI Support Organization

Wrap-up

What We’ll Cover …
resources
Project Management Metrics, KPIs, and Dashboards: A Guide to Measuring and Monitoring Project Performance by Harold Kerzner, Aug. 2011

Agile Project Dashboards - Bringing value to Stakeholders and top management (What can you do for your PO Today?)by Leopoldo Simini, July 2011

Key Performance Indicators: Developing, Implementing, and Using Winning KPIs by David Parmenter, Jan 2007

Resources
7 key points to take home
Use a RAD, JAD or XP approach for your dashboards

Have multiple meetings with the user groups

Build interactive prototypes and expect requirements changes

Plan for a formal load testing of the dashboard

Have a rollout plan and a long-term vision of how to get there

Requirements gathering is interactive and users are discovering what they want

Spend serious time planning for a support and on-going enhancement of your dashboards, or they will become useless in a very short time...

7 Key Points to Take Home
your turn
Your Turn!

How to contact me:

Dr. Berg

bberg@comerit.com

disclaimer
SAP, R/3, mySAP, mySAP.com, SAP NetWeaver®, Duet®, PartnerEdge, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Wellesley Information Services is neither owned nor controlled by SAP.Disclaimer