Alliance Alliance Performance Status - CREQ

1. AllianceAlliance Performance Status - CREQ Régis ELLING July 2011

2. Alliance – Performance Status – Objectives Describe the performance enhancements with objectives: • Current results of the Stress tests with highlights on slow features. • Remaining or on-going actions needed to enhance performances.

3. Stress Tests – CREQStrategy & results

4. Alliance – Performance Status – Stress Tests - Strategy Strategy: • Stress tests have been updated and completed to have a global overview of the Alliance OEFE application, testing AFE Order, B2B and now also CDR & CREQ. • Considering the high number of scenarios and also to ease the analysis, we have: • Executed the runs on a feature by feature basis (per module): all the scenarios related to a given functionality of the application are executed. This way it is also possible to optimize a “module” (errors, response times, load …). • To execute all the scenarios at the same time: only once all the modules of the Alliance application have been validated. • The following slides present the results for the CREQ module. • The last stress test and following results are from the R4.0.

5. Alliance – Performance Status – Stress Tests results - CREQ (1/7) CREQ Management • The following features are included in this part of the test: • CREQ: Create Generic & Specific CREQ, search with high number of results, low number of results, Reply to Generic and Specific, Accept CREQ. • AFE management: Open Order, line creation by file upload. • Strategyapplied: • 2 hours of execution of scenarios whileincreasing the number of users up to 50 during 1st hour, then up to 100 for the 2ndhour. • Crash test not performed considering that there is a problem remaining on the search CREQ that is increasing quickly and highly the load on the SQL Server CPU.

6. Alliance – Performance Status – Stress Tests results - CREQ (2/7) CREQ Management - Scenarios

7. Alliance – Performance Status – Stress Tests results - CREQ (3/7) Errors during the stress test • The error rate is very low by scenario. • Many application errors occurred , “Timeout expired” errors happened during searches on collaborative requests.

8. Alliance – Performance Status – Stress Tests results - CREQ (4/7) Global Response times during the stress test

9. Alliance – Performance Status – Stress Tests results - CREQ (5/7) Top 30 highest response times • Overall, the module response times have decreased by 63% compared to the previous release. • Apart from the search CREQ which have increased, all response times have greatly decreased. • All response times are low (under 5 seconds) apart from: • Search CREQ (under analysis). • Duplicate (some additional actions are also done in the steps) • Attach file to CREQ (5 seconds think time) • Continue (0,1 second over the 5)

10. Alliance – Performance Status - Stress Tests results - CREQ (6/7) Activity during the Stress tests (100 users) • The 0% or 100% figures are because either the scenario was not run previously (application could not manage the load in R3.4) or is not executed in R4.0 (functional change). • The activity decreases are as follows: • CRASXX scenarios: decrease is related to the fail of the previous step of search CREQ. As a consequence the steps after the search have been less executed. • CRSMXX scenarios: Little decrease of the number of executions. • The CREQ module in R4.0 supports an average of 43% more activity compared to R3.4.

11. Alliance – Performance Status - Stress Tests results - CREQ (7/7) CPU load during stress test • The load is higher on the SQL server than on the web servers: • SQL server: around 95 % for 50 users / up to 60 % for 80 users and more • Web servers: from 10 % to 20 % for 50 users / between 20 % and 40 % for 100 users • From a memory usage perspective, there is a usage of 250 Mb RAM on each server out of the 2500 available. This part is also good. • The collaborative request module generates a too high load on the SQL server This is mainly due to the CREQ search feature which falls into timeout after 80 users.

12. Alliance – Performance Status - Stress Tests - CREQ - Conclusions Conclusions • The collaborative request module still generates a too high load on the SQL server (AS web server results are now good); this is due to the SQL queries on the collaborative request searches (high response times and timeouts using a lot of SQL server resources). • Apart from the Search CREQ, there are no errors and the response times are good and stable. • A final action is needed on the CREQ Search feature.

13. Remaining or on-going actions

14. Alliance – Performance Status – Action Plan Needed or on-going actions • Needed actions: • Analyse the Search CREQ feature to defineoptimizations to reduce the SQL server CPU usage and response times. • Re-execute a Stress test to validate the results, if good thenlaunch a crash test.

15. Alliance – Performance Status – Search Collaborative Request Search Collaborative requests • The search collaborative requests is one key limitation in the module that also affects the resource usage of the server. • Even though a optimizations have already been performed in R3.4 and R4.0, it is not enough when this search facility is under heavy activity. • Investigations are as follows: • Analyze the first part of the query which extracts the data of the Collaboration matrix to inject the user rights to define the results. This part has been identified as CPU and time consuming and increasing with the number of elements in the matrix itself. • Compare (after having done the run on the CDR) the behavior of the search CDR which deals with the same kind of complexity, but is technically built differently (CDR is a dynamic query, CREQ is a static SQL query). If results are better and stable with the CDR search, it means the search CREQ must be re-written dynamically.

16. www.alcatel-lucent.com www.alcatel-lucent.com