MassConf : Automatic Configuration Tuning By Leveraging User Community Information

1. Computer Science MassConf: Automatic Configuration Tuning By Leveraging User Community Information Wei Zheng, Ricardo Bianchini, Thu Nguyen Rutgers University

2. Introduction • Large software is complex • May have hundreds of configuration parameters • Selecting proper values is important • Configuring software is difficult • Depends on hardware, workload, load intensity, and target • Hard to understand the relationship between them • Large configuration space • Existing approaches are far from ideal • Hard to find related parameters • Tuning performance involves many time-consuming experiments

3. MassConf • Our approach: vendor helps new users’ configuration process • Collect configurations of existing users for new users to try • Rank configurations to minimize the number of experiments • Key observations: • A configuration may work well for many users • Multiple configurations may work well for each user • Main challenges: • Ranking configurations from most to least promising configurations • Incomplete info about how well each configuration would work

4. Incomplete Information Configuration Space User Space C9 U5 U3 C2 U7 C4 U1 User Space Configuration Space C9 U5 U3 C2 U7 C4 U1 MassConf wants to rank C4 highly.

5. MassConf Overview Existing User 1 Existing User 2 Existing User N 1. Inform environment and configuration Vendor 3. Rank configurations 6. Change ranked list 2. Inform environment and target 4. Provide ranked list of configurations New User 1 New User M 5. Try configurations in turn (resort to Simplex, if needed)

6. Adaptive Ranking • Dynamically adapt to place good configurations at the top • Three approaches: slow, fast, and fastest First Configuration C7 C7 C7 C7 C2 C2 C2 C2 C3 C3 C3 C3 C5 C5 C5 C5 2ndUser C9 C9 C9 C9 C1 C1 C1 C1 C8 C8 C8 C8 C4 C4 C4 C4 1st User C6 C6 C6 C6 Last Configuration Original Slow Fast Fastest

7. Case Study: Apache Performance • Synthetic population of users due to lack of real data • Workloads: small files, large files, dynamic CGI scripts • A “user” is a combination of workload & performance target • 219 existing users • Evenly spread in the space of workloads, intensity, and target • 195 new users • Evenly spread but not overlapping with existing users

8. Configuration Popularity Some configurations work well for many users.

9. Popularity vs Meeting Users’ Target Some good configurations are not popular.

10. Evaluation • MassConf: Adaptive ranking (low, fast, and fastest) • Popularity ranking – the intuitive and obvious approach • Simplex – a well-known optimization algorithm • Metric: number of experiments to satisfy new users

11. Results • MassConf successfully reached all performance targets • Adaptive ranking beats popularity-based ranking • Adaptive ranking: the faster, the better • MassConf reaches more users’ targets than Simplex • MassConf is also faster than Simplex

12. Conclusions • MassConf uses existing configurations to help new users • Case study shows that MassConf efficiently achieves the performance targets • MassConf can be applied to other software, types of targets • Future works: Multi-tier systems • In the paper and TR: bootstrapping; optimized MassConf; more experiments, analysis, and results

13. MassConf Overview Existing User 1 Existing User 2 Existing User N 9. Warn about configuration 1. Inform environment and configuration Vendor 2. Cluster environments 4. Rank configurations 7. Store selected configuration 8. Change ranked list 3. Inform environment and target 5. Provide ranked list of configurations New User 1 New User M 6. Try configurations in turn (resort to Simplex, if needed)