Providing QoS with Virtual Private Machines

1. Providing QoS with Virtual Private Machines Kyle J. Nesbit, James Laudon, and James E. Smith

2. Motivation for QoS • Multithreaded Chips • Resource sharing • Higher utilization • E.g., Niagara • Inter-thread interference • Applications • Soft real-time applications • Cell-phones and game consoles • Fine-grain parallel applications • Scheduling and synchronization • Server consolidation • Hosting services

3. QoS Objectives • Isolation • Priority • Fairness • Performance • Objectives are combined • E.g., Isolation and performance

4. QoS Framework • Separation of Objectives, Policies, Mechanisms • Well structured solutions Objectives Proc. 1 Proc. 4 Proc. 3 Proc. 2 L1 Cache L1 Cache L1 Cache L1 Cache Interconnect Mechanisms Global Policy Local Policy Bandwidth K Local Policy L2 Cache (Capacity C) Memory Controller Local Policy Bandwidth L Main Memory (Capacity M)

5. Local PolicyResource-Directed Service Isolation Allocations – Minimum Service Unallocated or Unused Work Conserving Policies Allocated Consumed Priority Vector – Relative Service Same Priority Unused Higher Priority Consumed Fairness Multiple Definitions?

6. Global Policy Virtual Private Machines Real-Time Thread Background Thread Background Thread Background Thread Fairness Policy VPM 4 VPM 2 VPM 3 VPM 1 Proc. 4 Proc. 2 Proc. 3 Proc. 1 L1 Cache L1 Cache L1 Cache L1 Cache BW .1K BW .1K BW .5K BW .1K L2 Cache (Capacity .1C) L2 Cache (Capacity .1C) L2 Cache (Capacity .1C) L2 Cache (Capacity .5C) Memory Cntl. Memory Cntl. Memory Cntl. Memory Cntl. BW .5L BW .1L BW .1L BW .1L Main Memory Main Memory Main Memory Main Memory Priority = 3 Priority = 0 Priority = 0 Priority = 0

7. Global PolicyPerformance-Directed • Global optimization problem • Use local policies to control resources • Optimize one bottleneck and the bottleneck appears somewhere else • Performance-directed policies need to fit into the VPM policy • E.g., optimize aggregate performance within a priority level

8. Status • Completed • Secondary cache [ISCA ’07] and SDRAM memory system mechanisms [Micro ‘06] • Bandwidth mechanisms • Cache capacity mechanisms • Ongoing and Future Work • Multithreaded Processors • Work conserving cache capacity policy • Priority policy • Aggregate performance policy

9. Conclusion • Objectives: Isolation, Priority, Fairness, Performance • Implementation: Separation of policies and mechanisms • Abstraction: Virtual Private Machines • Composable global policies that coexist on a per application basis

10. Questions and Comments • Do Virtual Private Machines meet all of the requirements of software controlled micro-architecture resource management?