Peter X. Gao , Andrew R. Curtis, Bernard Wong, S. Keshav Cheriton School of Computer Science

1. Peter X. Gao, Andrew R. Curtis, Bernard Wong, S. Keshav Cheriton School of Computer Science University of Waterloo August 15, 2012

2. = ~1M servers CO2 of 280,000 cars

3. Datacenters and Request Routing DC 2 Dynamic DNS DC 1

4. Where to route?

5. Where to route? A.M. P.M. Electricity carbon footprint in California

6. How to split? DC 1 20% 80% DC 2

7. FORTE and its Contributions FORTE: Flow Optimization based framework for Request-routing and Traffic Engineering Contributions: • Principled framework for managing the three-way trade-off between access latency, electricity cost, and carbon footprint • Green datacenter upgrade plans • Impact of carbon taxes on datacenter carbon footprint reduction

8. Surprising Results • FORTE can reduce datacenter carbon footprint by 10% with no increase in electricity cost and access latency • Carbon Tax is not effective because taxes are only about 5% of electricity price

9. Outline • Model • P1: Assigning users to datacenters • P2: Assigning data objects to datacenters • P3: Datacenter upgrade • Evaluation

10. Model Datacenters: nj Data Objects: dk User Groups: ui Carbon emission: c(nj) Electricity price: e(nj) Capacity: cap(nj) NY LA DC Requests r(ui, dk)

11. Model Datacenters: nj P3 Data Objects: dk is placed at User Groups: ui serves Carbon emission: c(nj) Electricity price: e(nj) Capacity: cap(nj) P1 P2 Access latency: l(ui, nj, dk) NY LA DC Requests r(ui, dk)

12. Latency Cost Function Latency sensitivity latency cost: l(ui,nj) lmax latency

13. Outline • Model • P1: Assigning users to datacenters • P2: Assigning data objects to datacenters • P3: Datacenter upgrade • Evaluation

14. Assigning Users to Datacenters Datacenters: nj Data Objects: dk User Groups: ui P1

15. Objective Function { } Minimize: ∑ Weighted Carbon Cost: λ1c(nj) + Weighted Electricity Cost: λ2e(nj) + Weighted Latency Cost: λ0l(ui, nj, dk) Datacenters: nj n1 n1 Data Objects: dk User Groups: ui Carbon emission: c(nj) Electricity price: e(nj) Access latency: l(ui, nj, dk) u1 d2 n3

16. Datacenters: nj Demand Satisfaction Constraints n1 n1 Data Objects: dk User Groups: ui Carbon emission: c(nj) Electricity price: e(nj) Access latency: l(ui, nj, dk) u1 d2 n3 n3 Requests r(ui, dk)

17. Datacenters: nj Datacenter Capacity Constraints Data Objects: dk User Groups: ui u2 n4 n4 u3 Capacity: cap(nj)

18. Scale of Linear Program • Evaluation problem size: • Over 1 million variables • FORTE can solve it in approximately 2 min • Actual problem: • Can be over 1 billion variables

19. Fast-FORTE • Greedy Heuristic • Running time O(N logN) vs Simplex O(~N6) • Reduces running time from 2 minutes to 6 seconds • 0.3% approximation error

20. Outline • Model • P1: Assigning users to datacenters • P2: Assigning data objects to datacenters • P3: Datacenter upgrade • Evaluation

21. Assigning Data Objects to Datacenters Datacenters: nj Data Objects: dk User Groups: ui P2

22. Assigning Data Objects to Datacenters Datacenters Data Objects User Groups Σ Flow size = 101 Σ Flow size = 100 Σ Flow size = 1 requests

23. Outline • Model • P1: Assigning users to datacenters • P2: Assigning data objects to datacenters • P3: Datacenter upgrade • Evaluation

24. Using FORTE for upgrading datacenters Datacenters: nj P3 Data Objects: dk User Groups: ui

25. Using FORTE for upgrading datacenters Datacenter operators need to decide: • Which datacenters should be upgraded? • How many servers in that datacenter should be upgraded? The upgrade decisions are based on: • Estimation of future traffic demands • Annual budget on upgrading • Trade-off between cost and benefit

26. Using FORTE for upgrading datacenters Datacenters Data Objects User Groups requests Can also be used for selecting new datacenter locations by adding zero size datacenters into the network

27. Outline • Model • P1: Assigning users to datacenters • P2: Assigning data objects to datacenters • P3: Datacenter upgrade • Evaluation

28. Datasets Akamai traffic data • Akamai delivers about 15% - 20% Internet traffic • 3 weeks coarse-grained data in U.S. • Aggregated every 5 minutes U.S. Energy Information Administration • Carbon footprint • Electricity cost Data Objects: Synthetic with long-tail popularity, 10% latency tolerant

29. Different Level of Carbon Reduction Latency Only Small Reduction Large Reduction Medium Reduction

30. Three-way Tradeoff Tradeoff between carbon emissions, average distance, and electricity costs.

31. Two-way Tradeoff between Carbon Emission and Electricity Cost (987, 5.83) (1010, 5.73) Electricity Cost ($/hour)

32. Will Carbon Taxes or Credits Work? Akamai uses ~2 * 108 kWh per year • Electricity cost of 2 * 108 kWh: 2 * 108 kWh * 11.2c/kWh = $22.4 M • “Carbon cost” of 2 * 108 kWh : 2 * 108 kWh * 500g/kWh = 105 t 105 t * $10/t = $1 M

33. Green Upgrades • Low Electricity Price • Use Green Energy WA1 NY1 • Use Green Energy • Reduce Access Latency NJ1 CA1 NJ2 • Reduce Access Latency CA2 TX1 Year 3 Year 2 Year1 Reduces carbon emission by ~25% compare to carbon oblivious plan

34. Related Work • Qureshi et. al., Cutting the electric bill for internet-scale systems, SIGCOMM 09 • Doyle et. al., Server Selection for Carbon Emission Control, GreenNet 11 • Other related work can be found in our paper FORTE: • Considers data allocation problem • Supports datacenter upgrade • Explores the three-way trade-off

35. Conclusions • FORTE is a request routing framework that can reduce carbon emissions by ~10% without affecting latency and electricity cost • Surprisingly, carbon taxes do not provide sufficient incentives to reduce carbon emissions • A green upgrade plan can further reduce carbon emissions by ~25% over 3 years

36. Acknowledgement • We thank Prof. Bruce Maggs for providing us access to Akamai traces • We thank our shepherd Prof. Fabian Bustamante and the reviewers for their insightful comments