Cross Stratum Optimization (CSO) Bar BOF

1. Cross Stratum Optimization (CSO) Bar BOF 79th IETF @ Beijing, China

2. Agenda • IETF CSO Effort Background (Young Lee, 5 min) • NA-ARAM (Ning So, 15 min) • NS Query (Young Lee, 15 min) • Q & A (25 min) 79th IETF @ Beijing, China

3. IETF CSO Effort • CLO Bar BOF was held in Maastricht IETF meeting (78th) • 40 + attendants from carriers, vendors, academia and research institutes • Created IETF mailing list: cso@ietf.org • Public Archive: http://www.ietf.org/mail-archive/web/cso • CLO -> CSO (Name changes) • CSO := {NS Query, NA-ARAM} Where NS Query = Network Stratum Query NA-ARAM = Network Aware Application Resource Assignment and Mobility • NS Query BOF Request didn’t go through the initial IESG approval • Need more clarification 79th IETF @ Beijing, China

4. Network Aware Application Resource Assignment and Mobility in Data Centersdraft-so-network-aware-application-problem-01.txt Ning So (UTD) Young Lee (Huawei) Dave McDysan (Verizon) Greg Bernstein (Grotto) Tae Yeon Kim (ETRI) Kohei Shiomoto (NTT) Oscar Gonzalez de Dios (Telefonica) 79th IETF @ Beijing, China

5. Problem Description • Many application services offered by Data Center make significant use of the underlying networks resources • The same application service can be offered by multiple geographically dispersed data centers. • allowing the application to be closer to the end-users to enhance service performance and user experience • Problem #1: The decision as which server/data center to select for an application request from end-users can negatively affect the quality of experience (QoE) of the users if not done correctly. 79th IETF @ Beijing, China

6. Problem Description (Cont’d) • VMs supporting the applications can be actively distributed within and/or between data centers. • Improve server utilization • Prevent service performance degradation during the peak usage time period, and during equipment failures. • Problem #2: When instantiating new VMs or migrating existing VMs across data centers, the underlying network loading conditions within a data center (LAN) or between data centers (MAN/WAN) need to be considered 79th IETF @ Beijing, China

7. Exemplary Cloud Data Center Global Load Balancer DC 1 1 2 3 End-User 1 Application Servers CE 1 CE 4 Access Network PE 1 Carrier Backbone Network PE 1 PE 4 2 DC 2 CE 2 PE 2 PE 3 1 2 Access Network PE CE 3 DC 3 CE 4 End-User 2 79th IETF @ Beijing, China

8. Current Server Selection Technology • The server selection for an application/VM is done by load-balancer. • The load balancer is aware of a certain level of server usage data and distributes the application requests based on that data. 79th IETF @ Beijing, China

9. Deficiencies of Current Server Selection Technology • The current load balancing technology is insufficient in providing an optimal decision across multiple VLANs and multiple Data Centers • No standard solution for the communication exchange among load balancers located in different Data Centers • load balancers from different vendors cannot communicate to each other • load balancers know little about the underlying network conditions • When migrating existing VMs/applications from one data center to another, the underlying network load condition in LAN/MAN/WAN can be constraining factors. • Load balancers know little about user condition 79th IETF @ Beijing, China

10. Optimized Server Selection Criteria • Factors need be considered in choosing the right server in the right data center for an application request or in instantiating new or migrating existing VMs/applications • Server level load condition in a data center • Intra data center network condition • Carrier MAN/WAN network condition • User Condition 79th IETF @ Beijing, China

11. Server Selection Criteria Details • Server level load condition • VM supportability limitations • CPU utilization • Memory segmentation and consumption • Application limitations e.g. max # of simultaneous instances of the application supportable • Storage access speed (disk, RAM, etc.) • Environment considerations such as server temperature, power load, and electrical cost at the time • Operational and managerial considerations such as location of peer servers and storages • VM to NIC switching in a virtual switch • Intra-DC network condition • Server NIC to Top of Rack (ToR) Switch • TOR switch to Layer 2 Switch - link and node level • Between L2 Switches and L2 switch to L2 core/gateway switch/router - link and node level • L2 gateway router to provider edge (PE) router - link and node level. 79th IETF @ Beijing, China

12. Server Selection Criteria Details • Carrier MAN/WAN network condition • Type of networks and the technical capabilities of the networks • Bandwidth capabilities and availability • Latency • Jitter • packet loss • other Network Performance Objective (NPO) as defined in section 5 of [ITU-T Y.1541] • User condition • User access capabilities and limitations (e.g., user terminal information such as codec for video application) • User location • Optional user preferences (for some application, user may be able to specify its preferences. For example, the preferred server location for gaming). 79th IETF @ Beijing, China

13. Requirements for Optimized NA-ARAM • End-users send requests to the Application Controller (global load balancer), which makes server assignment decision for the user application. • End-users may send the following • Required application: this may be a simple URL request • Specific server identity or location. • Additional security requirements • Modified application specs. (for mobile users) • Optional end-user terminal information • Inter DC communication: the application controller need to be aware • Server/Application Status from each of the Data Centers where the application servers are located • Intra-DC network conditions from each of the Data Centers where the application servers are located • Data Center-Network Stratum Communication • The details of how this can be accomplished are addressed in Network Stratum Query draft. 79th IETF @ Beijing, China

14. Thanks and Questions? 79th IETF @ Beijing, China

15. NQ Querydraft-lee-network-stratum-query-problem-01.txt Young Lee (Huawei) Dave McDysan (Verizon) Ning So (UTD) Greg Bernstein (Grotto) Tae Yeon Kim (ETRI) Kohei Shiomoto (NTT) Oscar Gonzalez de Dios (Telefonica) 79th IETF @ Beijing, China

16. Backgrounds • Most Internet Services are offered by the “servers” geographically spread. • Data Centers Networks have emerged as physical infrastructure for Content Delivery Networks and Cloud Computing. • Application services, e.g., data centers, make significant use of the underlying network services • Application services have access to little or no information about network services, e.g., available bandwidth, congestion level, etc. 79th IETF @ Beijing, China

17. Context of NS Query:Data Center Networks (DCN) 79th IETF @ Beijing, China

18. Application Stratum Distributed Resources: Data Centers with servers, content, data sets, computing power, cache/mirror Uses Network Resources Different QoS requirements for each application Cross-Stratum Application Stratum NS Query Network Stratum • Bandwidth, Connections, Links, • Connection Processing (Creation, Deletion, Management) • Admission Control, Resource Reservation • Applications uses resources in IP, MPLS, and/or Optical Transport Networks, Layer 2 Network Stratum 79th IETF @ Beijing, China

19. Application Service Profiles Characteristics & QoS Requirement of application service from a network perspective: • Location profile: locations of both the clients and the sources • QoS profile: (i) Delay Tolerance Bound; (ii) Jitter Tolerance Bound; (iii) PacketDelivery Ratio Tolerance; (iv) Network Availability, etc. • Connectivity profile: (i) P-P; (ii) P-MP; (iii) MP-MP; (iv) Any Cast • Directionality profile: (i) uni-directional; (ii) bi-directional • Bandwidth profile: Maximum, average, and minimum bandwidth requirements for the connectivity, maximum burst rate, maximum burst duration, etc. • Duration of service profile: service time of the application • Network media profile: (i) optical only; (ii) no microwave, etc. • Restoration profile: (i) Reroute required; (ii) do not re-route, etc. • Security profile: (i) dedicated end-to-end VPN-like resource allocation; (ii) dedicated physical resource allocation Currently, network is not informed of the nature of the application --- there is no good way to convey the application service profile to network stratum 79th IETF @ Beijing, China

20. Carrier MAN/WAN network condition • Type of networks and the technical capabilities of the networks; • Bandwidth capabilities and availability; • latency; • jitter; • packet loss; • And other Network Performance Objective (NPO) as defined in section 5 of [ITU-T Y.1541]. 79th IETF @ Beijing, China

21. CSO NS Query Architecture Global Load Balancer Intra DC Resource End-User Interfaces Application Control Gateway Inter DC Gateway Remote Data Centers Interfaces NS Query (First Step) Network Control Gateway IPPM NS Query (Second Step) TED BGP-RIB IGP-RIB LSDB 79th IETF @ Beijing, China

22. Two-stage Query • A Vertical Query • First Stage: A vertical query from an application entity (i.e., the Application Control Gateway (ACG) in Data Center) to an entity representing the network (i.e., the Network Control Gateway (NCG)) for highly summarized and abstracted network related information for a particular point in network; and • A Horizontal Query • Second Stage: Internal "horizontal query" at the network layer along with summarization and abstraction of the network information in a form that preserves network confidentiality and significantly reduces the amount of information that needs to be transferred. • The raw information needed to perform this summarization/abstraction is defined in existing and emerging network management standards and protocols (SNMP, Netflow, sFlow, IPPM, IGP, RIB, etc...). • NS Query would not necessarily standardize how this "internal horizontal queries" and summarization would be performed but would illustrate how such processes can be accomplished via standards, emerging standards or common commercial practices. 79th IETF @ Beijing, China

23. NS Query Example • For a particular point in networks, the NS Query can ask the following network load data in a summarized/abstract form: • b/w availability (this case the minimum b/w requirement should be provided) • latency • jitter • packet loss • Note that this can be asked in a different way. For example, the query can simply ask: • Can you give me if you can route x amount of b/w (from server to end-user) within y ms of latency? • Can you give me if you can route x amount of b/w (from server to end-user) with no packet loss? 79th IETF @ Beijing, China

24. Thanks! 79th IETF @ Beijing, China