1 / 23

BDTS and Its Evaluation on IGTMD link

This research paper explores the Bulk Data Transfer Service (BDTS) and its evaluation on the IGTMD link. It discusses the motivations for data transmission in a Grid environment, the BDTS architecture, key concepts, flow control mechanisms, estimation techniques, collaboration with existing protocols, and the validation of the system. The paper concludes with open problems and the deployment of BDTS and network resource management in a Grid environment.

patriciac
Download Presentation

BDTS and Its Evaluation on IGTMD link

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BDTS and Its Evaluation on IGTMD link C. Chen, S. Soudan, M. Pasin, B. Chen, D. Divakaran, P. Primet CC-IN2P3, LIP ENS-Lyon 2008-06-30

  2. Outline • Bulk Data Transfer Service • A framework for Service Differentiation in Grid

  3. Motivations • Data transmission in Grid • Large volume, Long term • Between countable sites • Between storage systems • Between computation nodes and local storage system • Over private shared network • End-to-End quality of service. • Strict deadline • High reliability • Best-Effort with TCP • “Fair sharing” VS. Throughput intensive • ASAP VS. Strict deadline • Burst traffic

  4. BDTS • Bulk Data Transfer Service • A centralized bandwidth allocation system • Provides deadline concerned data movement • End-to-End traffic control • Smooth traffic • Optimizes link utilization

  5. BDTS Architecture • User Interface • Submit Transfer Job (t-job) • Network IS • Provide network information (topology, bandwidth, etc.) • Job Management • Optimization (BDTSh) • Flow control (FLOC) • Control the actual transfer according to the profile from JM

  6. Key Concepts • Network Model • Static layer 2 topology and characters • T-job • Volume: application layer data size • Time: Start time, End time • Source Destination pair • Path from NIS • Max-rate: Limitation due to end systems • Profile • Time-rate: layer 2 rate.

  7. Optimization • Minimize the Congestion Factor (allocated-throughput / Link-bandwidth) • Guarantee full usage of link bandwidth • Reduce the average package delay experienced by the coexisting interactive traffic B. Chen and P. Primet. Schedulling deadline-constrained bulk data transfer to minimize network congestion. In CCGRID’07, Pages 410-417. May 2007

  8. Flow Control • TCP + Precise Software Pacer (PSPacer) • PSP • A module for iproute2 • Precise network bandwidth control • Traffic Shaper • TCP with congestion avoid mechanism disabled • Optimized in restramssion and memory management • Collaborating with existing TCP based application

  9. The Estimation • No Ideal Transport Protocol: Layer 2 profile VS. Layer 7 Date Transfer • Protocol overhead (L7 protocol, TCP/IP header) • Synchronization between the profile and implementation • TCP’s re-act to the network • … • Linear estimation: Vs = a * Vr + b

  10. Collaboration

  11. The Project • Job Management (Java, c++) • User Interface (Java) • Floc / API (C, Linux Kernel, PSP) • Gridftp client (c++, globus)

  12. Testbed

  13. On Grid5000 • GridFTP • Security • File transfer

  14. Validation of System (Resource allocation)

  15. Validation of System (Stream Isolation)

  16. Long latency Average 106ms Background Traffic IGTMD

  17. Conclusions • Conclusions • Bandwidth resource management based on End-to-End traffic control • Provide a deadline file transfer based on IP • Open problems • The effect of arriving time of t-jobs to the optimization • The efficiency of Flow control • The deployment of BDTS and network resource management in Grid

  18. Outline • Bulk Data Transfer Service • A framework for the Service Differentiation in Grid

  19. Grid-Managed Network Resources • Why the network (bandwidth/Qos) need to be managed in a Grid running on a dedicated private Giga/Tera network? • More bandwidth, more bandwidth consuming applications • each member will have its own demand into the network. • Grid network • Applications sharing common infrastructure • Middle-ware service • Hosts, storage systems, networks, • Network is transparent for Applications • Introduce differentiation of service to application level objects?

  20. How? • Static • Different quality of service is provided by different instances • Applications make a choice • Dynamic • Service identify application • Assign SLA to instance of applications • Knowledge of both its users and its underlying service

  21. BDTS and GMNR

  22. A “ulimit” like network resource management interface • Adapt networking policy to traffic according to the process it belongs to • Identify • Traffics from a process and its children • Traffics from a user • Traffics from a VO • Traffics from a certain application • Traffics to certain destinations • Policy • Output Bandwidth • Diffserv QoS • … • Running time adjusting • Inner adjusting: • Grid-Job Wrapper: Deployment-User application-Uploading Result • Outer adjusting • Grid network Resource Management • Wrapper

  23. Thank you! • Comments and Questions?

More Related