Inter domain socket communications supporting high performance and full binary compatibility on xen
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08 PowerPoint PPT Presentation


  • 49 Views
  • Uploaded on
  • Presentation posted in: General

Inter-domain Socket Communications Supporting High Performance and Full Binary Compatibility on Xen. March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08. Outline. Motivation Design & Implementation Socket creation Connect Accept Send/recv Close

Download Presentation

March 5, 2008 Kangho Kim System Software Team Internet Server Group VEE ‘08

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Inter domain socket communications supporting high performance and full binary compatibility on xen

Inter-domain Socket Communications Supporting High Performance and Full Binary Compatibility on Xen

March 5, 2008

Kangho Kim

System Software Team

Internet Server Group

VEE ‘08


Outline

Outline

  • Motivation

  • Design & Implementation

    • Socket creation

    • Connect

    • Accept

    • Send/recv

    • Close

    • Live migration

  • Evaluation results

  • Conclusion


Motivation

Motivation

  • Low bandwidth (Xen 3.0.3)

We need XWAY!!


Motivation cont d

Motivation(cont’d)

  • Low latency (Xen 3.0.3)

  • Existing socket applications


Design implementation

Design & Implementation


Shared memory

Shared memory

Domain 0

Domain U

Domain U

  • Direct communication

  • TCP/IP bypass

  • Direct data copy

User

  • Shared memory

TCP

Kernel

VM

VM

VM

Hardware

Shared memory

XWAY


Xway channel

XWAY channel

Domain A

Domain B

Event channel

SQ

SQ

Head

Tail

Head

Tail

RQ

RQ

Head

Tail

Head

Tail

Shared memory

for one XWAY channel


Xway architecture

XWAY architecture

socket application

library

user

kernel

VFS

BSD socket

INET

UNIX

X25

XWAY switch

UDP

XWAY

protocol

TCP

IP

XWAY driver

Network device driver


Dual channel

Dual channel

  • XWAY channel for data

  • TCP/IP channel for socket options and socket state

socket application

socket application

XWAY socket

XWAY socket

TCP/IP channel

XWAY channel


Socket data structure for xway

Socket data structure for XWAY

llseek: no_llseek

poll: sock_poll

open: sock_no_open

release: sock_close

readv: sock_readv

writev: sock_writev

sendpage: sock_sendpage

release: inet_release

bind: inet_bind

connect: xsw_stream_connect

accept: xsw_accept

poll: xsw_poll

listen: inet_listen

shutdown: inet_shutdown

sendmsg: inet_sendmsg

recvmsg: sock_common_recvmsg

sendpage: inet_sendpage

inode

socket

close: xsw_close

connect: xsw_v4_connect

disconnect: xsw_disconnect

accept: xsw_csk_accept

ioctl: tcp_ioctl

init: xsw_v4_init_sock

shutdown: xsw_shutdown

setsockopt: tcp_setsockopt

getsockopt: tcp_getsockopt

remote_dom_id

eventchn_port

eventchn_irq

xring_send

xring_recv

xring_send_h

xring_send_t

xring_recv_h

xring_recv_t

xway_sock

sock


Create

Create

Sock


Connect

TCP channel

Connect

Sock

XWAY channel

Sock

SQ

SQ

RQ

RQ

Connection

helper


Accept

Accept

TCP channel

Sock

Sock

XWAY channel

XWAY channel

XWAY channel

XWAY channel


Send recv

TCP channel

Send/Recv

Sock

Sock

XWAY channel

SQ

SQ

RQ

RQ


Close

TCP channel

TCP channel

Close

XWAY channel

Sock

Sock

SQ

SQ

XWAY channel

RQ

RQ


March 5 2008 kangho kim system software team internet server group vee 08

Poll

TCP session

Sock

Sock

XWAY session


Live migration

Live migration

  • Cut XWAY channels underneath XWAY socket

    • Redirect all data send/receive calls to TCP channels

  • Ready to live migration

Domain 0

Domain 0

Domain U

Domain U

User

Kernel

TCP/IP

VM

VM

VM

VM

Hardware

Hardware


Live migration cont d

TCP channel

Live migration (cont’d)

Sock

Sock

XWAY channel

SQ

SQ

RQ

RQ


Evaluation

Evaluation


Bandwidth

Bandwidth

  • Netperf TCP_STREAM


Latency

Latency

  • Netperf (TCP_RR)


Application performance

Application performance


Dbt 1 performance

DBT-1 performance


Execution time of api

Execution time of API


Binary compatibility

Binary compatibility

  • Scp

  • Ssh

  • Apache – TCP_DEFER_ACCEPT

  • Vsftpd – connect, accept, MSG_PEEK

  • Proftpd – shutdown, connect, SO_LINGER

  • Telnet

  • MySQL

  • Netperf


Conclusion

Conclusion

  • XWAY design and implementation

    • Full binary compatibility for socket interface

    • High performance

    • Live migration support

  • Remaining work

    • UDP, …

    • Domain crash resiliency

    • Reducing CPU usage

  • Code release

    • April 20, 2007: xway ver 0.6

    • June 4, 2007: xway ver 0.6.1

    • By the end of March, 2008: xway ver 0.7 (live migration included)

  • Download

    • http://sourceforge.net/projects/xway


Questions or comments

Questions or Comments

  • Kangho Kim

    • [email protected]

  • Cheiyol Kim

    • [email protected]

  • Hyun-Sup Shin

    • [email protected]


  • Login