Juxmem an adaptive supportive platform for data sharing on the grid
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid PowerPoint PPT Presentation


  • 74 Views
  • Uploaded on
  • Presentation posted in: General

JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid. Gabriel Antoniu, Luc Bougé, Mathieu Jan IRISA / INRIA & ENS Cachan , France. Grid Data Service (GDS) meeting, Rennes, September 22 th 2003. Context: Data Management on the Grid.

Download Presentation

JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


JuxMem: An Adaptive Supportive Platform for Data Sharing on the Grid

Gabriel Antoniu, Luc Bougé, Mathieu Jan

IRISA / INRIA & ENS Cachan, France

Grid Data Service (GDS) meeting, Rennes, September 22th 2003


Context: Data Management on the Grid

  • Distributed numerical simulations (code coupling)

    • Needs data sharing

    • No many functional systems

Designing a satellite

Solid mechanics

Optics

Dynamics

Thermodynamics


Existing Data Management Systems

  • Non-transparent large scale data management

    • GridFTP (Globus) and MPI-IO

    • Internet Backplane Protocol (IBP)

    • Explicit transfert

    • No consistency ensured

  • Transparent small-scale data management

    • Distributed shared memory (DSM)

    • Consistency models and protocols

    • Transparent access

    • Small-scale, static and homogeneous architecture

  • TreadMarksTM


    Another approache: peer-to-peer systems

    • Peer-to-peer systems (P2P)

      • Distributed (large-scale)

      • Volatile peers

      • Peers have the same capacities and responsabilities

  • Sharing of immutable data

    • Centralized (Napster)

    • Flooding (Gnutella, KaZaA)

    • Distributed hash table (CFS, PAST)

  • Sharing of mutable data

    • One writer per data + static architecture (OceanStore)

    • Conflicts have to be manually resolved (Ivy)


  • Design principles

    • Proposition: data sharing service for the grid

      • DSM systems: consistency and transparent access

      • P2P systems: scalability and high dynamicity


    A Data Sharing Service for the Grid

    Persistent storage

    Transparency of localization

    Internet

    ?

    Data transfert


    A Data Sharing Service for the Grid

    Optimization of access

    Data consistency

    Internet

    Internet

    Data transfert


    A Data Sharing Service for the Grid

    Internet

    Internet

    Scalability of the architecture


    A Data Sharing Service for the Grid

    Internet

    Internet

    Handling volatility


    JXTA: a framework for P2P

    • Open-source platform for programming P2P applications

      • Specify a set of protocols

  • A peer

    • Uniquely identified (ID)

    • Address independent of the physical location

    • Several network access point (TCP, HTTP, etc)

  • Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer

    Peer

    Peer

    TCP/IP

    Peer

    Peer

    Peer

    Peer

    Peer

    Peer

    Firewall

    Firewall

    Peer

    Peer

    Peer

    Peer

    Peer

    HTTP


    JXTA: peer groups

    • Set of peers that share a common set of interests

      • Scope of communications

      • Specific policy of management

      • Peer group services

    NetPeerGroup

    PeerGroupA

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    Peer ID

    PeerGroupB


    JXTA: Advertisements

    • Every resource is described by an advertisement

      • Peers

      • Peer groups

      • Communication

        channels

      • Services

    PeerGroup Advertisement:

    <?xml version="1.0"?>

    <!DOCTYPE jxta:PGA>

    <jxta:PGA>

    <GID>

    urn:jxta: uuid- BCBCDEABDBBBABEABBBABA000000

    </GID>

    <MSID>

    urn:jxta:uuid-BFEFDEDFBABAFRUDBACE00000001

    </MSID>

    <Name>

    My Group

    </Name>

    <Desc>

    This group is to be used for my own testing

    </Desc>

    </jxta:PGA>


    JuxMem: a prototype

    Peer group juxmem

    Peer group data

    Peer group cluster A

    Peer group cluster C

    Peer group cluster B

    Logical architecture

    Physical architecture


    API of JuxMem

    • Alloc (size, attribs)

    • Map (id, attribs)

    • Put (id, value)

    • Get (id)

    • Lock (id)

    • Unlock (id)


    Managing Memory Resources

    • Advertisement of type provider: peer group cluster

    • Advertisement of type cluster: peer group juxmem

    Peer group cluster

    Memory provided

    Peer group juxmem

    Size 10

    Size 10


    Managing Shared Data Blocks

    • Allocation of a memory area = creation of a peer group data

      • Data blocks identified by the ID of the peer group

      • Advertisement published in the peer group juxmem

      • Shared access for clients by knowing the ID

  • Consistency

    • Data blocks are replicated on providers

    • Updated simultaneously (logical multicast)

    • Clients are not notified of updates

  • Synchronization

    • Lock for each data block


  • Handling the Volatility of Peers

    • A manager by peer group (cluster and data)

      • Dynamic monitoring of available peers

      • Data blocks are automatically replicated (data)

      • Updates advertisement of type cluster (cluster)

  • Volatility of managers

    • Periodic exchange of hearbeats

    • Dynamic replication of managers if needed on other peers


  • Implementation of JuxMem

    • JuxMem

      • JXTA service

      • + 5000 lines

      • Graphical tool


    Preliminary Evaluations

    • Cluster

      • PentiumII: 450 Mhz and 256 Mb of RAM

      • Network used: FastEthernet 100 Mb/s

      • Number of nodes used: 20

  • Experiments

    • Overhead memory consumption

      • Low: @ 6 % with respect to the underlying JXTA peers used

    • Study of the volatility of providers


  • Study of the Volatility of Providers (1)

    Data of one byte

    Degree of redundancy = 3

    Data manager is not killed

    1 client: 100 iterations lock-put-unlock

    16 providers

    Peer group juxmem

    Peer group data

    Peer group cluster


    Study of the Volatility of Providers (1)

    Data of one byte

    Degree of redundancy = 3

    Data manager is not killed

    1 client: 100 iterations lock-put-unlock

    16 providers

    Peer group juxmem

    Peer group data

    Peer group cluster


    Study of the Volatility of Providers (2)

    • Internal lock when replicating

      • Ensure the consistency of the new replica

      • Client is blocked

    Peer group juxmem

    Peer group data

    Peer group cluster


    Study of the Volatility of Providers (3)

    • JXTA and Java

    • Time-out detection adapted for the WAN

    • Independent of the data size

    Reconfiguration time @

    11 seconds

    Targeted volatility is weaker ( >> 80 seconds)


    Conclusion

    • Defines an hierarchical architecture for a data sharing service for the grid

      • Specific policy for each cluster

      • Scalability

  • Storage and transparent access to data blocks

    • Application level: data block ID

    • Persistency

    • Shared consistency

  • Handling volatility of peers

    • Actively taken into account


  • Future Work

    • Transparent mobility of data blocks

      • Unavaibility of nodes or even clusters

      • Affinity data – computations

      • Affinity data – data

  • Parameterizable consistency

    • Specific for each client

  • Hierarchical synchronization


  • Allocation Request

    6

    4

    3b

    1

    2

    5

    3a

    3b

    3a


  • Login