1 / 12

Distributed Transactional Memory for Clusters and Grids EuroTM, Paris, May 20th, 2011

Distributed Transactional Memory for Clusters and Grids EuroTM, Paris, May 20th, 2011 Michael Schöttner. Distributed Transactional Memory for Clusters The Plurix Operating System. Plurix project (1996-2004) at Ulm University with Prof. Peter Schulthess http:// www.plurix.de

kathie
Download Presentation

Distributed Transactional Memory for Clusters and Grids EuroTM, Paris, May 20th, 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Distributed Transactional Memory for Clusters and Grids EuroTM, Paris, May 20th, 2011 Michael Schöttner

  2. Distributed Transactional Memory for Clusters The Plurix Operating System • Plurixproject (1996-2004) at Ulm University • with Prof. Peter Schulthess • http://www.plurix.de • Distributed Operating System for PC-basedclusters • Language-basedapproachusing Java • Everything (includingkernelanddrivers) written in Java Clusters

  3. Distributed Transactional Memory for Clusters Transactional Distributed Shared Memory • DSM withtransactionalconsistency = DTM • Data exchangebetweennodesthrough DTM not throughmessagepassing • Almosteverythingstored in the DTM • Applicationdataandcode • Includingkernelanddriversdataandcode (exceptdriverbuffers) • Smart buffersfordeviceaccesswithintransactions Clusters – The Plurix project

  4. Distributed Transactional Memory for Clusters Transactional Distributed Shared Memory • MMU-basedmemoryaccessdetection • Automaticshadowcopycreation • Restartabletransactionwithoptimisticsynchronization • First winsstrategy • Serializazionimplementedby a token • Write sets send toothernodes forwardvalidation • Commit numbersstored in tokenusedtorecoverfrommissedcommits Clusters – The Plurix project

  5. Distributed Transactional Memory for Clusters Programming Model • Event-drivenarchitecture implicittransactions • Programsprovidecommands • Users executecommands • System automaticallyinserts BOT and EOT • Explicit transactions • For parallel computations • Reduceconflictprobability Clusters – The Plurix project

  6. Distributed Transactional Memory forGrids Object Sharing Service • XtreemOS Project (2006-2010) EU FP6 project • Linux-based Operating System forgrids • Object Sharing Service • Includes a DTM • Implemented in C (sharedlibrary) • Objective: Simplifydistributedstatemanagement Grids

  7. Distributed Transactional Memory forGrids Object Sharing Service • Replicamanagement • Sharedobjectsareautomaticallyreplicated • Update vsinvalidate (adaptive approach) • Consistencymanagement • Supporting different consistencymodels • Including a distributedtransactionalmemory Grids – Object Sharing Service

  8. Distributed Transactional Memory forGrids Programmingmodel • Sharedobjectsexplicitlyallocated (PGAS-model) • Speculativetransactionsdefinedbytheprogrammer • MMU-basedaccessdetection(millipagesavoidingfalseconflicts) • Deferredabortwhen in transaction-unsafecode Grids – Object Sharing Service

  9. Distributed Transactional Memory forGrids Commit protocols • P2P-based commitprotocol • Using a token, firstwinsstrategy & forwardvalidation • but optimizedbyrequestqueueand holder prediction • Coordinatedcommitprotocol • Backwardvalidation on coordinator • Combinedwithforwardvalidation on clients • Super-peer-basedcommitprotocol • Allowinggroup-localcommits, • Ifdatais not replicated outside groups Grids – Object Sharing Service

  10. Distributed Transactional Memory for Clusters andGrids Lessonslearned • Evaluation performed on clusters and grids • Up to 128 nodes on the Grid‘5000 platform • Parallel benchmarks (SOR, ray tracing, pi, ...) • MMVE: Wissenheim http://www.wissenheim.de • Unfortunately, conflicts are not seldom • Requires programer to reason about access patterns, transaction boundaries, local and shared data, ... • False conflicts • Reduce conflict unit size  bulk network transfers Clusters andgrids

  11. Distributed Transactional Memory forClouds Lessonslearned • Even iftherearenoconflictswehavethecommitoverhead • Localcommitshelp (partially) • Difficulttoadaptexistingapplications/algorithms • Itiseasiertowritenewtransactionalapps but ... • Non-transparent optimizations, e.g. allowingtoreadoldversionsmakedevelopmentharder Same storyasforconsistencymodels in DSM systems? Clouds

  12. Distributed Transactional Memory forClouds DTM perspectivesforcloudapplications • DTM usefulfordistributedapplicationsrunningin datacenterswhere strong consistencyisrequired • Server-basedMMVEs • Extended MapReduceapplications • Transactionalcachesfor Web applications(seeCloudTM, TxCache, CumuloNimbo) Clouds

More Related