Introduction to software distributed shared memory systems
This presentation is the property of its rightful owner.
Sponsored Links
1 / 29

Introduction to Software Distributed Shared Memory Systems PowerPoint PPT Presentation


  • 125 Views
  • Uploaded on
  • Presentation posted in: General

Introduction to Software Distributed Shared Memory Systems. Chang-Yi Lin 2004 / 02 / 26. Outlines. What is a software DSM system? Message-passing vs. Shared-memory How does it work? Memory Consistency Models Cache Coherence Implementation Levels Granularity.

Download Presentation

Introduction to Software Distributed Shared Memory Systems

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Introduction to software distributed shared memory systems

Introduction to Software Distributed Shared Memory Systems

Chang-Yi Lin

2004 / 02 / 26


Outlines

Outlines

  • What is a software DSM system?

  • Message-passing vs. Shared-memory

  • How does it work?

  • Memory Consistency Models

  • Cache Coherence

  • Implementation Levels

  • Granularity


What is a software dsm system

What is a software DSM system?

  • A distributed-memory system (often called a multicomputer) consist of multiple independent processing nodes with local memory modules, connected by a general interconnection network.

  • Global shared memory.


What is a software dsm system1

What is a software DSM system?

  • A DSM system logically implements the shared-memory model on a physically distributed-memory system.

  • The DSM system hides the remote communication mechanism from the application writer, preserving the programming ease and portability typical of shared-memory systems.


Message passing vs shared memory

Message-passing vs. Shared-memory

  • Two different programming type.

  • Shared-memory programming is easier.


Message passing vs shared memory1

Message-passing vs. Shared-memory

  • Message-passing

    • Point-to-point communication


Message passing vs shared memory2

Message-passing

buffers and data types

blocking and nonblocking

Message-passing vs. Shared-memory


Message passing vs shared memory3

Message-passing vs. Shared-memory


Message passing vs shared memory4

Message-passing vs. Shared-memory

  • Pthread

Thread 1

Lock

A = A++;

unlock

Thread 2

Lock

A = A++;

unlock

Thread 3

Lock

A = A++;

unlock


How does it work

How does it work?


How does it work1

How does it work?


How does it work2

How does it work?


Memory consistency models

Memory Consistency Models

  • What is Memory consistency?

在單機上 P1: w(x)1 R(x)1

---------------------------------- time

在兩台機器上 P1: w(x)1

P2: R(x)?

---------------------------------------- time


Memory consistency models1

Memory Consistency Models

  • A consistency model is essentially a contract between the software and the memory.

  • If the software agrees to obey certain rules, the memory promises to work correctly.

  • If the software violates these rules, correctness of memory operation is no longer guaranteed.


Memory consistency models2

Memory Consistency Models

  • Strict consistency

  • Sequential consistency (SC)

  • Release consistency (RC)

  • Scope consistency (ScC)


Memory consistency models3

Memory Consistency Models

  • Strict consistency

    • Definition: any read to memory location x returns the value stored by the most recent write operation to x.

    • Impossible to DSM

P1: w(x)1

P2: R(x)1

---------------------------------------- time


Memory consistency models4

Memory Consistency Models

  • Sequential consistency (SC)

    • Definition: the result of any execution is the same as if the operations of all processors were executed in some sequential order, and the operations of each individual processor appear in this sequence in the order specified by its program.

    • Any valid interleaving is acceptable behavior, but all processes must see the same sequence of memory reference.


Memory consistency models5

Memory Consistency Models

  • Sequential consistency (SC)

P2

(C) b=1;

(D) Print(a,c);

P3

(E) c=1;

(F) Print(a,b);

P1

(A) a=1;

(B) Print(b,c);

(A)

(C)

(D)

(B)

(E)

(F)

(C)

(E)

(F)

(D)

(A)

(B)

(A)

(C)

(E)

(B)

(D)

(F)

(A)

(B)

(C)

(D)

(E)

(F)


Memory consistency models6

Memory Consistency Models

  • Release Consistency (RC)

    • Two types of access

      • Ordinary access: read and write

      • Synchronization access: acquire lock, release lock and barrier

    • Rules:

      • Before an ordinary access to a shared variable is performed, all previous acquires done by the process must have completed successfully.

      • Before a release is allowed to be performed, all previous reads and writes done by the process must have completed.


Memory consistency models7

Memory Consistency Models

  • Release Consistency (RC)

Rule1: …Acq(L1)…Rel(L1) …Acq(L5)…Acq(L3)…R(x)

Rule2: Acq(L2) w(x) R(y) R(z)Rel(L2)

Example:

P1: Acq(L) w(x)1 w(x)2 Rel(L)

P2: Acq(L) R(x)2 Rel(L)

P3: R(x)?


Memory consistency models8

Memory Consistency Models

  • Scope consistency (ScC)


Memory consistency models9

Memory Consistency Models

  • Relaxing consistency permits temporary inconsistencies (delayed updates)

    • Lazy release consistency (LRC) (TreadMarks, CVM)

    • Scope consistency (ScC) (JIAJIA, JUMP)


Cache coherence

Cache Coherence

  • Write invalidate

    • Suffer from false sharing

  • Write update

    • Too expansive when many replicas

    • Work best in application with tight sharing


Implementation levels

Implementation Levels

  • Modifying OS kernel

    • IVY (SC): modifying the memory management unit (MMU) of OS to map between the shared virtual memory address space and the local memory.

  • Language level

    • Linda, Orca

  • User-level runtime library

    • Trademarks, CVM, JIAJIA, JUMP, Brazos

  • Combination of multiple implementation levels, even hardware support

    • Munin, Midway, NCP2


Granularity

Granularity

  • The choice of the block size depends on

    • the cost of communcation

      • 1 byte message v.s. 1024 byte message

    • Locality of reference in the application

  • Most DSM systems use a page-based granularity with 1K byte to 8K byte.

  • Larger page size, better locality of reference


  • Login