View oriented parallel programming for multi core systems
This presentation is the property of its rightful owner.
Sponsored Links
1 / 28

View-Oriented Parallel Programming for multi-core systems PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

View-Oriented Parallel Programming for multi-core systems. Dr Zhiyi Huang World 45 Univ of Otago. An age of CMT. CMT offers us the power of parallel computing To harness the power relies on good parallel applications and competent parallel programmers

Download Presentation

View-Oriented Parallel Programming for multi-core systems

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


View oriented parallel programming for multi core systems

View-Oriented Parallel Programming for multi-core systems

Dr Zhiyi Huang

World 45

Univ of Otago


An age of cmt

An age of CMT

  • CMT offers us the power of parallel computing

  • To harness the power relies on good parallel applications and competent parallel programmers

  • Sound parallel programming methodology is the key


Two camps

Two camps

  • Message passing vs. shared memory

    • Message passing style is complex

    • Communication with shared memory is simple and easy, but…


Problems for sm based pp 1

Problems for SM-based PP (1)

  • Data race condition is a pain

    • Data race: there are concurrent accesses to the same memory location, and at least one of them is write access

    • To debug a data race condition is difficult since a parallel execution is normally not repeatable


Problems for sm based pp 2

Problems for SM-based PP (2)

  • Deadlock is another pain

    • Mutual exclusive primitives such as locks are required to prevent data races, but

    • it may result in deadlock, a situation where multiple threads/processes wait for each other due to competing for locks

    • Mutual exclusion has complicated the mental model of parallel programming


Problems for sm based pp 3

Problems for SM-based PP (3)

  • Poor portability is yet another pain

    • Parallel applications are system dependent

    • Mutual exclusive primitives such as lock are not standardized

    • Synchronization primitives such as barrier are not standardized

    • Shared memory allocation is not standardized


Solutions

Solutions?

  • A parallel programming style with the following features

    • Data race free

    • Mutual exclusion free

    • Deadlock free

    • Portable to any systems with shared memory


View oriented parallel programming

View-Oriented Parallel Programming


What is a view

What is a view?

  • Suppose M is the set of data objects in shared memory

  • A view is a group of data objects from the shared memory

    •  V, VM

  • Views must not overlap each other

    •  Vi, Vj, i  j, Vi  Vj = 

  • Suppose there are n views in shared memory

    • ∑ Vi=M


Vopp requirements

VOPP Requirements

  • The programmer should divide the shared data into a number of views according to the data flow of the parallel algorithm.

  • A view should consist of data objects that are always processed as an atomic set in a program.

  • Views can be created and destroyed anytime.

  • Each view has a unique view identifier


Vopp requirements cont

VOPP Requirements (cont.)

  • View primitives such as acquire_view and release_view must be used when a view is accessed.

    acquire_view(View_A);

    A = A + 1;

    release_view(View_A);

  • acquire_Rview and release_Rview can be used when a view is only read by a processor.


Vopp requirements cont1

VOPP Requirements (cont.)

  • When a process/thread accesses multiple views at the same time, only one acquiring primitive is used.

    acquire_3_views(V_A, V_B, V_C);

    C = A + B;

    release_views();


Example

Example

  • A VOPP program for a producer/consumer problem

If(prod_id == 0){

acquire_view(1);

produce(x);

release_view(1);

}

barrier(0);

acquire_Rview(1);

consume(x);

release_Rview(1);


Vopp features

VOPP features

  • No concern of data race condition

    • The programmer is only concerned about views, not mutual exclusion

    • Mutual exclusion is implemented by the system which detects potential data races as well by checking view boundaries

  • Deadlock free

    • Mutual exclusion is implemented by the system and can be implemented data race free and deadlock free

  • Portability?

    • By standardization of API


Requirements for the system

Requirements for the system

  • Keep track of view locations

  • Capable to check view boundaries

  • Guarantee deadlock free when implementing mutual exclusion


Advantages of vopp

Advantages of VOPP

  • Keep the convenience of shared memory programming

  • Focus on data partitioning and data access instead of data race and mutual exclusion

    • View primitives automatically achieve mutual exclusion

    • View primitives are not extra burden

  • The programmer can finely tune the parallel algorithm by careful view partitioning


Advantages of vopp cont

Advantages of VOPP (cont.)

  • Implementation independent

    • View access can be based on mutual exclusion or Transactional Memory (TM)

    • TM is a memory system that checks access conflicts

  • Programming language independent

    • Can be implemented as a user space library

  • Performance advantage

    • Cache pre-fetching when a view is acquired

    • Can cache a view until the view is not acquired by any other threads/processes


Philosophy of vopp

Philosophy of VOPP

  • Shared memory is a critical resource that needs to be used with care

    • If there is no need to use shared memory, don’t use it

    • Justification is wanted before a view is created

    • Compatible with Throughput Computing which encourages multiple independent threads running in a chip


Vopp vs mpi

VOPP vs. MPI

  • Easier for programmers than MPI

    • For problems like task queue, programming with MPI is horrific.

  • Can mimic any finely-tuned MPI program

    • Shared message  view

    • Send/recv  acquire_view

  • Essential differences

    • View is location transparent

    • More barriers in VOPP


Implementation

Implementation

  • VOPP is supported by our DSM system called VODCA

    • DSM: Distributed Shared Memory system provides a virtual shared memory on multi-computers

    • VODCA: View-Oriented, Distributed, Cluster-based Approach to parallel computing

  • VODCA version 1.0

    • Will be released as an open source software

    • A library run at the user space

    • Its implementation will be published on DSM06


Experiment

Experiment

  • Use a cluster computer

    • The cluster computer, in Tsinghua Univ., consists of 128 Itanium 2 running Linux 2.4, connected by InfiniBand. Each node has two 1.3 GHz processors and 4 Gbytes RAM. We run two processes on each node.

  • We used four applications, Integer Sort (IS), Gauss, Successive Over-Relaxation (SOR), and Neural Network (NN).


Related systems

Related systems

  • TreadMarks (TMK) is a state-of-the-art Distributed Shared Memory system based on traditional parallel programming.

  • Message Passing Interface (MPI) is a standard for message passing-based parallel programming. We used LAM/MPI.


Performance of nn

Performance of NN


Performance of is

Performance of IS


Performance of sor

Performance of SOR


Performance of gauss

Performance of Gauss


Future work on vopp

Future work on VOPP

  • API for multi-core systems

  • Implementation on Niagara

  • More benchmarks/applications, especially telecommunication applications

  • Performance evaluation on CMT

  • A view-based debugger for VOPP


Questions

Questions?


  • Login