View oriented parallel programming for multi core systems
1 / 28

View-Oriented Parallel Programming for multi-core systems - PowerPoint PPT Presentation

  • Uploaded on

View-Oriented Parallel Programming for multi-core systems. Dr Zhiyi Huang World 45 Univ of Otago. An age of CMT. CMT offers us the power of parallel computing To harness the power relies on good parallel applications and competent parallel programmers

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' View-Oriented Parallel Programming for multi-core systems' - benecia-urban

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
View oriented parallel programming for multi core systems

View-Oriented Parallel Programming for multi-core systems

Dr Zhiyi Huang

World 45

Univ of Otago

An age of cmt
An age of CMT

  • CMT offers us the power of parallel computing

  • To harness the power relies on good parallel applications and competent parallel programmers

  • Sound parallel programming methodology is the key

Two camps
Two camps

  • Message passing vs. shared memory

    • Message passing style is complex

    • Communication with shared memory is simple and easy, but…

Problems for sm based pp 1
Problems for SM-based PP (1)

  • Data race condition is a pain

    • Data race: there are concurrent accesses to the same memory location, and at least one of them is write access

    • To debug a data race condition is difficult since a parallel execution is normally not repeatable

Problems for sm based pp 2
Problems for SM-based PP (2)

  • Deadlock is another pain

    • Mutual exclusive primitives such as locks are required to prevent data races, but

    • it may result in deadlock, a situation where multiple threads/processes wait for each other due to competing for locks

    • Mutual exclusion has complicated the mental model of parallel programming

Problems for sm based pp 3
Problems for SM-based PP (3)

  • Poor portability is yet another pain

    • Parallel applications are system dependent

    • Mutual exclusive primitives such as lock are not standardized

    • Synchronization primitives such as barrier are not standardized

    • Shared memory allocation is not standardized


  • A parallel programming style with the following features

    • Data race free

    • Mutual exclusion free

    • Deadlock free

    • Portable to any systems with shared memory

What is a view
What is a view?

  • Suppose M is the set of data objects in shared memory

  • A view is a group of data objects from the shared memory

    •  V, VM

  • Views must not overlap each other

    •  Vi, Vj, i  j, Vi  Vj = 

  • Suppose there are n views in shared memory

    • ∑ Vi=M

Vopp requirements
VOPP Requirements

  • The programmer should divide the shared data into a number of views according to the data flow of the parallel algorithm.

  • A view should consist of data objects that are always processed as an atomic set in a program.

  • Views can be created and destroyed anytime.

  • Each view has a unique view identifier

Vopp requirements cont
VOPP Requirements (cont.)

  • View primitives such as acquire_view and release_view must be used when a view is accessed.


    A = A + 1;


  • acquire_Rview and release_Rview can be used when a view is only read by a processor.

Vopp requirements cont1
VOPP Requirements (cont.)

  • When a process/thread accesses multiple views at the same time, only one acquiring primitive is used.

    acquire_3_views(V_A, V_B, V_C);

    C = A + B;



  • A VOPP program for a producer/consumer problem

If(prod_id == 0){









Vopp features
VOPP features

  • No concern of data race condition

    • The programmer is only concerned about views, not mutual exclusion

    • Mutual exclusion is implemented by the system which detects potential data races as well by checking view boundaries

  • Deadlock free

    • Mutual exclusion is implemented by the system and can be implemented data race free and deadlock free

  • Portability?

    • By standardization of API

Requirements for the system
Requirements for the system

  • Keep track of view locations

  • Capable to check view boundaries

  • Guarantee deadlock free when implementing mutual exclusion

Advantages of vopp
Advantages of VOPP

  • Keep the convenience of shared memory programming

  • Focus on data partitioning and data access instead of data race and mutual exclusion

    • View primitives automatically achieve mutual exclusion

    • View primitives are not extra burden

  • The programmer can finely tune the parallel algorithm by careful view partitioning

Advantages of vopp cont
Advantages of VOPP (cont.)

  • Implementation independent

    • View access can be based on mutual exclusion or Transactional Memory (TM)

    • TM is a memory system that checks access conflicts

  • Programming language independent

    • Can be implemented as a user space library

  • Performance advantage

    • Cache pre-fetching when a view is acquired

    • Can cache a view until the view is not acquired by any other threads/processes

Philosophy of vopp
Philosophy of VOPP

  • Shared memory is a critical resource that needs to be used with care

    • If there is no need to use shared memory, don’t use it

    • Justification is wanted before a view is created

    • Compatible with Throughput Computing which encourages multiple independent threads running in a chip

Vopp vs mpi

  • Easier for programmers than MPI

    • For problems like task queue, programming with MPI is horrific.

  • Can mimic any finely-tuned MPI program

    • Shared message  view

    • Send/recv  acquire_view

  • Essential differences

    • View is location transparent

    • More barriers in VOPP


  • VOPP is supported by our DSM system called VODCA

    • DSM: Distributed Shared Memory system provides a virtual shared memory on multi-computers

    • VODCA: View-Oriented, Distributed, Cluster-based Approach to parallel computing

  • VODCA version 1.0

    • Will be released as an open source software

    • A library run at the user space

    • Its implementation will be published on DSM06


  • Use a cluster computer

    • The cluster computer, in Tsinghua Univ., consists of 128 Itanium 2 running Linux 2.4, connected by InfiniBand. Each node has two 1.3 GHz processors and 4 Gbytes RAM. We run two processes on each node.

  • We used four applications, Integer Sort (IS), Gauss, Successive Over-Relaxation (SOR), and Neural Network (NN).

Related systems
Related systems

  • TreadMarks (TMK) is a state-of-the-art Distributed Shared Memory system based on traditional parallel programming.

  • Message Passing Interface (MPI) is a standard for message passing-based parallel programming. We used LAM/MPI.

Future work on vopp
Future work on VOPP

  • API for multi-core systems

  • Implementation on Niagara

  • More benchmarks/applications, especially telecommunication applications

  • Performance evaluation on CMT

  • A view-based debugger for VOPP