Download
on the role of the programmer the compiler and the runtime when facing accelerators in openmp 4 0 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Salvador/Brazil , IWOMP’10 2014, Sep 28-30th PowerPoint Presentation
Download Presentation
Salvador/Brazil , IWOMP’10 2014, Sep 28-30th

Salvador/Brazil , IWOMP’10 2014, Sep 28-30th

111 Views Download Presentation
Download Presentation

Salvador/Brazil , IWOMP’10 2014, Sep 28-30th

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. On the role of the programmer, the compilerand the runtime when facing accelerators inOpenMP 4.0 Guray Ozen Eduard Ayguade and Jesus Labarta Salvador/Brazil, IWOMP’10 2014, Sep 28-30th

  2. Motivation • MACC: MercuriumACCelerator Model • Evaluation • Conclusion

  3. Motivation • GPUs have become popular • Performance / WATT • GPGPU Challenges • Productivity is low, due to the different programming languages • Takes more time to learn and program • A lot of new concepts to consider • Thread divergence (due to conditional statements) • Using efficient Multi-GPU | Concurrencyof kernel computation • Minimizing Data movement (Slow bandwidth because of PCIe) • Appropriate use of GPU’s memory hierarchy (private vs. shared vs. global memory) and memory access patterns (coalesced memory accessing) • Optimization is hard even for experts • Code generation for GPGPU? Guray Ozen - IWOMP'14

  4. Outcome • Approach Analyzed Directive Based APIs • OpenACC proposal based on directives and compiler to translate to GPU code • OpenMP 4.0 accelerator model included in the OpenMP standard • OmpSs programming model at BSC • Outcome  MACC=MercuriumACCeleratorcompiler • CUDA code generatorby OpenMP 4.0 Accelerator directives • Involves little GPU specific compiler optimization techniques • Trying to influence the evolution of the OpenMP 4.0 • Data transfer minimization automatically(HOST-2-GPU) • Extended OpenMP 4.0with experimental new clauses • In order to use more team memory • Become available Multi-GPU task scheduling | Device-to-Devicedata transfer • Based on OmpSs suite • Already supports different memory address space such as GPU • Generated CUDA kernels by MACC can be OmpSs task • All kind of tasks (SMP | CUDA | ACC ) works Asynchronously • Manages CUDAConcurrency Guray Ozen - IWOMP'14

  5. OmpSs Programming Model Task implementation for a GPU device The compiler parses CUDA kernel invocation syntax Ask the runtime to ensure consistent data is accessible in the address space of the device #pragmaomptargetdevice ({ smp | cuda | opencl}) \ { copy_deps | [ copy_in(...)] [ copy_out (...)] [ copy_inout (...)] } \ [ ndrange (…) ] #pragmaomptask[ in (...)] [ out (...)] [ inout (...)] { <<.. function or code block ..>> } #pragmaomptaskwait Wait for sons To compute dependences Support kernel based programming • Extended OpenMP & Task based programming model • Mercurium Compiler • Nanos Runtime • Forerunner for OpenMP • Tasking and tasks dependences are two examples of OmpSsinfluence • OmpSs Current Accelerator Supports • Tasks to be executed on GPU programmed in CUDA or OpenCL • Runtime system takes care of data movement, overlapping and scheduling • Doesn’t generate gpu code Guray Ozen - IWOMP'14

  6. VectorADD: MACC vs OmpSs vs OpenMP 4.0 Guray Ozen - IWOMP'14

  7. MACCCode Generation • Offload • Starts after #pragma omptarget device(acc) • Device clause extended to specify device type, not physical device number (much better support for multiple GPUs) • Besides task level parallelism for target directive • Generated CUDA codes from task’s code region will be OmpSs task • Works asynchronously • Kernel configuration • #pragma omp teams | num_threads(int) | thread_limits(int) • If not specified MACC defaults to one iteration per block/thread • Work-sharing Directives • #pragma omp distributeIterations of loop distributed among CUDA blocks • #pragma omp parallel for Iterations of loop distributed among CUDA threads Nesting to enable multiple thread dimensions (2D/3D) Guray Ozen - IWOMP'14

  8. MACCCode Generation MACC MACC • Cyclic distribution • 1 iteration  1 CUDA Block / Thread • If at all possible, remove redundant iteration • Thread Divergence in CUDA • Assign one iteration to one thread/block Guray Ozen - IWOMP'14

  9. MACCCode Generation • Data Transfer Minimized Automatically (GPU-HOST) • OpenMP 4.0 • Need to specify target datain order to stay data on device • Sometimes download / upload is performed with target updateby hand • MACC • Ignoredtarget data & target update • Programmer only specifies directionality of task data, not the actual data movement • #pragma omptask in(list) out(list) inout(list) • Doesn’t download data from GPU until taskwait • Task scheduling with Multi-GPU • OpenMP 4.0 • device_id is given by hand  device(int) • Multi-Gpu scheduling is managed by user! • Device-to-device data transfer is unavailable! • target data device(device_id) • MACC • No device_id • Runtime can schedule Multi-GPU | Concurrent Kernel • Became available device-2-device transfer

  10. MACCMinimized Data Transfers & MultiGPU intmain(…) { doubleA[N], B[N],C[N] , D[N]; while(0-> 2) { #pragmaomp target device(cuda) ndrange(...) copy_deps #pragmaomp task inout(C) out(D) <..Optimized CUDA Kernel Invocation..> #pragmaomp target device(acc) copy_deps #pragmaomp task in(A) out(B) #pragmaomp teams distribute parallel for for(i=0 ; i< N; ++i) <..Sequential Codes to generate CUDA..> #pragmaomp target device(acc) copy_deps #pragmaomp task inout(A,B) #pragmaomp teams distribute parallel for for(i=0 ; i< N; ++i) <..Sequential Codes to generate CUDA..> } #pragmaomp target device(acc) copy_deps #pragmaomp task inout(C,B) in(D) #pragmaomp teams distribute parallel for for(i=0 ; i< N; ++i) <..Sequential Codes to generate CUDA..> #pragmaomp target device(smp) copy_deps #pragmaomp task in(A, C) <..Sequential codes / Result Test..> #pragmaomptaskwait } EXE 1 1 IDLE memcpyH2D(C) memcpyH2D(A) 2 2 3 3 2 1 4 3 memcpyD2D(B) 5 4 memcpyD2H(A) memcpyD2H(C) Guray Ozen - IWOMP'14 5

  11. MACCCode Generation • Team – 1 • Memory GPU thread Fast BW Slow BW GPU thread PCIe GPU thread Host thread GPU thread (HOST) Main Memory (Device) GPU Memory Host thread Slow BW GPU thread GPU thread • Team – 2 • Memory Fast BW GPU thread Host thread GPU thread Host thread GPU thread Slow BW GPU thread Team – 3 Memory Fast BW GPU thread • GPU Global Memory • Slow & Big ! • Use Team Memory • Correspond shared memory in CUDA • Each thread groups (CUDA blocks) have one shared memory. • Shared memory faster than global memory • Very limited in size (e.g. 48 KB compared to 6 GB of global memory) • In some hand-written CUDA codes we observed the use of shared memory for shared data, using blocking to overcome limited size GPU thread

  12. MACC Code Generation • Data sharing clauses with teams | private | first_private • Offers experimental 3 new clauses for distribute directive • dist_private([CHUNK]data1, [CHUNK]data2 …) • dist_firstprivate([CHUNK]data1, [CHUNK]data2 …) • dist_lastprivate([CHUNK]data1, [CHUNK]data2 …) Data movement to Device Memory Using TeamMem with Small DATA Using TeamMem with Big DATA Main Memory B B C A C A Device Memory C C C C B B B B A A A B B D D A Team Memories B B B B Guray Ozen - IWOMP'14

  13. Jacobi (A*x=B) • Transparent management of data movement in MACC • No need for data scoping directives in OpenMP 4.0 / OpenACC Hardware 2 x Xeon E5649 2 x NVidia Tesla M2090 24GB Main Memory Software OpenACC HMPP NVCC 5.0 GCC 4.6

  14. NAS Parallel Benchmark CG • NAS-CG Solves an unstructured sparse linear system by the conjugate gradient method • 3 Problem Set C > B > A • Effects of Runtime • How important Task-Scheduling • Multiple-GPU • Device-to-Device transfer • With 2-GPU • Easy to develop with MACC • MACC is better even with one GPU • Supports CUDA concurrency by streaming • Optimized task scheduling by Nanos runtime Guray Ozen - IWOMP'14

  15. DG Kernel • To calculate climate benchmark developed by NCAR (National Center for Atmospheric Research) • 4 versions of DG-Kernel • CUDA hand optimized code developed at NCAR • OmpSs + CUDA kernel • OpenACC code developed NCAR • MACC • Used to demonstrate: • MACC can have better results than hand-optimized CUDA • MACC optimization techniques • Compare MACC with hand optimized CUDA program Guray Ozen - IWOMP'14

  16. DG Kernel • 3 Optimization Techniques of MACC are used • Opt1New team memory techniques • Opt2Removing redundant For iterations • Opt3Start assign with 2 dim of thread • MACC has better result! Guray Ozen - IWOMP'14

  17. Conclusion • Presented a MACC research compiler to include new accelerator directives in the OmpSs programming model • Avoid the use of kernel programming using CUDA/OpenCL • Programming productivity and performance • New extensions proposed to OpenMP 4.0 • Compilersplays key factor • Code generation • Applying GPU specific optimizations • Effects of runtime & programmer are also important • Managing many kernels with many GPU? • Ability to use multi GPU • Using different pragma directives Guray Ozen - IWOMP'14

  18. Thank you! For further information please contact guray.ozen@bsc.es

  19. MACCMercuirumAcceleratorcompiler

  20. MACCMercuriumACCeleratorcompiler • Approach • StartfromOmpSs • Developed at BSC • Alreadyprovidingsupportfortaskdependencies and offloading CUDA/OpenCLkernels to accelerators • Addtheminimum set of OpenMP 4.0 acceleratormodeldirectivesintotheOmpSsprogramming in orderto avoidkernelprogramming • Add extra directivesforadditionalprogrammingproductivity and performance, ifnecessary • OmpSsprogrammingmodelimplementedwith • MercuriumCompiler • Source-2-Source compiler • Easy to prototype new codetransformations and generation • MACC requiredsomechanges in existingcompilationphases and a new phase • Nanos++ runtimesystem • Extremelygoodtaskparallelism • SupportsHeterogeneoustask (CUDA, OpenCL, SMP) • No changesrequired to support MACC codegeneration Guray Ozen - IWOMP'14

  21. Compiler phases in Mercurium • Parser(modified) • To parse new OpenMP 4.0 directives • Added new IR for OpenMP 4.0 • Nanos++ Lowering (modified) • It lowers OpenMP directives • Some semantics are changed • MACC lowering (new) • CUDA code generation • CompilationDriver • Backend compiling & linking Guray Ozen - IWOMP'14

  22. MACC Offers New Clauses in order to use Team Memory! MACC MACC • IF DATA SMALL? (existing OpenMP clauses) • Existing Clauses for #pragma omp teams • private(list) • firstprivate(list) • IF DATA BIG? (new MACC clauses) • New Clauses for #pragma ompdistribute • dist_private([CHUNK]data1, [CHUNK]data2 …) • dist_firstprivate([CHUNK]data1, [CHUNK]data2 ) • dist_lastprivate([CHUNK]data1, [CHUNK]data2 ) Guray Ozen - IWOMP'14 22