Research Computing: Critical Needs and Opportunities at a University Based Academic Computing Center
1 / 20

Vijay K. Agarwala Senior Director, Research Computing and Cyberinfrastructure - PowerPoint PPT Presentation

  • Uploaded on

Research Computing: Critical Needs and Opportunities at a University Based Academic Computing Center HPC User Forum October 13 -14, 2008 High Performance Computing Center Stuttgart (HLRS),Germany October 16 th , 2008 Imperial College, London. Vijay K. Agarwala

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Vijay K. Agarwala Senior Director, Research Computing and Cyberinfrastructure' - tameka

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Research Computing: Critical Needs and Opportunities at a University Based Academic Computing Center

HPC User Forum

October 13 -14, 2008 High Performance Computing Center Stuttgart (HLRS),Germany

October 16th, 2008 Imperial College, London

Vijay K. Agarwala

Senior Director, Research Computing and Cyberinfrastructure

The Pennsylvania State University

University Park, PA 16802 USA

[email protected], 814.865.2162

Its organization

Vice Provost for Information Technology and CIO University Based Academic Computing Center


Vice Provost

Financial Services

ITS Organization



Research Computing and

Cyberinfrastructure (RCC)

Security Operations

and Services (SOS)

Marketing and


Human Resources



Teaching and Learning

with Technology (TLT)

Digital Library

Technologies (DLT)



Telecommunications and

Networking Services (TNS)

Administrative Information

Services (AIS)

Consulting and

Support Services (CSS)


Information Technology Services University Based Academic Computing Center

Faculty Advisory Committee

on Research Computing

and Cyberinfrastructure

Senior Director

Research Computing and Cyberinfrastructure

Meets the high-end computing technology needs of scholars in their research and teaching endeavors. The group partners with faculty members and collaborates with technology companies and other research organizations.

Domain Expertise


Consulting Support

Software Development


Programming Support

Visualization and



High Performance

Computing Systems

Research computing and cyberinfrastructure
Research Computing and Cyberinfrastructure University Based Academic Computing Center

  • Provide systems services by researching current practices in operating system, file system, data storage, job scheduling as well as computational support related to compilers, parallel computations, libraries, and other software support. Also supports visualization of large datasets by innovative means to gain better insight from the results of simulations.

  • Enable large-scale computations and data management by building and operating several state-of-the art computational clusters and machines with a variety of architectures.

  • Consolidate and thus significantly increase the research computing resources available to each faculty participant. Faculty members can frequently exceed their share of the machine to meet peak computing needs

  • Provide support and expertise for using programming languages, libraries, and specialized data and software for several disciplines.

  • Investigate emerging visual computing technologies and implement leading-edge solutions in a cost-effective manner to help faculty better integrate data visualization tools and immersive facilities in their research and instruction.

  • Investigate emerging architectures for numerically-intensive computations and work with early-stage companies. For example: interconnects, networking, and graphics processors and FPGA for computations.

  • Help build inter- and intra-institutional research communities using cyberinfrastructure and grid technologies.

  • Maintain close contacts with NSF and DoE funded national centers, and help faculty members with porting and scaling of codes across multiple architectures.

Compute engines
Compute Engines University Based Academic Computing Center


144 nodes

288 quad-core procs

Memory-16GB /node

IB interconnect


170 nodes

340 processors

340 GB RAM

35.1 TB storage


16 nodes

64 processors

Memory-128GB /node

Infiniband interconnect

Unisys ES7000

3 nodes

64 processors

192 GB RAM

Myrinet interconnect


144 nodes

288 quad-core procs

Memory-32GB /node

IB interconnect


132 nodes

368 processors

1280 GB RAM

Silverstorm Infiniband



16 nodes

128 processors

512 GB RAM

Pathscale Infinipath

Infiniband interconnect

HPC Storage Farm

100 TB of disk

needs to be

1000 TB


140 nodes

560 processors

1664 GB RAM

Programs libraries and application codes in support of computational research
Programs, Libraries, and Application Codes in Support of Computational Research

  • Compilers and Debuggers: AbsoftProFortran, GNU Pascal, IBM XLF, IBM XLC/C++, Intel Fortran, Intel C/C++, Lahey/Fujitsu Fortran 95 Pro, Portland PGI Compilers and Tools, Java, PathScale EKO compiler suite, TotalView, DDT, Valgrind

  • Computational Biology: BLAST, Blastall, Cister, ClustalW, ClustalX, Dotter, FASTA, fastDNAml, GeneMachine, GENSCAN, HMMgene, MrBayes, MZEF, PHRED/PHRAP/Consed, PHYLIP, ReadSeq, RepeatMasker, SEG, sim4, Sputnik, Treetool, wuBlast

  • Computational Chemistry and Material Science: FHI98MD, Gamess, Gaussian 03, GaussView, SemiChem, NWChem, Jaguar, Maestro, CHARMM, WIEN2K, VASP, ThermoCalc, Accelrys Material Studio, ADF, tmolex, Amber, Gromacs, NAMD, WxDragon, Molden, CPMD, Rosetta, CCP4

  • Finite Element Solvers: ABAQUS, ANSYS, FLUENT, GAMBIT, FIELDVIEW, LS-DYNA, MD/Nastran, OpenFOAM

  • Mathematical and Statistical Libraries and Applications: ATLAS, BLAS, ESSL, IMSL, LaPack, ScaLaPack, MASS, GOTO, Intel MKL, AMD ACML, Mathematica, MATLAB, Maple, Distributed MATLAB, PETSc, NAG, StarP, Watson Sparse Matrix Solver

  • Solid Modeling: MD/Patran

  • Statistics: R, SAS

  • Parallel Libraries: MPICH, Optimized versions of MPICH for high-performance cluster interconnects, Parallel IMSL, Distributed MATLAB, StarP, Distributed Maple

  • Optimization: GAMS / CPLEX, Csim, Tomlab

  • Multiphysics: Comsol

All software installations are driven by faculty. The software stack on every system is customized and entirely shaped by faculty needs.

Participating Research Centers Computational Research

Joint research and education initiative (NSF, DOE, PSU) focused on understanding molecular issues related to environmental chemical kinetics, geochemical cycling of elements, fate and transport of contaminants, and carbon sequestration.

(Dr. Susan L. Brantley, Professor of Geosciences)

Virtual center that integrates genetic, immunological, ecological and other studies to understand how disease processes work, and how they inter-relate across time and length scales. (Dr. Ottar Bjornstad, Associate Professor of Entomology and Biology; Dr. Bryan Grenfell, Alumni Professor of Biology )

Center for Gravitational Wave Physics (NSF, PSU) fosters research of a truly interdisciplinary character linking the highest caliber astrophysics, gravitational wave physics and experimental gravitational wave detection in the pursuit of the scientific understanding of gravity. (Dr. Lee S. Finn, Professor of Physics, Astronomy and Astrophysics)

Interdisciplinary center (NSF, NIH, PSU) focused on identifying issues in statistics, research design, and measurement emerging in the prevention and treatment of problem behaviors, particularly drug abuse. (Dr. Linda M. Collins, Professor of Human Development and Family Studies, Statistics)

Participating Research Centers Computational Research

Collaborative effort (NSF, PSU, Georgia Tech) aimed to educate the next generation of scientists and engineers in the emerging field of materials design. (Dr. Zi-Kui Liu, Professor of Materials Science and Engineering)

[email protected] State

Large collaboration (NSF, DOE, PSU, and others) of about 150 scientists working to use the AMANDA and IceCube telescopes to detect ultra-high energy neutrinos. (Dr. Douglas F. Cowen, Professor of Physics, Astronomy and Astrophysics)

Center aimed at describing, modeling, and understanding the Earth's climate system. (Dr. Michael E. Mann, Associate Professor of Meteorology)

A hub to connect experimental and simulation activities through the organization of collaborative projects, short courses and workshops. (Dr. Jorge O. Sofo, Associate Professor of Physics, Astronomy and Astrophysics)

Low level Computational Research

High level

-90.4 ppm

2-layer ONIOM Method



Performance of ls dyna
Performance of LS-DYNA Computational Research

Case Study – Blast Loading

  • Blast load using ConWep algorithm

  • 450,000 dofs, spherical blast for 11ms

  • Study of mesh convergence for plastic strain stability

Professor: Ashok D.Belegundu

Student : Vikas Argod

Dept of Mechanical and Nuclear Engineering

Visualization services
Visualization Services Computational Research

Staff members provide consulting, teach seminars, assist faculty and support facilities for visualization and VR.

  • Recent support areas include:

  • Visualization system design and deployment

  • 3D modeling and geometry exchange (e.g. FormZ)

  • Visualization development applications and programming toolkits (e.g. OpenDX, VTK )

  • VR development and device libraries (e.g. VRPN, VMRL, JAVA3D, OpenGL, OpenSG, CaveLib)

  • Domain specific visualization tools (e.g. VMD, SCIRun)

  • Telecollaborative tools and facilities support (e.g. Access Grid, VNC)

  • Parallel graphics and online visualization (e.g. Paraview, DCV)

  • Programming for graphics (e.g. C/C++, JAVA3D, Tcl/Tk, Qt)

Visualization facilities
Visualization Facilities Computational Research

Our goal is to foster more effective use of visualization and VR techniques in research and teaching across colleges and disciplines via strategic deployment of facilities and related support.

  • Locating facilities strategically across campus for convenient access by targeted disciplines and user communities

  • Leveraging existing applications and workflows so that visualization and VR can be natural extensions to existing work habits for the users being served

  • Providing outreach seminars, end-user training and ongoing staff support in use of the facilities

  • Working on an ongoing basis with academic partners to develop and adapt these resources more precisely to fit their needs

  • Helping to identify and pursue funding opportunities for further enhancement of these efforts as they take root

Immersive Environments Lab (IEL) Computational Researchin partnership with School of Architecture and Landscape Architecture208 Stuckeman Family Building

  • Focused on teaching and research in the experiential understanding of design spaces by architecture and landscape architecture students

  • 3-screen 3D-stereo multi-OS display offers multi-modal immersive environment

  • Experiential design review in design studio and digital media courses

  • Telecollaborative studio using standard definition video and 3D application sharing with Carleton University, Spring 2007 (lab also supports Access Grid)

  • Research into immersive and collaborative tools for design professions

  • Develop application and data integration workflows for ARCH/LARCH (Building Information Modeling, energy and structural analyses, land use planning, etc.) in conjunction with ICON lab under internal Bowers support

Immersive Construction Lab (ICon) Computational Researchin partnership with Architectural Engineering306 Engineering Unit C

  • Focused on research and teaching in the use of immersive visualization and VR techniques for planning and management of large construction projects

  • 3-screen, 3D-stereo, Windows desktop, immersive information environment

  • VR extensions to commercial construction planning applications, custom development of “VR-like” teaching modules

  • Industry partners provide real world use cases for studying the practical application of tools under development

  • Linked SMART Board allows dynamic VR updates from scheduling applications, etc.

  • Laptop display sharing facilitates group collaboration among students

  • Building upon ICON lab, IEL and related work, Penn State hosted CONVR 2007 international conference on construction applications of virtual reality

Visualization vr lab 215 osmond in partnership with materials simulation center
Visualization/VR Lab, 215 Osmond Computational Researchin partnership with Materials Simulation Center

  • VR facility for central campus science community (Materials Science, Molecular Biology, Physics and more)

  • 8’ x 8’ active stereo display

  • Tracked devices for user interaction

  • Linux console workstation

  • Complement of open source data visualization tools (VMD, VTK, Paraview - can be built upon in response to user needs)

  • Seminars for teaching graduate students on use of VR tools

  • Initial users in Materials Science and Molecular Biology

Visualization vr lab 336 ist building in partnership with computer science and engineering
Visualization/VR Lab, 336 IST Building Computational Researchin partnership with Computer Science and Engineering

  • Facility targets compute-intensive applications in science, engineering and related disciplines

  • Large-format 3D stereo display (6.75 x 9 ft., 1400 x 1050 pixel) for VR applications

  • 2 x 2 tiled display (6 x 8 ft., 2800 x 2100 pixel) for high-resolution applications

  • Linux console workstations

  • Interactive device support

  • Initial complement of data visualization tools (VMD, VTK, Paraview, SCIRun) to be built upon in response to user needs

  • Opened in October 2007

  • Seminars and outreach activity underway

Sports Medicine VR Lab Computational Researchassisting a partnership between Kinesiology, Athletics and HMC Lasch Football Building

  • Special purpose lab supports study of perception action disruptions in posture and balance related to mild traumatic brain injury (e.g. concussions), elderly populations, etc.

  • Motion in VR display is synchronized with measurement from EEG, postural tracking and force plate instrumentation

  • Enhanced two-screen lab (wall and floor) in development for Recreation Building for broader use by faculty in kinesiology, psychology, engineering science and mechanics, SSRI and Hershey Medical Center

Collaborative Tools in Research Computational Research

  • Telecollaboration

    • Adobe Connect (site license)

    • Vyew

    • Access Grid (many-to-many)

  • Document / Source Sharing

    • Subversion

    • Wiki

      • MediaWiki (public domain)

      • Confluence Wiki (commercial)

  • Web-Based Science Gateways

    • Materials Simulation Center Gateway (with Materials Research Institute)

Access grid teleconferencing facility room 140 computer building
ACCESS Grid Teleconferencing Facility Computational ResearchRoom 140 Computer Building

  • Allows faculty to participate in Access Grid events with international academic community

  • Scalable, multi-group telecollaboration (voice, video, application sharing) using multicast internet connections.

  • Small number of highly satisfied users

  • Needs greater awareness and adoption

Ongoing research collaboration: Dean Snow (ANTHY), Craig Cameron (BMB)

Research reporting: Richard Alley (GEOSCI), Donald Bryant (BMB), Mark Gahegan (GEOG)

Virtual conferences: Genomics and Bioinformatics, SC Global, SC Desktop

Industry Outreach Computational Research

in partnership with

Institute for Computational Science (ICS) and Industrial Research Office (IRO)

" ... to out-compete, we must out-compute .... "

  • Putting Pennsylvania based companies at a competitive advantage by

    helping meet their large-scale computing needs.

    Many small-to-medium sized companies, and even larger ones, do not have enough in-house expertise and resources for large-scale computations and as a result have not been able to use simulation and analysis tools with far greater frequency to help them innovate faster and become more competitive.

  • Helping develop stronger relationship between faculty and industry by

    providing computational services; may lead to more alignment between

    faculty research areas and industry needs and positive impact on economic development along the I-99 corridor and throughout the commonwealth.