1 / 16

APAC Initiatives for Large-Scale Data Sets and Grid Computing

APAC Initiatives for Large-Scale Data Sets and Grid Computing. Robin Stanton Bernard Pailthorpe Australian Partnership for Advanced Computing. Presentation to NeSC 28 May 2003. Topics. Infrastructure for eResearch Australian Partnership for Advanced Computing APAC GrangeNet

kalkin
Download Presentation

APAC Initiatives for Large-Scale Data Sets and Grid Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. APAC InitiativesforLarge-Scale Data Sets and Grid Computing Robin Stanton Bernard Pailthorpe Australian Partnership for Advanced Computing Presentation to NeSC 28 May 2003

  2. Topics • Infrastructure for eResearch • Australian Partnership for Advanced Computing • APAC • GrangeNet • Grid projects at ANU supported by APAC • APAC Initiatives

  3. Changing How Science is Done • Collect data from digital libraries, laboratories and observation • Analyze data with models run on the Grid • Visualize and share data over the Web • Publish results in a Digital Library From Sid Karin, SDSC/NPACI

  4. Grid Services for eResearch User Communities Bio-informatics Astronomy - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Physics Environment Distributed Computing Collaborative Visualisation Cooperative Environments Information Access On-Line Instruments Web Services Advanced Communications Services

  5. APAC Achievements • The APAC partnership formed June 2000 • A partner in each State as well as ANU and CSIRO • The APAC National Facility operational April 2001. • APAC and partner facilities serviced over 1,100 users. • Over 110 projects supporting users and developing expertise in 13 computational science and engineering themes. • Over 50 university courses prepared and delivered in computational science and engineering.

  6. APAC National Facility • Computing Systems • HP AlphaServer SC ES45 (127 nodes) • ranked number 63 in latest TOP500 list • Dell Linux cluster, HP Marvel • Mass Storage • Storagetek (robotic silo) tape library • Capable of a petabyte (1015 bytes) of storage • Visualisation • visualisation & virtual reality systems • Staff • staff at the Australian National University (ANU) http://nf.apac.edu.au

  7. GrangeNet:A GRid And Next GEneration Networkwww.grangenet.net Supported by the Federal Government’s BITS Advanced Networks Program

  8. Internet2 Canarie GeantAPAN APAC Partners and Backbone Networks Darwin Brisbane QPSF USA Canberra ANU Perth IVEC CSIRO Sydney ac3 Adelaide SAPAC APAC National Facility Melbourne VPAC CSIRO GrangeNet Backbone AARNet Links Hobart TPAC CSIRO

  9. Gravitational Wave Astronomy • GWA involves exchange and simultaneous data processing between multiple detectors • Gravity wave detectors + environmental monitoring • ACIGA – Australia • LIGO: USA VIRGO,GEO: Europe TAMA: Japan • Technical collaborations with GriPhyN and iVDGL • Operational data-pipeline centred on APAC MDSS • Upgrading to Lightweight Data Replicator (LDR) Australian Consortium for Interferometric Gravitational Astronomy

  10. MDSS ACIGA Data Grid Australian Consortium for Interferometric Gravitational Astronomy Rsync/LDR GridFTP ACIGA + APAC resources Environmental Monitors

  11. High-Energy Particle Physics • Belle Physics Collaboration • K.E.K. B-factory detector, Tsukuba, Japan • Matter/Anti-matter investigations, Atlas “test-run” • 45 Institutions, 400 users worldwide • 10 TB data currently • Universities of Sydney and Melbourne active participants • Australian collaborators leading Grid adoption • Australian Data-grid centred on APAC MDSS • Exploiting Globus 2.x, Gfarm • Atlas Experiment • Large Hadron Collider (LHC) at CERN • Collaboration 2000 people, 150 institutes internationally, 34 countries • 3.5 PB data per year • operational in 2007

  12. Virtual Observatories • MACHO Project Data • Largest online astrophysical data set in Australia • ~10TB Data collected over ~10 years • Hosted on APAC MDSS • Web interface at wwwmacho.anu.edu.au • Currently using Z39.50 metadata standard • Mapping metadata to VOTable 1.0 standard • Emerging IVO metadata standard • International Virtual Observatory • MACHO data being incorporated into SDSC SRB system • www.ivoa.net • Australian Virtual Observatory • IAU demo of Data Grid and Visualisation testbed • Distributed data-sets, Tomcat rendering software • www.atnf.csiro.au/projects/avo/

  13. Bioinformatics • Many initiatives to support bio-community • Bio-mirror supported by AARNet and ANU • www.bio-mirror.net • Bio-database search by VPAC and Ausbiotech • www.ausbioinfo.com • Australian National Genomic Information Service (ANGIS) • www.angis.org • ARC Centre for Bioinformatics (M Ragan) (www.imb.uq.edu.au) • Plan to coordinate infrastructure • Replicate access mechanisms to data sets • Provide common Web interfaces to applications • Provide specialised systems (Gaussian, Blast..)

  14. Earth Observation • GADS: Grid Access Data Service • for oceanographic and climate data • World Ocean Circulation Experiment (WOCE) • project funded by APAC • TPAC (Uni of Tasmania..), Bureau of Meteorology, University of Reading • Grid access via Web services • Interface to DODS/OPeNDAP • Used by Earth Systems Grid and NERC DataGrid

  15. Cultural & Language Archives • PARADISEC • Pacific and Regional Archive for Digital Sources in Endangered Culture • Uni of Sydney, Uni of Melbourne, ANU • Digitised oral and music recordings from Asia-Pacific region • Integrate with sociological data sets • International archival standard for digital audio • 24bit 96KHz Stereo + metadata • APAC MDSS to host 10,000 hours • 2GB/Hr => 20TB total

  16. APAC Initiatives • Provide more support for ‘data-intensive’ computing • APAC support for large-scale data sets • ask research organisations for proposals to have large-scale data sets managed by APAC and its partners • concentrate on national and international data access • develop plans for managing these data sets • Install and operate an APAC Grid • Consider a pilot project for eResearch • support a research community through data management

More Related