STFC ’ s Scientific Computing Department – An Introduction Ilian Todorov

1. STFC’s Scientific Computing Department – • An Introduction • Ilian Todorov • Scientific Computing Department • and • STFC Hartree Centre • STFC Daresbury Laboratory • Ilian.todorov@stfc.ac.uk

2. Organisation HM Government (& HM Treasury) RCUK Executive Group

3. Daresbury Laboratory Daresbury Science and Innovation Campus Warrington, Cheshire UK Astronomy Technology Centre, Edinburgh, Scotland Polaris House Swindon, Wiltshire Rutherford Appleton Laboratory Harwell Science and Innovation Campus Didcot, Oxfordshire Chilbolton Observatory Stockbridge, Hampshire STFC’s Sites Isaac Newton Group of Telescopes La Palma Joint Astronomy Centre Hawaii

4. Scientific Computing Department Established 1st April 2012 from the merger of Computational Science and Engineering and e-Science David Corney, Acting Director World leading computational expertise to support science and UK Industry "Big Science, Big Data, Big Compute"

5. Scientific Computing Department Major funded activities • 190 staff at DL and RAL supporting over 7500 users • Applications development, optimization and support • Hartree Centre supporting UK industry • Compute and data facilities and services • Research: over 100 publications per annum • Deliver over 3500 training days per annum • High-performance computing, numerical analysis, software engineering & visualization • Major science themes and capabilities • Expertise across the length and time scales from processes occurring inside atoms to environmental modelling

6. Scientific Computing Department Acting Director David Corney Group Leader Group Leader Martyn Winn Biology and Life Sciences Shirley Miller DL Administration Hannes Loeffler Eugene Krissinel Ville Uski David Waterman Marcin Wojdyr Charles Ballard Ronan Keegan Narayanan Krishnan Chris Morris AndreyLebedev Chang Sik Kim Agnel Joseph Chris Wood Valeria Losasso Tom Burnley James Gebbie ChristelleGendrin Laura Johnston Esme Williams Damian Jones RAL Administration Dave Emerson Engineering and Environment Division Head APPLICATIONS Benzi John Charles Moulinec Rob Barber XiaojunGu Stefano Rolfo Malgorzata Zimon Yin Yue Karen McIntyre Jean Pearce Carol Malpass CORE ACTIVITIES Barbara Montanari Theoretical and Computational Physics Finance Paul Sherwood Applications Keith Refson Leonardo Bernasconi Leon Petit Martin Lueders Martin Plummer DominikJochym Simone Sturniolo Nic Harrison Tracey Kelly Jenny Williams TBC Computational Chemistry CECAM Ilian Todorov John Purton David Gunn Laurence Ellison Richard Anderson Micheal Seaton Sebastian Metz Thomas Keal Henry Boateng Chin Yong Dawn Geatches David Bray Paul Durham Leon Petit Chadwick and RAL Libraries Group Leader Group Leader Jens Jensen Data Services Division Head Debbie Franks Daresbury Laboratory DATA Brian Davies Matt Viljoen Bruno Canning Shaun de Witt Christopher Prosser KashyapManjusha Carmine Cioffi Roger Downing Kevin O'Neill Juan Sierra Richard Blake Rob Appleyard Cheney Ketley Mark Swaisland Brian Matthews Research Data Rutherford Appleton Laboratory Juan Bicarregui Data KunalArora Steve Fisher Alistair Mills Shirley Crompton Kevin Phipps Antony Wilson VasilyBunakov Simon Lambert Catherine Jones Alastair Duncan Brian Ritchie Wayne Chung Erica Yang Holly Zhen Denise Small Thomas Kirkham Tracy Colborne Maureen Williamson Linda Gilbert Group Leader Dave Cable High Performance Systems Acting Division Head Tim Franks ViliamKalavsky Colin Morey Tavia Stone Stephen Hill Don Parkin Sarah Steele Karl Richardson SYSTEMS Nick Hill Research Infrastructure Andrew Sansum Systems Jonathan Churchill Suleman Tariq Ian Johnson Derek Ross Mohit Mittal Sam Worley Dave Meredith Claire Devereux John Kewley Cristina del Cano Novales Stuart Pullinger Ahmed Sajid Matt Langthorpe John Gordon Adrian Coveney Alison Packer Andrew Sansum Peta Scale Computing and Storage Martin Bly Tim Folkes KashifHafeez James Adams DimitriosZilaskos Ian Collier CatalinCondurache Gareth Smith John Kelly TijuIdiculla Rob Harper Emile YoumbiMbuenmo Alistair Dewhurst PPD Andrew Lahiff PPD Division Head HARTREECENTRE Group Leader Jennifer Scott Numerical Analysis Cliff Brereton Hartree Centre Michael Gleaves Adrian Toland Terry Hewitt Lee Hannis David Moss Angela Walsh Iain Duff Nick Gould Jonathan Hogg Tyrone Rees Sue Thorne EvgueniOvtchinnikov Chris Greenough Software Engineering GRADUATES Division Head Alan Kyffin Gemma Poulter TECHNOLOGY Frazer Barnsley Greg Corbett George Ryall HONORARY SCIENTISTS Mike Ashworth Application Performance Engineering Bill Smith DziidkaSzotek Valerie Burke Paul Kummer Bob McMeeking John Reid Paul Strange CoraliaCartis Jack Dongarra Kirk Jordan StankoTmoic VISITING SCIENTISTS Stephen Pickles Lucian Anton XiaohuGuo Andrew Porter Andrew Sunderland Rupert Ford Liam Jones OCF Jeremy Appleyard NVIDIA Peter Oliver Technology VISITORS Jonathan Follows Graham Riley Rob Allan VendelSzeremi Mark Mawson Martin Turner Visualisation September 2014 Barry Searle Ronald Fowler SrikanthNagella

7. Government Investmentin e-infrastructure - 2011 • 17th Aug 2011: Prime Minister David Cameron confirmed £10M investment into STFC's Daresbury Laboratory. £7.5M for computing infrastructure • 3rd Oct 2011: Chancellor George Osborne announced £145M for e-infrastructure at the Conservative Party Conference • 4th Oct 2011: Science Minister David Willetts indicated £30M investment in Hartree Centre • 30th Mar 2012: John Womersley CEO STFC and Simon Pendlebury IBM signed major collaboration at the Hartree Centre Clockwise from top left

8. STOP PRESS – Autumn Statement 3/12/14 13.10 • Today we commit to a massive, quarter of a billion investment in a new Sir Henry Royce Institute for advanced material science in Manchester, with branches in Leeds, Liverpool and Sheffield. • And we back the brilliant work on ageing being conducted at Newcastle University and big data computing at Hartree. • We’re also committing to the industry of the North with investment in new high value manufacturing research.

9. Facilities and Resources • IBM BlueGene/Q, 98,304 computer cores, 1.2 Pflop/s Linpack • Over 5 PB disc and 15 PB tape stores • IBM iDataplex, 8192 Sandy Bridge cores, 157 Tflop/s • Collaboration with IBM Research • Four major visualisation facilities: two DL, two RAL; three walls, one immersive

10. Our Clients Include

11. HC Client Projects • Engineering & Manufacturing • Vehicle Design & Testing • Consumer Electronics Design • Consumer Packaged Goods Design • Environment • Weather modelling • Life Sciences • Genomics for better crop yields • Energy • Advanced Battery Cell Design • Efficient Well Head Oil extraction • Financial Services • Risk Management • Service Modelling

12. 2014 Procurement for Energy-Efficient Computing & Big Data • £19M investment in energy-efficient and data intensive technologies • ClusterVision 1920 E5-2650 cores with Green Revolution Cooling • IBM NextScale 8460 E5-2697 cores • IBM iDataPlex 2016 E5-2697 cores with 42 Intel Xeon Phis • IBM NextScale 1152 ARM 64-bit cores • IBM BG/Q two racks BGAS with 128 HS4 cards, 256TB • 5 Maxeler MPC-X2000 nodes, 40 Maia FPGA dataflow engines • Instrumented machine room • Systems will be made available for development and evaluation projects with Hartree Centre partners from industry, government and academia

13. UK GridPP Tier-1 supporting CERN's LHC Tier 0 Offline farm CERN computer centre Tier 1 National centres Online system RAL,UK USA Germany Italy France Tier 2 Regional groups ScotGrid NorthGrid SouthGrid London Glasgow Edinburgh Durham Institutes Useful data model which may be adapted for use elsewhere Workstations

14. JASMIN/CEMS • The JASMIN “super-data-cluster” • Bringing the compute to the data for all NERC sciences • Data : Measured and Simulation • Facilitating comparison of models with data • UK and Worldwide climate and weather modelling community • CEMS (Climate and Environmental Monitoring from Space) with UKSA and Industry • CMIP5 / IPCC (Climate Change) Data Analysis (~1PB) • Met Office, ECMWF etc. • JASMIN2: now all of NERC environmental sciences • E.g. Genomics, Hydrology, Oceanography, Oil & Gas… • 10,000 UK and Worldwide users via 2PB CEDA archive (Centre for Environmental Data Archive) • http, ftp, scp, etc. + helpdesk services • 500 UK and World direct login users • ‘Long tail’ users via Cloud SaaS and PaaS Cloud technologies E.g. “Environmental Workbench” • High speed dedicated network links to:MetOffice, ARCHER, Leeds Uni., KNMI Holland, ESGF (Australia and USA) • Holds over 50% of modelling data for latest IPCC report

15. Gung-Ho • Quasi-Uniform Grids (icosahedral; kites; balanced triangles; cubed-sphere; Yin-Yang) • Advection schemes (conservation, SL, etc.) • Time schemes (explicit vs. implicit) • Test cases • Computational aspects GUNG-HO: Globally Uniform Next Generation - Highly Optimized Met Office, NERC, STFC collaboration over five years: STFC Hartree Centre project “To research, design and develop a new dynamical core suitable for operational, global and regional, weather and climate simulation on massively parallel computers of the size envisaged over the coming 20 years.” Triangles Cube-sphere Yin-Yang

16. SKA • Full partner; MA member of Exec Board of the SDP • SCD with RAL Space & IBM • Task lead for ARCH.SWE Software Engineering effort in the ARCH architecture element necessary to build the "horizontal" systems prototype for the SDP • PROT.ISP Service task to build, run and support the SDP software prototypes on hardware prototypes based on available technology

17. Scientific Computing Department Summary • 190 staff at DL and RAL supporting over 7500 users • Applications development, optimization and support • Hartree Centre supporting UK industry • Compute and data facilities and services • Research: over 100 publications per annum • Deliver over 3500 training days per annum • High-performance computing, numerical analysis, software engineering & visualization World leading computational expertise to support science and UK industry "Big Science, Big Data, Big Compute"

18. Meeting Agenda 1400 – 1420 Welcome and Introduction Mike Ashworth STFC Daresbury 1420 – 1440 CCP4 Charles Ballard – CCP4 1440 – 1500 Visualisation Matters - The Need to put the Human Back in the Computational Loop Martin Turner - STFC/University of Manchester 1500 – 1520 Materials Science: Accelerating Discovery through Computational Modelling and Simulation Barbara Montanari - STFC RAL 1520 – 1540 Engineering Stefano Rolfo – STFC Daresbury 1540 – 1600 Improving Software Quality Through Pragmatic Software Engineering Christopher Greenough – STFC RAL 1600 – 1605 Wrap up and Close Mike Ashworth

19. If you have been … … thank you for listening Ilian Todorov ilian.todorov@stfc.ac.uk http://www.stfc.ac.uk/scd