Software Consolidation Tasks
1 / 10

Software Consolidation Tasks B . Hegner (for the CERN PH/SFT group) - PowerPoint PPT Presentation

  • Uploaded on

Software Consolidation Tasks B . Hegner (for the CERN PH/SFT group) Preparation for ECFA HL-LHC Workshop 17- 7 -2013. The Context. Thinking about the future of SW in the next 10 to 20 years we will be challenged by Increasing processing demand Increasing parallelism in hardware

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Software Consolidation Tasks B . Hegner (for the CERN PH/SFT group)' - xannon

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Software Consolidation Tasks

B. Hegner

(for the CERN PH/SFT group)

Preparation for ECFA HL-LHC Workshop


The context
The Context

Thinking about the future of SW in the next 10 to 20 years we will be challenged by

  • Increasing processing demand

  • Increasing parallelism in hardware

  • Uncertainty of production platforms

    Bigger problems means our software must become simpler if we want to be able to solve them!

Making software simpler
Making software simpler

Rule #1 of simpler software:

Only consider functionality that is really needed

Rule #2 of simpler software:

Only provide functionality if it isn’t covered otherwise already

Rule #3 of simpler software:

Don’t try to be smarter than you actually are

Rule #4 of simpler software:

Re-assure from time to time on #1, #2 and #3

Evolution of languages
Evolution of Languages

  • A lot of high energy physics software projects are still in their 1st iteration with OOP and C++

    • The standard library, Boost and other libraries evolved since then

    • Following the evolution is a pre-requisite to stay future-proof

  • Transition to C++11 started

    • C++14, C++17, etc to come

    • E.g. major investment in ROOT 6.0

  • Transition to Python 3.0 rather slow

    • Will that ever fly in our community?

  • We should systematically review whether the approaches we chose are still state-of-the-art

Review existing code base
Review existing code-base

  • LHC software is now > 15 years old

  • Evolution of requirements since initial design

    • Thread-safety wasn’t on the radar for a long time

    • How many other requirement changes are there?

  • With present LHC experience the requirements are much clearer than before

    • Common understanding allows us to set up common projects

    • One common project emerging is the share of physics validation infrastructure

  • SW designers will be gone by HL-LHC, so knowledge preservation is a concern

    • Fix now everything that can be/ has to be fixed

    • And document everything else!

  • We should have a review of our libraries and should not limit ourselves when

    • Refactoring code for thread-safety

    • Changing interfaces to make them long-term maintainable

    • Deprecating functionality

  • Such reviews should not be a one-time effort, but translate into baseline activities

    • E.g. Geant4 relies on and does regular review, update and removal of physics models

Focus on core competencies
Focus on Core Competencies

HEP traditionally has a strong do-it-yourself attitude

  • … even within experiments!

    Core competencies of our community are I/O, statistical analysis and simulation

  • Need to parallelize them from the ground up

    Other important needs of our community need to be addressed as well

  • mathematical operations (e.g. VDT)

  • physics data structures (e.g. SoA vs. AoS)

  • tracking and clustering (e.g. fastjet)

  • experiment frameworks (e.g. GaudiHive)

    We have to focus and invest with common projects in these fields

Software validation
Software Validation

  • Testing and validation has ever been a core part of HEP software development

    • Our workflow is different from other communities

  • Importance for validation increases

    • May need to support another major production platform

    • Multithreaded reproducibility is much more complex

  • Profiling tools essential

    • First rule of optimization: know where to optimize!

    • We should use more standards in the community!

      • igProf could be such a standard

  • Validation tools

    • We ought to be able to have a common project among experiments!

Portability and preservation
Portability and Preservation

  • We need to facilitate the transition to new platforms and resources

  • Virtualization technology can help

    • Separate hardware and software update cycles

    • Versioned SW distribution a viable solution for long term environment preservation

      • E.g. CernVM for Na49/NA61

    • Provides out-of-the-box solution for clouds

      • Validated environment can be cloned basically everywhere

Dealing with bugs
Dealing with Bugs

  • Don’t let bugs enter the software in the first place

    • Better training of the community

    • Test-driven development

      • Important for platform migrations as well

  • Spot bugs early on

    • Run-time testing and validation

    • Static code analysis

  • Make debugging easier

    • What are good tools for multi-threaded debugging?

      We should seriously consider investing more here!


  • Problems are getting bigger

    • Let’s make software simpler!

  • Functional requirements much clearer than for LHC

  • We have to make debugging, validation and profiling easier

    • Invest in common tools and knowledge

  • We have to consolidate existing code base

    • Languages and external libraries evolved

    • Fix for ‘outdated’ decisions now

    • Review of software projects

    • We should dare refactoring things!

  • We have to consolidate on functionality

    • What are the core needs of the community?

    • We should dare deprecating things!

  • We have to consolidate on knowledge

    • Use more standards

    • Extend training for community