slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Two E xamples of Open Source Software Developed at CERN: and PowerPoint Presentation
Download Presentation
Two E xamples of Open Source Software Developed at CERN: and

Loading in 2 Seconds...

play fullscreen
1 / 34

Two E xamples of Open Source Software Developed at CERN: and - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

RMLL visits at CERN – July 2012. Two E xamples of Open Source Software Developed at CERN: and. Digital Library Software http://invenio-software.org/. What is it used for?. ~350GB of PDFs at CERN ~20TB of images and videos 1M records . Depositing Archiving Organizing

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Two E xamples of Open Source Software Developed at CERN: and


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
    Presentation Transcript

    1. RMLL visitsat CERN – July 2012 TwoExamples of Open Source Software Developedat CERN: and

    2. Digital Library Softwarehttp://invenio-software.org/

    3. What is it used for? ~350GB of PDFs at CERN ~20TB of images and videos 1M records Depositing Archiving Organizing Disseminating Any type of document

    4. Whatis • Integrated Digital Library / Repository software • A platform of choice for managing documents in HEP • also adopted in other fields (medium to big repositories) • Web application • Open-source GPL-2 project • LAMP stack: Python (mostly), MySQL and Apache • Based on open standardsMARCXML, OAI-PMH, OpenURL, OpenSearch, etc. • Flexible, scriptable

    5. Invenio’sgears Lots of Python, with a sprinkle of C and Lisp(!) 630K lines of Python code MySQL ISAM for storing data Native indexing engine Apache + mod_wsgi + mod_xsendfile

    6. Invenio’s History

    7. Open Access at CERN “Consistent with the stated position of the Collaborations and the General Conditions applicable to Experiments at CERN, every effort will be made to publish papers under Open Access conditions, as defined by the SCOAP3 initiative. As at the date of this document, the Creative Commons Attribution ("cc by") license meets these conditions.” OA at CERN has a long history, the CERN Convention of 1953 states: "...the results of its experimental and theoretical work shall be published or otherwise made generally available". 

    8. Our development Environment Git distributed version control system Trac for ticket tracking VirtualBox + Vagrant for testing deployment We develop on SLC5/6 (based on RHEL5/6), on Ubuntu, on Debian…

    9. Quality Assurance • Coding standards • Eg. PEP8 (Style Guide for Python), etc. • Documentation • "If the code and the comments disagree, then both are probably wrong."– attributed to Norm Schryer • Test suite • ~1,000 unit/regression/web tests • Security • XSS, CSRF, SQL injection, etc. • Code review • Kwalitee check: "measuring" quality • "It looks like quality, it sounds like quality, but it’s not quite quality.”– CPAN Testing Service (quoting Michael Schwern)

    10. Our community 30 institutions worldwide CERN + DESY + Fermilab + SLAC EPFL … ADS and arXiv joining forces Translated so far into 26 languages 45 committers (in the last year) Free + Paid support

    11. An example installation • 1 Load balancer (HAProxy + Apachemod_proxy + mod_evasive) • 5 Worker nodes: • 2 VMs for static files • 3 Real machines for Python handled requests • 2 DB nodes (MySQLmaster + MySQLreplica) • AFS distributed FS for backups and file storage • Sustained recent Higgs announcement load (230 requests per second with peaks of 800 req/s)

    12. What’s next? Werkzeug/Flask + Jinja2 + WTForms for the web framework SQLAlchemy for DB abstraction Twitter Bootstrap + jQuery for the style Optional Solr indexing

    13. Conference Management Softwarehttp://indico-software.org

    14. History and Features Technologies Development

    15. WhatisIndico ? • Web-based event organization • Archive of events metadata and related documents (minutes, slides, etc) • Booking service and collaborationhub • Rooms • Videoconference • Webcast

    16. WhatisIndico ? • Started as an European Project - 2002 • First time used in 2004 • In production at CERN: http://indico.cern.ch • And in >100 institutions around the world • GSI, DESY, Fermilab,… • http://indico-software.org/wiki/IndicoWorldWide • Free and Open Source

    17. Indico @ CERN > 170.000 events > 700.000 presentations > 900.000 files

    18. Event Management withIndico All kinds of events

    19. Managing Simple Events

    20. Managing Meetings

    21. ManagingConferences

    22. Contributions Sessions Conferences ManagingConferences > Web Site Management > Conference Program Definition > Call for Abstract > Abstract Selection, refereeing > Timetable Setup > Slides and paper Submission > Attendant Registration, payment, badges > Paper Reviewing > Evaluation form > Long term archive and proceedings Full Lifecycle

    23. ManagingConferences

    24. Collaboration Hub Room Booking

    25. Collaboration Hub Collaboration service requests: Videoconference, webcast, recording

    26. Technology • Python >2.6 + WSGI • ​babel, webassets, ​pytz, ​zope.index, zope.interface, ​simplejson, ​suds, ​lxml, ​zc.queue, ​python-dateutil, ​pypdf, pyatom, reportlab, etc • ​Mako 0.4.1+ as template engine • ZODB as underlying database (http://www.zodb.org/) • Web frameworks: • jQuery • Backbone.js

    27. Infrastructure

    28. Compatibility • Many browsers compatibility: IE8+, FF3.6+, GChrome, Safari, etc • Working on mobile version

    29. DevelopmentTools Git as Control Version System ~ Eclipse + PyDev Unit and Selenium Test + Jenkins (Continuous Integration Server) Sphinx for Documentation Trac as Project Site Github: http://github.com/indico Transifexfor i18n: https://www.transifex.com/projects/p/indico/

    30. What’sNext ? Enhance the software: v1.0 end of 2012 Enlarge the community: more advertising

    31. Questions?