1 / 21

VectorBase Releases

VectorBase Releases. Dan Lawson, All Sites. A release cycle for VectorBase. Regular release every 2 months In place since June 2010 Latest release is the 10th incarnation Review the aims of the release cycle Evolution of the release procedure Tracking/Organisation of a release

randall-stanton
Download Presentation

VectorBase Releases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VectorBase Releases Dan Lawson, All Sites

  2. A release cycle for VectorBase • Regular release every 2 months • In place since June 2010 • Latest release is the 10th incarnation • Review the aims of the release cycle • Evolution of the release procedure • Tracking/Organisation of a release • New hardware • Assessing releases • Current site responsibilities VectorBase 2012 2

  3. Aims of the release cycle for VectorBase • Regular releases encourage • (More) rapid update of data content • Pre-sites for new organism data • Updates for code base • As VectorBase is semi-parasitic there is a need to be reasonable current with Ensembl web code base • Benefit from external development (useability, functionality) • De-mystify the procedure of making a release • Instill a service provision mentality for part of the project VectorBase 2012 3

  4. Timeline of release improvements • VectorBase used to release on an ad-hoc schedule based on data cues • Staff turnover rapidly made this process troublesome because of php hooks into the Ensembl web code • Effectively VectorBase could not update its browser code • ND developers visited EBI in May 2009 to overcome this bottleneck • at this point we were running e! 46 code with e!54 databases • Scott & Dan discussions in a Rennes cafe, Nov 2009 • EBI to configure browser, ND to install • Apr-May 2010. EBI configures and tests new browser code • First VectorBase release in June 2010 VectorBase 2012 4

  5. Timeline of release improvements • Problems with browser installation at ND continued and from the VB_2010-10 release EBI both configured and installed the browser on ND servers • Ongoing small issues with server configuration, dns addresses etc. • Mid 2011; proposal to use Virtual Machines (VMs) to encapsulate the browser code • Late 2011; Refinement of what is to be included in each VM, troubleshooting transfer and deployment of VMs from EBI to ND • End of 2011; New linux hardware comes online. Should provide performance improvements in terms of raw speed but also resolve some compatability issues between browser code and legacy Mac servers VectorBase 2012 5

  6. Release coordination • Release coordination is achieved via the Tuesday Developer calls and makes use of the project wiki and JIRA systems • Uses semantic wiki extension of MediaWiki (SMW) to track deliverables and progress within a release • Uses JIRA versions to document and track issues arising from a release • Developer calls are the main forum to communicate issues and updates within the group VectorBase 2012 6

  7. Overview of release VectorBase 2012 7

  8. Release coordination on Project Wiki VectorBase 2012 8

  9. Summary overview VectorBase 2012 9

  10. Summaries and reports VectorBase 2012 10

  11. Overview of release VectorBase 2012 11

  12. Assessing releases • The VectorBase site is complex • Many dependencies, not all of which are documented • The project needs to have some concept of: • Monitoring serve uptime and potential issues without relying on the user community to notify us • Expand knowledge and ability to rectify issues • Incremental improve ability to identify issues with release before going live VectorBase 2012 12

  13. Monitoring infrastructure • VectorBase has tested some systems for monitoring server uptime (e.g. nagios) • Improved documentation, SOP for site maintenance VectorBase 2012 13

  14. Assessing a release • Use of predefined tests to check site • Use of checklists to confirm pages/views are correct • Always test new organism or gene set • Test random organism for a release (we can’t do all of them!) • Use of testing suites such as Selenium • Under used, should be developed and ran more frequently VectorBase 2012 14

  15. Use of Virtual Machines (VMs) • “completely isolated guest operating system installation within a normal host operating system” • With reference to the Ensembl browser code • Configuration and testing of installation on linux server • Encapsulation inside a VM • Disk image sent to ND for deployment on their servers • Should be no compatability issues • Advantages • Significantly easier deployment at ND • Should reduce system architecture issues • Disadvantages • Potentially harder to make small corrections to deployment VectorBase 2012 15

  16. Should I stay or should I go • What should be within the VM? • Browser web code and configuration/plugins • MySQL database server (data) • BioMart • Funcgen • PopBio • Considerations • Testing an installation is time consuming • Testing is only truly possible when the whole site is deployed VectorBase 2012 16

  17. New hardware at Notre Dame, Linux servers VectorBase 2012 17

  18. Release Shangri-La • VM built by EBI • Genome browser webcode and configuration • MySQL server with all relevant databases • VM built by ND • Current php/postgres site (includes static content, CAP) • BioMart • VM built by Imperial • Functional genomics (potentially) • PopBio (potentially) VectorBase 2012 18

  19. Release wish list for next 6 months • The current release is the first one on the new linux servers. We need to monitor this and potentially review resource allocation (genome browsers are memory hungry) • Renew efforts with regard to project wiki tracking/coordination • Extend and make more visible Selenium tests for the site • When we encounter a bug/issue we should be thinking can I emulate this as a Selenium test. If yes then this should be shared and added to the regular suite of tests to be run. We are not saturated where running tests has become onerous. • Better reporting of server monitoring. Do we have stats for server uptime etc. Should we be internally discussing these? VectorBase 2012 19

  20. Release wish list for next 6 months • Agree (and adhere to) mechanism of building dev/pre/live. • Production/Live (Twilight) is essentially read-only • Rotate Galactica and Futurama between staging/pre and development • Development site needs to have components being tested • Staging/pre site needs to be a complete deployment • Release procedure then becomes synch/copy of VMs to the production server, dev server becomes staging and staging becomes new dev. • EBI will maintain genome browser and MySQL server within a VM on the staging server • ND will develop main site on dev server and support main site and BioMart on the staging server VectorBase 2012 20

More Related