1 / 32

The Eclipse Virgo Experience at CME Group

The Eclipse Virgo Experience at CME Group. Jan Fetyko. 03/29/2012. CME Group. CME Group, formerly known as Chicago Mercantile Exchange Holdings Inc., was founded in 1898 and operates the CME, CBOT, NYMEX, and COMEX regulatory exchanges worldwide.

alarice
Download Presentation

The Eclipse Virgo Experience at CME Group

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Eclipse Virgo Experience at CME Group Jan Fetyko 03/29/2012

  2. CME Group • CME Group, formerly known as Chicago Mercantile Exchange Holdings Inc., was founded in 1898 and operates the CME, CBOT, NYMEX, and COMEX regulatory exchanges worldwide. • It is the world’s leading derivatives marketplace with ~2500 full time employees and a market capitalization of over $15B. • http://www.cmegroup.com/

  3. Topics • About our application • Development strategy • Getting it running in Virgo • Web Layer • Impact on our team

  4. About our application • Surveillance tool – monitors the state of exchange and the market • Rewrite of an existing (legacy J2EE) app • Many services • Web application(s) that use the services • Uses: Spring, JDBC, LDAP, EHCache, H2, c3p0, Apache Tiles, snaps

  5. Development strategy

  6. Split all functionality into services / bundles • Defined interfaces for services • Skeletons only

  7. Implemented the interfaces • NOOSGi • Not running in Virgo • Unit tests with mocks

  8. Getting it running in Virgo

  9. Running in Virgo  Most bundles deployed and started without problems Exposed and used implementations 3rd party libs a big pain  

  10. Breaking in Virgo Unable to satisfy dependencies of bundle 'impl-use' at version '1.0.0': Cannot resolve: impl-use Resolver report: An Import-Package could not be resolved. Caused by missing constraint in bundle <impl-use_1.0.0> constraint: <Import-Package: com…some.internal; version="0.0.0"> Exposed and used implementations Solution • Anything internal needs to stay internal • Even a factory should be a “service”: Interface + Impl • Should prevent others to see what is inside the bundle

  11. Breaking in Virgo 3rd party libraries : EHCache • Not OSGi-fyied • Works great as long as no replication is needed • Replication using RMI probably works fine • Replication setup fails if using jgroups – it is loaded using Class.forName(…) • cacheEventListenerFactory.setClass("net.sf.ehcache.distribution.jgroups.JGroupsCacheReplicatorFactory"); • Ehcache-jgroups classes are not in Import-Package, it will throw ClassNotFoundException • Also, ehcache-jgroupsreplication.jar is not OSGi-fyied Solution • Use bnd tool to modify ehcache jar(s) • Create a bnd tool accepted settings file • Push the new ehcache jar into common place (maven repo, git)

  12. Breaking in Virgo 3rd party libraries : EHCache • ehcachebnd tool properties Bundle-SymbolicName: ehcache-core-2.3.1 Bundle-Version: 2.3.1 Bundle-Name: ehcache-core-2.3.1 Bundle-ManifestVersion: 2 Implementation-Version: 2.3.1 Implementation-Title: EH Cache Core Export-Package: *;version=2.3.1 Import-Package: org.jgroups.util,*

  13. Breaking in Virgo 3rd party libraries : EHCache • bndtool properties for ehcache-jgroupsreplication Bundle-SymbolicName: net.sf.ehcache.jgroupsreplication Bundle-Version: 1.4 Bundle-Name: net.sf.ehcache.jgroupsreplication Implementation-Version: 1.4 Implementation-Title: EH Cache jGroups Replication Export-Package: *;version=1.4

  14. What is happening in caching service Caching Service On Node 1 XYZ Service On Node 1 Objects Caching Service On Node 2 Serialize 010011001010110 Deserialize ClassNotFoundException in ehcache bundle

  15. Breaking in Virgo 3rd party libraries : EHCache • Replication is possible only for objects known to ehcache • Deserialization of keys and values fails with ClassNotFoundException, because your classes are not in the Import-Package Solution • Store only objects that are known to ehcache – serialize and deserialize them manually

  16. Breaking in Virgo 3rd party libraries : EHCache Solution - continued Public interface CachingService { CacheElement get(CacheKey key) } public interface CacheElement { Object getValue(Class<?> classLoadingClass); }

  17. Breaking in Virgo 3rd party libraries : MANIFEST.MF • It is easier than bnd tool • Faster results • You will learn a lot about MANIFEST.MF which is essential to know OSGi anyway • Most changed settings • Import-Package • Export-Package • Version(s) After hacking the MANIFEST.MF • Use bnd tool to have repeatable results

  18. Web Layer

  19. Web Layer Virgo snaps host

  20. Web Layer Apache Tiles & Spring MVC Fragments: content Snap Tiles definitions extend host definitions host Shared Tiles Definitions

  21. Web Layer Apache Tiles Host Tile fragment Tile

  22. Running in Virgo Mini Lessons learned • Design for modularity • Understand class loading • Understand Manifest • If possible use 3rd party libs that are OSGi ready • Avoid serialization across bundles Web Layer • For Virgo 2.1, we had to dig through the snaps source and request a tag for a version that worked in Virgo 2.1 (currently on Virgo 3.0.1) • Snaps + Tiles not the best combination • Provides good flexibility • Snaps provides a single point of entry (filters, security, etc.)

  23. ToolingDevelopmentDeploymentetc.

  24. Tools • STS (SpringSource Tool Suite) • Apache maven + bundlor plugin (SpringSource) • maven archetypes to create services and web bundles • Flyway for database versioning • Firefox + firebug • No Virgo tooling or Libra

  25. Developer Workflow • Maven + profiles for easy deployment • Custom plans in virgo while working on a bundle <plan name="com.cme.security.plan” ….> <artifact type="bundle" name="com.cme.web.security" version=”…" /> </plan>

  26. Deployment • QA and up: Package and deploy everything including Virgo every time • Everything is scripted including DB schema update (flyway) • QA time is reduced because changes are only made to a well defined set of bundles • Risk of running the wrong code in production is reduced to zero by not including the bundle in the deployment package • We have different deployment targets for 2 different internal customers. Their code cannot impact or risk being deployed together in production.

  27. Team • OSGi is not the easiest to understand for monolithic app developers • Virgo is just like any other container to most users • Once basic concepts are established, there are no issues • There are few experts who can help everybody else • 99% of code is not different from a non-OSGi code • We started with 2 people using Virgo, now there are 16 at CME and 6 in Brazil • Today we have 37 service bundles and 23 web bundles

  28. Conclusions of the Eclipse Virgo experience

  29. Virgo experience conclusions • Did not hit any Virgo bugs (yet), stable, didn’t experience any crashes • Problems only come from our code • Virgo 3.x release improved memory consumption comparing to 2.1 • The learning curve is steep • Design for modularity upfront is important • Cannot go back to monolithic app

  30. Q&A Jan Fetyko • jan.fetyko@cmegroup.com 03/29/2012 @jfetyko

  31. Give Feedback on the Sessions 1 Sign In: www.eclipsecon.org 2 Select Session Evaluate 3 Vote

  32. Futures trading is not suitable for all investors, and involves the risk of loss. Futures are a leveraged investment, and because only a percentage of a contract’s value is required to trade, it is possible to lose more than the amount of money deposited for a futures position. Therefore, traders should only use funds that they can afford to lose without affecting their lifestyles. And only a portion of those funds should be devoted to any one trade because they cannot expect to profit on every trade.  • The Globe Logo, CME®, Chicago Mercantile Exchange®, and Globex® are trademarks of Chicago Mercantile Exchange Inc. CBOT® and the Chicago Board of Trade® are trademarks of the Board of Trade of the City of Chicago. NYMEX, New York Mercantile Exchange, and ClearPort are trademarks of New York Mercantile Exchange, Inc. COMEX is a trademark of Commodity Exchange, Inc. CME Group is a trademark of CME Group Inc. All other trademarks are the property of their respective owners.  • The information within this presentation has been compiled by CME Group for general purposes only. CME Group assumes no responsibility for any errors or omissions. Although every attempt has been made to ensure the accuracy of the information within this presentation, CME Group assumes no responsibility for any errors or omissions. Additionally, all examples in this presentation are hypothetical situations, used for explanation purposes only, and should not be considered investment advice or the results of actual market experience.  • All matters pertaining to rules and specifications herein are made subject to and are superseded by official CME, CBOT, NYMEX and CME Group rules. Current rules should be consulted in all cases concerning contract specifications.

More Related