Infinispan based md sal data store poc
This presentation is the property of its rightful owner.
Sponsored Links
1 / 11

Infinispan based MD-SAL Data Store POC PowerPoint PPT Presentation


  • 139 Views
  • Uploaded on
  • Presentation posted in: General

Infinispan based MD-SAL Data Store POC. Goals. Validate the Data Store SPI by plugging an alternate implementation Gain experience with MD-SAL Internals Measure the overhead incurred by encode/decode when NormalizedNode is NOT the native format for the store

Download Presentation

Infinispan based MD-SAL Data Store POC

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Infinispan based md sal data store poc

Infinispan based MD-SALData Store POC


Goals

Goals

  • Validate the Data Store SPI by plugging an alternate implementation

  • Gain experience with MD-SAL Internals

  • Measure the overhead incurred by encode/decode when NormalizedNode is NOT the native format for the store

  • Provide a Data Point for comparison


Architecture

Architecture

DataStore Impl

Encode/Decode

Txn

Mgmt

ISPN

TreeCache

Change

Notifications


Implementation notes

Implementation Notes

  • Mapping to Tree Cache API

    • InstanceIdentier of Parent -> Tree Cache Node FQN

    • InstanceIdentifier of LeafNode/LeafSetEntryNode ->Map Key

    • LeafNode/LeafSetEntryNode Value->Map Value

  • Data Store Transaction mapped to TreeCache JTA Transaction

  • DataChange Notifications

    • Take a NormalizedNode Snapshot at beginning of Txn

    • Maintain a Transaction Log

    • Prepare ChangeEvents during Pre-Commit

    • Asynchronously send change events after commit


Learnings

Learnings

  • Mapping Data Store Transactions to ISPN JTA Transactions

  • Read Only Transactions may not get closed

  • Write and Delete methods in read-write transaction do not return a future

  • Data change events can be VERY expensive for anything but the In Memory store

  • Mapping to and from NormalizedNode can get complicated

  • TreeCacheRemoveNode API does not work reliably


Mapping datastore transactions

Mapping Datastore Transactions

  • Data Store supports multiple transactions per thread

  • JTA supports only one active transaction per thread

  • Transactions will need to be suspended/resumed appropriately

  • Suggestion

    • Allow only one active transaction per thread

    • Add an explicit suspend/resume method on a transaction


Closing read only transactions

Closing Read-only Transactions

  • For In-Memory Transactions, NOT closing a Read-Only Transaction is not an issue. It would be garbage collected

  • For JTA Transactions supported by other persistent data stores this may cause issues

  • Suggestion

    • Document that transaction close is mandatory for Read-Only Transactions as well


Write and delete methods without future

Write and Delete methods without Future

  • Write and Delete methods on DOMWriteTransactions return a void giving the impression that they are synchronous

  • Synchronous implementation may not be always possible

  • Suggestion

    • Return a ListenableFuture for Write and Delete for consistency


Datachange events

DataChange events

  • DataChange events require old and new data subtrees to be returned

  • Since the scope of the transactions is not known in advance, Entire data tree snap-shot has to be preserved

  • Tree snap-shot is trivial for in-memory store but could be VERY expensive for alternate implementations

  • Suggestion

    • Validate the use cases for returning entire sub-tree

    • Implementation must implement MVCC to support efficient data change notifications


Mapping to normalizednode

Mapping to NormalizedNode

  • For the In-Memory Store, NormalizedNode is the native format

  • For any alternate implementation, NormalizedNodes have to be constructed from native formats like SQL, K-V Store or Document store

  • Suggestion:

    • Provide utility classes to map NormalizedNodes to/from a simple tree structure


State of poc

State of POC

  • Fully functional. Not well tested

  • Integrated with the controller.

  • With Data Change Events performance is HORRIBLE

  • Without Data Change Events, performs same or better than the In Memory Data Store

  • Potential Optimization: Leverage ISPN MVCC or eliminate tree snap-shot at beginning and apply use Txn log to derive original

  • Seems to perform more consistently than the In Memory data store which slowly degrades over time

  • Next Steps:

    • No plans to pursue an Infinispan implementation at the moment.

    • Incorporate the learnings into data store design for Helium


  • Login