infinispan based md sal data store poc
Download
Skip this Video
Download Presentation
Infinispan based MD-SAL Data Store POC

Loading in 2 Seconds...

play fullscreen
1 / 11

Infinispan based MD-SAL Data Store POC - PowerPoint PPT Presentation


  • 186 Views
  • Uploaded on

Infinispan based MD-SAL Data Store POC. Goals. Validate the Data Store SPI by plugging an alternate implementation Gain experience with MD-SAL Internals Measure the overhead incurred by encode/decode when NormalizedNode is NOT the native format for the store

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Infinispan based MD-SAL Data Store POC' - vidal


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
goals
Goals
  • Validate the Data Store SPI by plugging an alternate implementation
  • Gain experience with MD-SAL Internals
  • Measure the overhead incurred by encode/decode when NormalizedNode is NOT the native format for the store
  • Provide a Data Point for comparison
architecture
Architecture

DataStore Impl

Encode/Decode

Txn

Mgmt

ISPN

TreeCache

Change

Notifications

implementation notes
Implementation Notes
  • Mapping to Tree Cache API
    • InstanceIdentier of Parent -> Tree Cache Node FQN
    • InstanceIdentifier of LeafNode/LeafSetEntryNode ->Map Key
    • LeafNode/LeafSetEntryNode Value->Map Value
  • Data Store Transaction mapped to TreeCache JTA Transaction
  • DataChange Notifications
    • Take a NormalizedNode Snapshot at beginning of Txn
    • Maintain a Transaction Log
    • Prepare ChangeEvents during Pre-Commit
    • Asynchronously send change events after commit
learnings
Learnings
  • Mapping Data Store Transactions to ISPN JTA Transactions
  • Read Only Transactions may not get closed
  • Write and Delete methods in read-write transaction do not return a future
  • Data change events can be VERY expensive for anything but the In Memory store
  • Mapping to and from NormalizedNode can get complicated
  • TreeCacheRemoveNode API does not work reliably
mapping datastore transactions
Mapping Datastore Transactions
  • Data Store supports multiple transactions per thread
  • JTA supports only one active transaction per thread
  • Transactions will need to be suspended/resumed appropriately
  • Suggestion
    • Allow only one active transaction per thread
    • Add an explicit suspend/resume method on a transaction
closing read only transactions
Closing Read-only Transactions
  • For In-Memory Transactions, NOT closing a Read-Only Transaction is not an issue. It would be garbage collected
  • For JTA Transactions supported by other persistent data stores this may cause issues
  • Suggestion
    • Document that transaction close is mandatory for Read-Only Transactions as well
write and delete methods without future
Write and Delete methods without Future
  • Write and Delete methods on DOMWriteTransactions return a void giving the impression that they are synchronous
  • Synchronous implementation may not be always possible
  • Suggestion
    • Return a ListenableFuture for Write and Delete for consistency
datachange events
DataChange events
  • DataChange events require old and new data subtrees to be returned
  • Since the scope of the transactions is not known in advance, Entire data tree snap-shot has to be preserved
  • Tree snap-shot is trivial for in-memory store but could be VERY expensive for alternate implementations
  • Suggestion
    • Validate the use cases for returning entire sub-tree
    • Implementation must implement MVCC to support efficient data change notifications
mapping to normalizednode
Mapping to NormalizedNode
  • For the In-Memory Store, NormalizedNode is the native format
  • For any alternate implementation, NormalizedNodes have to be constructed from native formats like SQL, K-V Store or Document store
  • Suggestion:
    • Provide utility classes to map NormalizedNodes to/from a simple tree structure
state of poc
State of POC
  • Fully functional. Not well tested
  • Integrated with the controller.
  • With Data Change Events performance is HORRIBLE
  • Without Data Change Events, performs same or better than the In Memory Data Store
  • Potential Optimization: Leverage ISPN MVCC or eliminate tree snap-shot at beginning and apply use Txn log to derive original
  • Seems to perform more consistently than the In Memory data store which slowly degrades over time
  • Next Steps:
    • No plans to pursue an Infinispan implementation at the moment.
    • Incorporate the learnings into data store design for Helium
ad