1 / 28

A Hybrid OLTP & OLAP Main Memory Database System

A Hybrid OLTP & OLAP Main Memory Database System. Alfons Kemper, Thomas Neumann, ICDE 2011. 林立夫 (0256817). Outline. OLTP & OLAP Hybrid architecture OLTP Processing OLAP Snapshot Management Multiple OLAP Sessions Multi-Threaded OLTP Processing Evaluation. OLTP & OLAP.

koren
Download Presentation

A Hybrid OLTP & OLAP Main Memory Database System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Hybrid OLTP & OLAP Main Memory Database System AlfonsKemper, Thomas Neumann, ICDE 2011 林立夫(0256817)

  2. Outline • OLTP & OLAP • Hybrid architecture • OLTP Processing • OLAP Snapshot Management • Multiple OLAP Sessions • Multi-Threaded OLTP Processing • Evaluation

  3. OLTP & OLAP • OLTP (Online Transaction Processing) • To control and run fundamental business tasks • OLAP (Online Analytical Processing) • To help with planning, problem solving, and decision support • BI (Business Intelligence) OLAP OLTP transaction Report

  4. OLTP & OLAP • Disadvantage of traditional OLAP • data staleness • Goal for planning systems • data at your fingertips

  5. HyPer • HyPerarchitecture • mixed workloads of OLTP transaction processing and OLAP query processing on the same database

  6. HyPer • HyPerarchitecture • processes OLTP transactions at rates of tens of thousands per second, and, at the same time • processes OLAP queries on up-to-date snapshots of the transactional data

  7. OLTP Processing • Snapshot

  8. OLTP Processing • HyPer architecture • Low-level script - Directly manipulates the in-memory data structures • High-level script - To lookup database entries then insert, update and delete data records • Snapshot • These snapshots are created by forking the OLTP process and thereby creating a consistent virtual memory snapshot

  9. OLTP Processing • Prohibiting interactive transactions • The OLTP transactions have to guarantee short response times in order to avoid long waiting times for subsequent transactions in the queue.

  10. OLAP Snapshot Management • Copy on Update to Preserve Consistent Snapshot

  11. OLAP Snapshot Management • Creating a child process of the OLTP process via the fork() system call • The fork() should only be executed in between two (serial) transactions, never in the middle of one transaction.

  12. OLAP Snapshot Management • The forked child process obtains an exact copy of the parent processes address space

  13. Multiple OLAP Sessions • Multiple snapshots

  14. Multiple OLAP Sessions • Multiple snapshots • Making good use of the multi-core servers is to create multiple snapshots • OLAP queries are read-only they could easily be executed in parallel in multiple threads that share the same address space

  15. Multiple OLAP Sessions • Multiple snapshots • Snapshot can simply be achieved by periodically (or on demand) fork()-ing a new snapshot and thus starting a new OLAP query session process. • A snapshot will be deleted after the last query of a session is finished. This is done by simply terminating the process that was executing the query session

  16. Multi-Threaded OLTP Processing • Rearranging the OLTP workload queue • In realistic applications we expect many more read-only transactions than update transactions • We could even be increased by (carefully) rearranging the OLTP workload queue

  17. Multi-Threaded OLTP Processing • Rearranging the OLTP workload queue • Admitting multiple read-only OLTP transactions in parallel • As soon as a read/write-transaction is at the front of the OLTP workload queue the system is quiesced and transferred back into sequential mode until no more update-transactions are at the front of the queue

  18. Multi-Threaded OLTP Processing • Rearranging the OLTP workload queue

  19. Multi-Threaded OLTP Processing • Rearranging the OLTP workload queue • The different users work on the same or similar database schemas but do not share their transactional data. Rather, they maintain their private partitions of the data. • Only some read-mostly data is shared among the different users.

  20. Multi-Threaded OLTP Processing • Rearranging the OLTP workload queue • HyPer's OLTP process can be configured as multiple threads – to increase performance even further via parallelism. • As long as the transactions access and update only their private partition and access (not update) the shared datawe can run multiple such transactions in parallel

  21. Multi-Threaded OLTP Processing • Rearranging the OLTP workload queue • Transactions reading across partitions or updating the shared data partition require synchronization • HyPer-prototype cross-partition transactions request exclusive access (exclusive lock) to the system

  22. Multi-Threaded OLTP Processing • OLTP Transaction Synchronization • partition-constrained transactions can read and update the data in their own partition as well as read the data in the shared partition • partition-crossing transactions are those that, in addition, update the shared data or access (read or update) data in another partition

  23. EVALUATION • Memory Consumption

  24. Transaction • Isolation of OLAP Query Sessions • HyPeris a main-memory database system that guarantees the ACID properties of OLTP transactions • ACID • A - Atomicity • C - Consistency • I - Isolation • D - Durability

  25. Transaction • Isolation level • MS SQL

  26. Transaction • Dirty read • Nonrepeatable read • Phantom

  27. Conclusion • Our HyPer architecture is based on virtual memory supported snapshots on transactional data for multiple query sessions. • OLTP transactions and OLAP queries – are executed on the same data without interfering with each other • Real-time OLAP vs Big Data

  28. End

More Related