Introduction à Couchbase Server 2.0

2. Introduction àCouchbase Server 2.0 Tugdual GrallTechnicalEvangelist@tgrall

3. {“about” : “me”} • Tugdual “Tug” Grall • Couchbase • Technical Evangelist • eXo • CTO • Oracle • Developer/Product Manager • Mainly Java/SOA • Developer in consulting firms • Web • @tgrall • http://blog.grallandco.com • Tgrall • NantesJUG co-founder • Pet Project : • http://www.resultri.com

4. Agenda • Introduction to Couchbase Server • Cluster Management • Operation and Document Design • Introduction to Views and Queries • Cross Data Center Replication • Q&A Couchbase Developer Day Paris is this Friday! http://couchbase.com/events

5. Couchbase Server 2.0

6. Couchbase Server NoSQL Document Database

7. Couchbase Open Source Project • Leading NoSQL database project focused on distributed database technology and surrounding ecosystem • Supports both key-value and document-oriented use cases • All components are available under the Apache 2.0 Public License • Obtained as packaged software in both enterprise and community editions. Couchbase Open Source Project

8. Couchbase Server Easy Scalability Consistent High Performance Grow cluster without application changes, without downtime with a single click Consistent sub-millisecond read and write response times with consistent high throughput Always On 24x365 Flexible Data Model No downtime for software upgrades, hardware maintenance, etc. JSON document model with no fixed schema.

9. New in 2.0 JSON support Indexing and Querying Incremental Map Reduce Cross data center replication

10. Couchbase Server 2.0 Architecture 11211 Memcapable 1.0 8092 Query API 11210 Memcapable 2.0 Moxi Query Engine Memcached Couchbase EP Engine Data Manager Cluster Manager REST management API/Web UI Heartbeat Process monitor Node health monitor Rebalance orchestrator Configuration manager vBucket state and replication manager Global singleton supervisor storage interface New Persistence Layer http on each node one per cluster Erlang/OTP HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 - 21199

11. REST management API/Web UI Heartbeat Process monitor Configuration manager Node health monitor Rebalance orchestrator Global singleton supervisor vBucket state and replication manager http on each node one per cluster Erlang/OTP HTTP 8091 Erlang port mapper 4369 Distributed Erlang 21100 - 21199 Couchbase Server 2.0 Architecture 11211 Memcapable 1.0 8092 Query API 11210 Memcapable 2.0 RAM Cache, Indexing & Persistence Management (C & V8) Server/Cluster Management & Communication (Erlang) Moxi Query Engine Object-level Cache Couchbase EP Engine storage interface New Persistence Layer Disk Persistence The Unreasonable Effectiveness of C by Damien Katz

12. Single node - Couchbase Write Operation App Server Couchbase Server Node 3 3 2 Managed Cache Replication Queue To other node Doc 1 Doc 1 Doc 1 Disk Queue Disk

13. Single node - Couchbase Read Operation GETDoc 1 App Server Couchbase Server Node 3 3 2 Managed Cache Replication Queue To other node Doc 1 Doc 1 Doc 1 Disk Queue Disk

14. Docs distributed evenly across servers Each server stores both active and replica docs Only one server active at a time Client library provides app with simple interface to database Cluster map provides map to which server doc is on App never needs to know App reads, writes, updates docs Multiple app servers can access same document at same time Basic Operation App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library Cluster Map Cluster Map READ/WRITE/UPDATE Server 1 Server 2 Server 3 Active Active Active Doc 2 Doc 7 Doc 8 Doc 1 Doc 4 Doc 5 Doc 9 Doc 4 Doc 2 Doc 8 Doc 1 Doc 9 Doc 7 Doc 2 Doc 5 Doc 3 Doc 6 Doc 6 Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc REPLICA REPLICA REPLICA Couchbase Server Cluster User Configured Replica Count = 1

15. Two servers addedOne-click operation Docs automatically rebalanced across cluster Even distribution of docs Minimum doc movement Cluster map updated App database calls now distributed over larger number of servers Add Nodes to Cluster App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library Cluster Map Cluster Map READ/WRITE/UPDATE READ/WRITE/UPDATE Server 4 Server 5 Server 1 Server 2 Server 3 Active Active Active Active Active Doc 6 Doc 3 Doc 7 Doc 1 Doc 4 Doc 4 Doc 1 Doc 5 Doc 2 Doc 9 Doc 2 Doc 9 Doc 2 Doc 8 Doc 5 Doc 8 Doc 7 Doc 6 Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc REPLICA REPLICA REPLICA REPLICA REPLICA Couchbase Server Cluster User Configured Replica Count = 1

16. Fail Over Node App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library Cluster Map Cluster Map • App servers accessing docs • Requests to Server 3 fail • Cluster detects server failed Promotes replicas of docs to active Updates cluster map • Requests for docs now go to appropriate server • Typically rebalance would follow Server 4 Server 5 Server 1 Server 2 Server 3 Active Active Active Active Active Doc 6 Doc 4 Doc 3 Doc 6 Doc 2 Doc 3 Doc 4 Doc 5 Doc 2 Doc 7 Doc 1 Doc 1 Doc 7 Doc 9 Doc 5 Doc 1 Doc 8 Doc 2 Doc 8 Doc 9 Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc REPLICA REPLICA REPLICA REPLICA REPLICA Couchbase Server Cluster User Configured Replica Count = 1

17. Demo Time

18. Operation & Document Design

19. Basic Store & Retrieve Operations • get(key)Retrieve a document • set(key, value)Store a document or replace if it exists • add(key, value)Store a document and fail if it exists • replace(key, value)Replace a document and fail if it does not exist

20. Lots of other Operations • Atomic Counters (increment, decrement) • Append/Prepend • CAS (Compare and Set) • Optimistic Locking • “Get with Lock” • Write Lock on Objects • Bulk Operations • Saves network overhead • Stats • View operations

21. Couchbase 2.0 Bonus Points: JSON • JSON is a lightweight format to represent document structure in a language-independent manner. • If JSON documents are stored, the View engine can be used. • Allows to build secondary indexes on your datasets. • Makes it possible to ask questions like“Give me all user documents by lastname”.

22. Flexible Data Model { “ID”: 1, “FIRST”: “Dipti”, “LAST”: “Borkar”, “ZIP”: “94040”, “CITY”: “MV”, “STATE”: “CA” } JSON JSON JSON JSON • No need to worry about the database when changing your application • Records can have different structures, there is no fixed schema • Allows painless data model changes for rapid application development

23. Aggregate View of Data http://martinfowler.com/bliki/AggregateOrientedDatabase.html

24. Store and Retrieve Aggregates • order::1001 • { • uid: ji22jd, • customer: Ann, • line_items: [ • { sku: 0321293533, quan: 3, unit_price: 48.0 }, • { sku: 0321601912, quan: 1, unit_price: 39.0 }, • { sku: 0131495054, quan: 1, unit_price: 51.0 } • ], • payment: { type: Amex, expiry: 04/2001, • last5: 12345 } • } • Easier to Distribute Data • More Flexibility • Reduced Latency

25. Demo Time

26. Ruby Clojure Go Python Official SDKs www.couchbase.com/develop Community SDKs

27. Indexing and Querying

28. Indexing and Querying – The basics • Define materialized views on JSON documents and then query across the data set • Using views you can define • Primary indexes • Simple secondary indexes (most common use case) • Complex secondary, tertiary and composite indexes • Aggregations (reduction) • Indexes are eventually indexed • Queries are eventually consistent • Built using Map/Reduce technology • Map and Reduce functions are written in Javascript

29. Indexing and Querying App Server 1 App Server 2 COUCHBASE Client Library COUCHBASE Client Library Cluster Map Cluster Map • Indexing work is distributed amongst nodes • Large data set possible • Parallelize the effort • Each node has index for data stored on it • Queries combine the results from required nodes Server 1 Server 2 Server 3 Active Active Active Query Doc 9 Doc 4 Doc 1 Doc 8 Doc 2 Doc 2 Doc 8 Doc 4 Doc 5 Doc 8 Doc 5 Doc 9 Doc 9 Doc 5 Doc 1 Doc 4 Doc 1 Doc 2 Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc REPLICA REPLICA REPLICA Couchbase Server Cluster User Configured Replica Count = 1

30. Demo Time

31. Cross Data Center Replication(XDCR)

32. Cross Data Center Replication – The basics • Replicate your Couchbase data across clusters • Clusters may be spread across geos • Configured on a per-bucket (per-database) basis • Supports unidirectional and bidirectional operation • Application can read and write from both clusters • Active – Active replication • Replication throughput scales out linearly • Different from intra-cluster replication

33. Cross Data Center Replication (XDCR) Server 2 Server 3 Server 1 Couchbase Server Cluster US Active Active Active RAM RAM RAM DISK DISK DISK Doc Doc Doc Doc Doc Doc Doc 9 Doc 2 Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc Doc 9 Doc Doc Doc Doc 2 Doc Doc Doc Doc Doc Doc Doc Doc Server 2 Server 3 Server 1 Active Active Active RAM RAM RAM Couchbase Server Cluster EMEA DISK DISK DISK

34. Couchbase Handles Real World Scale

35. Q & A

36. Couchbase DeveloperDays Paris • This Friday ! • Presentations & Labs • Information • http://www.couchbase.com/events • Use the promo code TUG

37. tug@couchbase.com@tgrall Thank you! Get Couchbase Server at http://www.couchbase.com/download