WTT 2014 - Workshop de Tendências Tecnológicas 2014

1. WTT 2014 - Workshop de Tendências Tecnológicas 2014

2. Agenda • NoSQL Concepts • MongoDB Concepts • MongoDB Demos

3. NoSQL databases? • “NoSQL” = “No SQL” = Not using traditional relational DBMS • “No SQL”  Don’t use SQL language Alternative to traditional relational DBMS • Flexible schema • Quicker/cheaper to set up • Massive scalability • Relaxed consistency  higher performance & availability • No declarative query language  more programming • Relaxed consistency  fewer guarantees

4. NoSQL Systems • Map Reduce Framework Originally from Google, open source Hadoop • Key-Values-Stores Google BigTable, Amazon Dynamo, Cassandra, HBase, … • Document Stores • Data model: (key, document) pairs • Document: JSON, XML, other semistructured formats • CouchDB, MongoDB, SimpleDB, … • Graph Database Systems Neo4j, FlockDB, Pregel, … • Big-Table Implementation Teradata, Exadata, GreenPlum, MonetDB, …

5. ACID versus BASE ACIDAtomicity, Consistency, Isolation, Durability  Traditional Databases CAP Strong Consistency + HighAvailability + Partition-tolerance The CAP-Theorem postulates that only two of the three different aspects of scaling out are can be achieved fully at the same time. Many of the NOSQL  BASEBasically Available, Soft-state, Eventually consistent 

6. Quiz. NoSQL Applications ? [ ] Web Log Analysis URL, timestamp, number of accesses [ ] Social-network graphs user1, user2, Find friends of a friends [ ] Wikipedia Pages Large collections, structured and unstructured data [ ] Twitter messages unstructured data [ ] Blog maintenance unstructured data [ ] Account credits and debts

7. MongoDB • MongoDB (from "humongous") is an open-source document database, and the leading NoSQL database. Written in C++, MongoDB features • Document-Oriented Storage • Querying • Full Index Support • Replication & High Availability • Auto-Sharding • Map/Reduce • Geospatial support • Text Search

8. DB-Engines Ranking

9. SQL to MongoDB Mapping Chart

10. JSON x SQL x BSON JSON JavaScript Object Notation. A human-readable, plain text format for expressing structured data with support in many programming languages. BSON A serialization format used to store documents and make remote procedure calls in MongoDB. “BSON” is a portmanteau of the words “binary” and “JSON”.

11. var p = { ‘_id’: ‘3432’, ‘author’: DBRef(‘User’, 2), ‘title’: ‘Introduction to MongoDB’, ‘body’: ‘MongoDB is an open sources.. ‘, ‘timestamp’: Date(’01-04-12’), ‘tags’: [‘MongoDB’, ‘NoSQL’], ‘comments’: [{‘author’: DBRef(‘User’, 4), ‘date’: Date(’02-04-12’), ‘text’: ‘Did you see.. ‘, ‘upvotes’: 7, … ] } > db.posts.save(p); JSON Document Model

12. Create Index on any field in the document • // 1 means ascending, -1 means descending • > db.posts.ensureIndex({‘author’: 1}); • //Index Nested Documents • > db.posts.ensureIndex(‘comments.author’: 1); • // Index on tags • > db.posts.ensureIndex({‘tags’: 1}); • // Geo-spatial Index • > db.posts.ensureIndex({‘author.location’: ‘2d’}); Indexes

13. // find posts which has ‘MongoDB’ tag. > db.posts.find({tags: ‘MongoDB’}); // find posts by author’s comments. > db.posts.find({‘comments.author’: DBRef(‘User’,2)}).count(); // find posts written after 31st March. > db.posts.find({‘timestamp’: {‘gte’: Date(’31-03-12’)}}); // find posts written by authors around [22, 42] > db.posts.find({‘author.location’: {‘near’:[22, 42]}); Queries? $gt, $lt, $gte, $lte, $ne, $all, $in, $nin, count, limit, skip, group, etc…

14. db.posts.update({_id: ‘3432’}, {‘title’: ‘Introduction to MongoDB (updated)’, ‘text’: ‘Updated text’, ${addToSet: {‘tags’: ‘webinar’}}); $set, $unset Updates? Atomic Operations $push, $pull, $pop, $addToSet $inc, $decr, many more… MongoDB does not support TRANSACTIONS !

15. • Geo-spatial Indexes for Geo-spatial queries. $near, $within_distance, Bound queries (circle, box) • Map/Reduce GROUP BY in SQL, map/reduce in MongoDB. • GridFS Stores Large Binary Files. Some Cool features, but not in this lab 

16. Demo: my bond girls database 

17. Demo: mongodb install http://www.mongodb.org/downloads http://docs.mongodb.org/manual/tutorial/install-mongodb-on-windows/ C:\mongodb\bin\mongod.exe --dbpath d:\test\mongodb\data

18. Demo: mongodb as a Service https://mongolab.com/welcome/

19. Demo: pymongo

20. Demo: mongolab create a account

21. Demo: after a Collection, create a document

22. Demo: find( ) with Python

23. Demo: find( ) with mongo shell

24. Demo: load a .CSV collection with mongo shell

25. Demo: results of load a .CSV collection

26. Demo: find( ) a document with mongo shell

27. Demo: find( ) a document with Python

28. Demo: find( ) select a field

29. References The little Mongodb Book by Karl Seguin http://openmymind.net/mongodb.pdf Mongodb http://docs.mongodb.org/manual/reference/sql-comparison/ http://www.mongodb.org/downloads http://docs.mongodb.org/manual/installation/ http://docs.mongodb.org/manual/tutorial/getting-started-with-the-mongo-shell/ http://docs.mongodb.org/manual/ https://www.mongodb.com/reference https://university.mongodb.com Mongodb-as-a-Service https://mongolab.com/welcome/ https://www.mongohq.com/ Pymongo, mongodb drive for Python http://api.mongodb.org/python/current/installation.html http://api.mongodb.org/python/current/tutorial.html Easy Install, to install pymongo for Windows https://pypi.python.org/pypi/setuptools

30. Conclusion: When to use MongoDB • Schema-less • Writes db.createCollection(‘logs’, {capped: true, size: 1048576}) • NO TRANSACTIONS • Data Processing ~ MapReduce • Geospatial • Full Text Search • NoSQL + tools and maturity

31. WTT 2014 - Workshop de Tendências Tecnológicas 2014 Prof. Dr. Rogério de Oliveira roger.oliveira@mackenzie.br http://meusite.mackenzie.br/rogerio