1 / 26

A Tale of Two Technologies

A Tale of Two Technologies. A Story of the IoT “revolution”. It was the age of connectedness. It was the age of disconnectedness. Failure tolerance is not optional. It was the age of wisdom. It was the age of foolishness. It was the epoch of complexity. It was the epoch of simplicity.

allie
Download Presentation

A Tale of Two Technologies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Tale of Two Technologies A Story of the IoT “revolution”

  2. It was the age of connectedness

  3. It was the age of disconnectedness Failure tolerance is not optional

  4. It was the age of wisdom

  5. It was the age of foolishness

  6. It was the epoch of complexity

  7. It was the epoch of simplicity

  8. What is the Internet of Things? • “The Internet of Things (IoT) refers to uniquely identifiable objects and their virtual representations in an Internet-like structure.” • -Wikipedia

  9. No really what is IoT? • It’s literally the act of connecting “things” to the Internet • It predates the World Wide Web • It shouldn’t be surprising to anybody

  10. So IoT is old news? Most definitely

  11. Medical devices Sensors Fitness trackers Home security and automation Your (driverless?) car Industrial equipment So IoT is just hype? NO!

  12. Been there done that. • DataStax and Cassandra have been dealing with event streams (time series) information at scale for years. • But Cassandra is relatively dumb storage. • Put data in. Get data out.

  13. What is Cassandra? Chocolate! • A massively scalable distributed database • Chooses availability over strong consistency (yes, that really is a fundamental tradeoff) • With its wide partitions it is able to take advantage of data locality even at “web scale”

  14. What is Spark? Peanut butter! • DAG is a logical superset of M/R • Adopts much of the Hadoop ecosystem, without being bound by it • Intelligent use of caching (RDDs) for massive performance gains • Incorporate Streaming to make ingestion-time processing a first class citizen

  15. What does IoT need from big data? • Log time-series events -- at scale • Gather meaning from that data -- at scale • Report on that data -- at scale • Take action on that data -- at scale

  16. Logging events at scale

  17. Gathering meaning at scale

  18. Reporting at scale • Canned reports • Ad-hoc querying and reporting • Drill down / exploratory • Alerting • Aggregation • Clustering (K-means, et al) • Generalized machine learning

  19. Take action at scale • Stateless application servers • Horizontally scaled and co-located with Cassandra and Spark in each DC • Any platform with a CQL driver

  20. The architecture...

  21. Spark to Cassandra

  22. Cassandra to Spark

  23. Multi-DC DC2 DC1 Write anywhere.

  24. Things that go together

  25. Things that go together

  26. A Tale of Two Summits

More Related