180 likes | 262 Views
Explore the world of data analytics and visualization, from demystifying big data hype to utilizing free tools for data storage and analysis. Learn to ask the right questions of your data and gain valuable insights for your business or organization. Discover how to make your data dance and leverage visualization techniques to communicate meaningful patterns effectively.
E N D
Make Your Data Dance Demystifying Data Analytics & Visualization
Today’s Agenda • This guy? • Definition & Discussion: “Big Data Hype” • What is an analytic? • How do we visualize • Demo: of Data Analytics and Visualization • Questions/Discussion
My Wife! This Guy? Creepy Kids My Wife Made
Big Data or Big Hype? • Its everywhere • We all hear it, but what does it mean? • Does it really mean anything or is it just more marketing hype? • Is bigger really better?
Logs Logs Everywhere • How many logs do we have now? • Too many to count • Not just on your file system, but in traffic too! • Human – Human • Machine – Human • Machine - Machine • Linux/Unix/Mac(BSD) • Microsoft • Bro Logs • Or plain Netflow generation • Snort or other IDS • Switches/Routers
Get Them In Your Database • How do you decide which logs you want? • Compliance • Policy • Curiosity • Just because • Normalization • On the fly (streams) • On the remote/local file system (batch)
Some Free Tools To Help • Tools for Transport: • Flume, fluentd, rsyslog, syslog-ng, sqoop, logstash • Tools for Storage: • Note: Relational/Non-relational is important • mySQL, cassandra, Hadoop (HDFS), Elasticsearch • Degree’s of Wholeness • ELSA, graylog2, Snare
Data is Big... But So What? • All data is not gold • You need a strategy that gets you the right data at the right time
Defining: Analytics • Wikipedia Definition – “the discovery and communication of meaningful patterns in data”
Simply a Question • Simple! • What! • A question?! • I can understand that! • These questions can be used to create • Metrics • Statistics • Network behaviors • These all help the application of Analytics as analytics help are used to create them.
Ask Questions of Your Data • I received an IDS alert, is there other similar behavior on my network that I did not receive an alert for? • I have an IP blacklist, what hosts on my network connected to those IP addresses? • Better yet, is there other similar behavior on my network to non–black-listed IP addresses?
What Other Kinds of Insight • Unpatched Systems • Misconfigured Devices • File access • Rates • Personnel • Visibility • Of your network • Of your hosts
Visualization. • So you normalized and stored the data • You’ve asked good questions of our data with analytics • Now what? • We visualize • But how?
Questions? Source links in the notes on this slide jlawler@21ct.com