Security analytics
1 / 62

Security Analytics - PowerPoint PPT Presentation

  • Uploaded on

Security Analytics. Dr. Char Sample February 2014. Agenda. Definitions & Tools Trends Cloud Big Data TIOT Behavior Summary Q&A. Section 1 : Definitions & Tools. Definitions.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Security Analytics' - tavia

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Security analytics

Security Analytics

Dr. Char Sample

February 2014


  • Definitions & Tools

  • Trends

    • Cloud

    • Big Data

    • TIOT

    • Behavior

  • Summary

  • Q&A

Section 1 definitions tools
Section 1: Definitions & Tools


  • Security Analytics – can be thought of as the process associated with developing insights to an environments actual security based on the inputs collected from the various security components.

  • Translation – Using data to inform a decision

Why do we care
Why Do We Care?

  • Security professionals are increasingly be required to be conversant in them.

    • Why?

      • Old solutions do not work.

      • The realization that the “easy button” is still elusive.

  • Analytics are where the “rubber meets the road”

  • Money! Analytics (for the time being) require human interactions and thought, generating JOBS.

Why analytics now
Why Analytics Now?

  • Drivers

    • Cloud computing

    • Big Data

    • Existing technology shortcomings

    • New devices.

  • Mostly, this effort is being driven by the need to see beyond the host and the LAN.

  • Situational awareness.

Why analytics now1
Why Analytics Now?

  • Changing roles and expectations for security professionals.

    • Inadequacies of scan and patch approach have been exposed.

    • Dynamic network environment

      • Exposes C&A shortcomings

      • Exposes architectural assumptions and shortcomings.

    • Attackers are becoming more stealth.

    • No clear solution leader has emerged, so DIY.

    • Uniqueness of environments.

Security analytic components
Security Analytic Components

  • Tools that we use to assist in gaining insight include:

    • Protocol Analysis

    • Traffic Analysis

    • Flow and Metadata

    • Logs (system, session, pcap, metadata, flow data)

    • System tools

    • Statistical models (Markov, Bayes, Fuzzy Logic, etc.)

    • Mathematical models (Clustering algorithms, neural networks)

Tools data
Tools & Data

  • Tools & Data Used for Analytic Creation

    • Protocol Analysis – IETF standards

      • Log data

      • Signature data

      • Packet capture data (pcap) – tcpdump

      • Anomalous data

Tools data1
Tools & Data

  • Tools & Data Used for Analytic Creation

    • Traffic Analysis – Data quantities

      • Flow data – IPFIX, others.

      • Metadata – IPFIX with statistical knowledge, others.

      • Log data – syslog, application logs, etc.

      • Signature data – IDS signatures

      • Packet capture data (pcap) – tcpdump

      • Anomalous data

    • Scripting languages - perl, python

Tools data2
Tools & Data

  • Protocol Analysis

    • Commonly used commandsvs less used commands (debug, trace, post vs put)

    • Commonly used parameters with commands (executable commands, wrong type parameters)

Tools data3
Tools & Data

  • Traffic Analysis*

    • Unexplained changes in volume to existing IP addresses and ports

    • New destination IP addresses, and ports

Tools data4
Tools & Data

  • Signature data

    • Commonly used.

      • NIDS, some firewalls, AV, anti-malware

    • Highly accurate with known attacks

      • Performs a basic pattern match, can be done by hardware.

      • Low false positives, but high false negatives

      • 10% effective at detecting new or 0 day attacks

    • Best use is in lowering false positives from AD, and keeping out less sophisticated attackers.

    • Ineffective against nation-state and other sophisticated attackers.

    • Removal of this type of data leaves analyst vulnerable to information overload.

Tools data5
Tools & Data

  • PCAP data

    • Entire packet

      • Preferably put together into sessions.

      • Storage is a problem

      • Legally admissible evidence

      • Provides details that other data types are incapable of providing.

    • Removal of this data type results in a lack of detailed understanding of events seen by flow and meta data.

Security analytic tools
Security Analytic Tools

  • DNS lookup tools: nslookup, dig, whois

  • Mapping tools: IP_address to ASN, traceroute, arptable, netstat

Security analytic tools dns info
Security Analytic Tools – DNS Info

  • DNS tool: nslookup

Security analytic tools dns info1
Security Analytic Tools – DNS Info

  • DNS tool: dig

Security analytic tools dns info2
Security Analytic Tools – DNS Info

  • DNS tool- whois: Internet domain name and network number directory service

Security analytic tools dns info3
Security Analytic Tools – DNS Info

  • DNS tool: whois (more from query)

Security analytic tools routing info
Security Analytic Tools – Routing Info

Security analytic tools routing info1
Security Analytic Tools – Routing Info

  • Tools – traceroute (limited use): print the path between hosts

Security analytic tools arp
Security Analytic Tools - ARP

  • Connectivity tool- ARP: address resolution display and control


  • Several tools are available to assist in determining the nature of the activity these tools are diagnostic tools.

    • Connectivity tools: ping, traceroute

    • Routing tools: ASN – IP address mappings

    • DNS tools: dig, nslookup, whois

    • Log data

Section 2 trends
Section 2: Trends

Cloud big data
Cloud & Big Data

  • The “Cloud” provides both the source of our data, and the ability to analyze it.

  • Cloud as defined by NIST – IaaS, PaaS, SaaS

    • Characterized by dynamic provisioning

    • Shared resources

Items to consider
Items to Consider

  • Virtual Machines

  • Network Infrastructure with multiple parties

    • Recall the earlier discussion on global routing and how routing works.

    • The importance of understanding the ASN relationships.

  • The role of layer 2 data

    • Remember the arp command

      • Multiple IP addresses associated with the same arp address indicates virtualization (cloud)

      • Even when arp addresses are unique there are certain ranges of arp addresses that are “fake” and are used with virtualization

  • Security products are layer 3 based so they typically do not see Layer 2 behavior.

Items to consider1
Items to Consider

  • Problems in virtual environment require analysis of the CSP’s environment.

    • If you are in a public cloud sharing space with other entities how will you know if their space has not been compromised?

    • How can you be sure that allocated resources are clean?

    • It is not enough to know that the security apparatus in your virtual environment is working, that is only one piece of the puzzle.

Items to consider3
Items to Consider

  • More on layer 2 data

    • Bugs in the hypervisor

    • Provisioning and de-provisioning.

      • Is data really wiped?

      • How are requests authenticated and processed?

Cloud iaas
Cloud IaaS

  • IaaS Analytic Basics

    • Understand the routing infrastructure of the cloud, all tenants and each cloud site.

    • Consider having data encrypted before it is sent to the cloud.

    • Understand the path between the CSP sites

      • Know the peering agreements between CSP locations.

      • Understand the routing between tenants. Most likely layer 2 technology will be used and virtual addresses will be set.

Cloud iaas1
Cloud - IaaS

  • IaaS Analytic Basics

    • Routing

      • Look for the sudden appearance of new routers in tenant space.

      • Look for changes in routes, especially tenants suddenly having new traffic come through their space.

      • Look for changes in the CSP’s virtual router routes or the switch that would allow cross tenant access.

Cloud iaas2
Cloud - IaaS

  • IaaS Analytic Basics

    • If CSP is managing DNS check delegation

    • If CSP is also running DNSSEC extra work must be done with DNSSEC key management issues, check the DNSSEC delegation dig +sigchase

    • Examine DNS data from inside the zone, cloud partition, also examine from the CSP and finally from an external point.

      • Verify views.

      • Disable recursion from external ANS.

      • Check DNS BIND logs, look for strange commands, executable statements, and other anomalous activity.

      • Examine for evidence of tunneling between tenants: queries from neighbors are a good clue.

      • Take note of queries to/from fast flux sites, especially those managed from other CSPs.

Cloud iaas3
Cloud - IaaS

  • IaaS Analytic Basics

    • DNS

      • Look for changes in DNS Server’s arp address.

      • Look for cross tenant DNS/TCP traffic.

      • Look for changes in tenant zone information that are out of sync with normal updates.

      • Look for changes in authoritative nameservers (SOA) advertising.

      • Look for changes in MX records that direct away from known tenant mail hubs.

      • Look for tenants advertising multiple domains, domains that they may not own, or fast flux behaviors.

      • Look for tenants that set up recursive resolvers for other tenants.

      • Look for tenants that forward requests to other tenants.

      • Look for tenants that set up DHCP servers for other cloud tenants.

Cloud iaas4
Cloud - IaaS

  • IaaS Analytic Basics

    • DNS

      • DNSSEC analytics

        • Look for changes in ZSK that are out of sync.

        • Look for changes in ZSK following new access to the server.

        • Look for activity surrounding changes to key management procedures or users for both ZSK and KSK

    • General

      • When dealing with a truly distributed attack it is possible that the IP addresses will change but the MAC addresses will be either the same or from the same pool of “fake” addresses. This would indicate that the attack is coming from a virtual environment.

Cloud paas
Cloud - PaaS

  • PaaS

    • How is PaaS being utilized?

      • Programmers are a common reason for the need for PaaS

      • Test data can also reveal information about the organization. How will that data be protected?

      • If programmers are using:

        • Is DEBUG available?

        • How is the development environment wiped?

        • What about object permanence?

      • On the development platform:

        • Check security logs for signs of activity on the cluster ports.

        • Check for odd changes in activity on cluster hosts and ports.

        • Check for evidence of misconfigurations.

Cloud paas1
Cloud - PaaS

  • PaaS - Hadoop

Cloud paas2
Cloud - PaaS

  • PaaS Analytics

    • Look for changes in activity levels on the following ports: 8020, 50010, 50020, 50030, 50060, 50070, 50075, 50090, 50100, 50105.

    • Look for activity from new hosts on the above ports.

    • Look for signs of imposter NameNode attempts

    • Look for new mac address for NameNode

    • Look for appearance of new DataNodes, especially in existing clusters.

    • Look for strange commands and/or parameters.

    • Check authentication logs.

Cloud saas
Cloud - SaaS

  • SaaS

    • Run lint checkers on new apps

    • Code review?

    • Signatures

    • Protocol Abuse

    • Command abuse

Big data
Big Data

  • The dynamic behavior of the cloud requires less traditional analysis to understand.

    • Unstructured, continuous data requires new methods

    • Finding patterns within Big Data is key to analytic development in the cloud

    • Architecturally, this will impact choices on security solutions.

Big data1
Big Data

  • Big Data

    • Volume – obvious

    • Variety – many new types of data are available in “big data” that haven’t been used by security in the past, and this data is unstructured, e.g. streaming.

    • Velocity – dynamic relationships, everything from flow, to proxies, to pcap come in. Analysts cannot keep up with the data.

    • The 3 V’s of BD makes flow data extremely important!

Big data2
Big Data

Big data3
Big Data

  • What does the Hadoop explanation mean, why do we care?

    • There are several potential exploit areas that must be considered.

      • The distributed nature of the cluster

      • The each individual cluster inter-node issues

      • Communications between the NameNode and the DataNodes

      • Additional copies of data used to support HA

      • Software vulnerabilities within MapReduce & various libraries.

Security analytics

  • The Internet of Things

    • NSA has declared 2014 the year of TIOT

    • Supervisory Control and Data Acquisition (SCADA) & Other Embedded Devices

      • Protocols: ARP, UDP, ICMP, DHCP,TCP, PPP, SNMP

      • Look for new entries into the arp table, look for new ports and new destinations.


  • Recognition that technical methods alone are insufficient.

    • Understanding attacker “patterns of thought”

    • Understanding and anticipating “automatic thought processes”

      • Group behaviors

      • Game theory

      • Cultural influences*

      • Psychological influences


  • The security products and services industry has failed to keep up with the changes in the industry.

  • Consultants are now being asked to define and create security analytics.

  • A good understanding of tools and capabilities will assist in most cases along with understanding of user requirements.

  • Cloud security adds a new dimension since other tenants, the supply chain and the platforms must all be considered.

  • TIOT will stress the signature and AD models.

  • Big Data requires thought models and new visualization techniques.

Thank you

Dr. Char Sample

+1 301.346.9953

Thank you!


  • PaaS - Hadoop

    • Namespace is stored in RAM on the Name Node and it contains:

      • Filenames

      • Location information

    • Name Node acts as the controller and queries the data nodes on behalf of the client.

      • Replication of data is standard and 3 copies is the default but this value is configurable.

      • Look for unexplained activity on HDFS ports.

      • Look for unexplained hosts accessing Hadoop ports.


  • Obtain info on BGP peers


  • SaaS

    • Check permissions.

    • Check activity levels

      • Traffic volume

      • Hosts accessing.

      • Changes in top users, or users using new features.

    • Check libraries

    • Evidence of object permanence

    • Evidence of cross tenant communications using apps

    • Evidence of one tenant acting as a server for another tenant.

Layer 7 analytics
Layer 7 Analytics

  • Analytics can be done at all layers of OSI

  • IP addresses and DNS provide basic background (against APTs)

  • Application and Web Server logs provide data capable of determining intent

  • These new logs sources are pushing the Big Data movement forward. Analysts are now asked to understand a far wider breadth then they have before in a much shorter period.

Security analytic tools1
Security Analytic Tools

  • DNS tool – nslookup: query Internet nameservers interactively

    • Easier than dig, but provides less information

    • Built in for Windows, Unix and MacOS

    • Does provide information on various entities

      • SOA

      • MX

      • A

      • IN PTR

      • CNAME

Security analytic tools dns info4
Security Analytic Tools – DNS Info

  • DNS tool: dig

Big data4
Big Data

  • Hadoop keying on common objects.

    • Framework for storing

      • Big Data

      • Distributed server clusters

      • Distributed analysis applications in each cluster. Nodes in the cluster used for storing data are known as data notes.

    • Apache

    • Rather than rely strictly on h/ware for HA Hadoop uses software.

      • This means that multiple copies of data have to be made in order to support HA

      • Library is used to detect failures at the application layer, delivery of HA services occurs on top of the cluster.

      • HDFS is a data warehouse which differs from a database.

      • Accessing Hadoop for queries relies on MapReduce

        • NameNode

        • DataNode