1 / 35

Network-level Malware Detection

Network-level Malware Detection. Mike McNett, Matthew Spear, Richard Barnes CS-851 – Malware 23 October 2004. Outline. Introduction: Design of a System for Real-Time Worm Detection Example 1: Detecting Early Worm Propagation through Packet Matching (DEWP)

Download Presentation

Network-level Malware Detection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Network-level Malware Detection Mike McNett, Matthew Spear, Richard Barnes CS-851 – Malware 23 October 2004

  2. Outline • Introduction: Design of a System for Real-Time Worm Detection • Example 1: Detecting Early Worm Propagation through Packet Matching (DEWP) • Example 2: Fast Detection of Scanning Worm Infections • Example Application: Therminator • Conclusions

  3. Introduction Questions Being Considered: • Why network level detection? • What are the alternatives? • Are there reasonable solutions? • What are the limitations, advantages, disadvantages compared to the alternatives?

  4. Introduction • Malware Detection Options? • Prevention vs. Treatment • Signature vs. Anomaly • Host-based containment • Network containment • Packet Header vs. Packet Payload • What are the advantages, disadvantages, and limitations of the above?

  5. Network-level Detection

  6. Design of a System for Real-Time Worm Detection • Hash • Count Vector • Character Filter • SRAM Analyzer • Alert Generator • Periodic Subtraction of Time Averages

  7. Design of a System for Real-Time Worm Detection • Scalable to high throughput • Solution depends on specialized hardware • Low false positive rate • What are the problems? • What are the advantages? • Are there other, more simplistic signatures? • Can similar attacks be detected at the host level?

  8. Detecting Early Worm Propagation through Packet MatchingXuan Chen and John Heidemann ISI-TR-2004-585February 2004

  9. DEWP • Router-based system: • automatically detects and quarantines Internet worm propagation • matches destination port numbers between incoming and outgoing connections (automated signature creation) • detects and suppresses worms due to unusual traffic patterns • detects worm propagation within about 4 seconds • protects > 99% hosts from random-scanning worms

  10. DEWP Thesis • Matches destination port numbers between incoming and outgoing connections. Two observations on worm traffic: • Worms usually exploit vulnerabilities related to specific network port numbers • Infected hosts will probe other vulnerable hosts exploiting the same vulnerability • So… high levels of bi-directional probing traffic with the same destination port number new worm • Scalable: Matching destination port numbers consumes low computational power

  11. DEWP • Two components of DEWP: worm detector and packet filter • Two step detection: destination port matching and destination address counting • Uses packet filtering to suppress worm spreading • Can deploy at different levels of network

  12. Worm Containment • DEWP uses traffic filtering – routers drop packets with the automatically discovered destination port • Worm containment: protect internal hosts from internal and external threats; notify other networks about attacks

  13. Design • Maintains one port-list for each direction (incoming and outgoing): records number of connections to different destination ports • Timer for each entry in port-lists: • If port has not been accessed for certain time interval, reset corresponding list entry • Monitor outgoing destination addresses of non-zero entries in both port-lists • Every T seconds, check number of unique addresses observed within last time interval. Worm traffic detected with the following condition: • Nis the number of unique addresses observed. • Long-term average: •  is the system sensitivity to changes

  14. Effectiveness of Worm Detection and Quarantine • Random scanning worm: detects worm traffic in 4.8 seconds when fully deployed with a 1 second detection interval. • Always detects worm probing traffic in 4-5 seconds when deployed to different layers. • Number of infected hosts in the protected network – primarily determined by the number of probing packets received from outside • Can protect almost all hosts from infection when only deployed on the access router.

  15. Local Scanning • Local scanning: Can detect worm probing traffic in 3.87 seconds. But, almost all vulnerable hosts in the protected network are compromised • Deployment has little impact on either detection delay or infection percentage. • The infection percentage increases as number DEWP deployed layers are reduced: When only on the access router  all vulnerable hosts compromised within 10 seconds • More frequent detection reduces vulnerability to local-scanning worms • DEWP quickly detects worm attacks regardless probing techniques. • With full deployment about 9% vulnerable hosts compromised in the protected network • Due to difficulty to effectively quarantine local-scanning worms  a very small detection interval and wide deployment is critical to protect vulnerable hosts

  16. Effect of Detection Intervals • Address-counting with an interval of T seconds. • Different detection intervals affect detection delay and infection percentage • Random-scanning worm. Detection delay and the number of infected hosts increases with detection intervals. • Local-scanning worms: 1) No significant difference in detection delay; 2) Infection percentage increases dramatically at larger intervals: • So, automatic system needs to react to worm traffic within small time intervals

  17. False Detections • No false positives • Discovered ~10 suspicious destination ports including 21 (FTP), 53 (DNS), and 80 (Web) • Depends on address-counting to reduce false positives • Worm scan rate C affects false negatives: when worm scan at low rate, probing traffic has less effect on overall traffic. DEWP routers have more difficulty distinguishing them from normal traffic. • With C = 500  worm traffic stands out compared to regular traffic • DEWP is not able to detect worms with scanning rate lower than C = 25.

  18. Conclusions • Detects and quarantines propagation of Internet worms • Uses port-matching and address-counting as the signature. • Detects worm attack within 4-5 seconds • By automatically blocking worm traffic, it protects most vulnerable hosts from random-scanning worms. • Authors believe that an automatic worm detection and containment system should be widely deployed and have very small detection intervals • Not realistic to deploy DEWP on all routers – for random scanning worms – sufficient to put on access router.

  19. Worm Detection Fast Detection of Scanning Worm Infection

  20. Detection Techniques • Reverse Sequential Hypothesis Testing (TH) • Detects worms based upon number of failed connection attempts • Uses probability to determine if a local host is scanning • Designed to be tied into a containment system • Signature Based Analysis (Early Bird System (EBS)) • Detects worms based upon Rabin signatures of content/port • Used in conjunction with a containment system

  21. Definitions

  22. Definitions

  23. Basic Algorithm • Maintain separate state information for each host (l) being monitored ( ), the hosts that have been previously contacted, and an FCC queue (FCCQ) of first contact attempts that have been attempted but have not been recorded in the observation (PCH). • When a packet is observed check to see if d is in the PCH ofl, if not then add d PCH andadd the attempt to FCCQ as PENDING. • When an incoming packet is sent to l and the source address exists in FCCQ update the record to SUCCESS in the FCCQ unless the packet is a TCP RST. • When the head entry of FCCQ has status of PENDING and has been in queue for longer than a predefined time limit set its status to FAILURE. • If the entry at the head of FCCQ has status other than PENDING update and compare it to η1

  24. Basic Algorithm Credit Based Connection Rate Limiting (CBCRL) • Simple scheme to limit the amount of connections l can make in a given slot of time by allotting each l a set number of credits (Cl) that is modified given events. • Used in conjunction with TH to limit number of connections a host can make allowing TH time to determine if a host is infected.

  25. Experiment • Conducted two experiments in 2003 (isp-2003) and 2004 (isp-04). • Worms identified via comparing traffic to known worm descriptions.

  26. Results

  27. Limitations, Future Work? Are there any serious flaws in this algorithm? Future work? • Warhol type scanning • Network outages can cause TH to decide that a host is a worm • Worms could conceivably collaborate to defy detection • Worms could remember hosts that it can contact and defy detection through them • Spoofing attack to get an uninfected host blocked • Interleave scanning with benign activities (i.e. for every scan visit a website that is known to be running) • Can trivially modify to work with the containment strategies discussed earlier

  28. THERMINATOR!!! Science comes to the aid of network-level anomaly detection

  29. Network behavior is complicated • How do we use “microscopic” packet-level data to make “macro” network-level decisions? • Too broad, e.g. keeping track of global traffic patterns. • Too refined, e.g. looking at individual packets. • Hmm… who else tries to make sense of the overall behavior of millions of single objects? • Physicists and Chemists!

  30. Idea • Given a computer network with >1000 nodes, • Want to detect anomalous traffic, without any foreknowledge. • Idea of THERMINATOR • Take advantage of lots of packet-level data. • Use physical techniques to distill information into relevant statistics: Temperature, entropy, etc.

  31. Data Reduction • Take the set of hosts and group them into “buckets” or “conversation groups”. • Observe communication among buckets. • Calculate physical statistics based on these higher-level communications. • By virtue of the mathematics, these are guaranteed to be the same as if we’d just looked at hosts.

  32. Physical Network Visualization • Based on reduced data, we know pseudo-physical statistics: • Bucket size • Temperature • Entropy • Heating rate • Work rate • Visualizing these data shows network events. Image courtesy of DISA

  33. Network Event Detection

  34. THERMINATOR Implementation • Jointly developed by DISA, NSA, and Lancope Inc. • Uses Lancope’s data-collection hardware to provide data to THERMINATOR. • THERMINATOR reduces data, computes stats, and provides visualization. • “Research tests validated that THERMINATOR detected anomalies that the intrusion detection systems did not capture.” -- NSA

  35. Conclusion • Combined approaches (host-based, network-based, visualization)? • Can signatures be automatically generated? • Can attacks be visualized? • Potential impacts of false positives (is the medicine worse than the sickness) and automated containment? • Need different solutions for local-scanning vs. non-local scanning worms? • Are there other scientific areas that malware research can leverage?

More Related