Detection of Encrypted Traffic in Peer-to-Peer Network

1. Detection of Encrypted Traffic in Peer-to-Peer Network Mário M. Freire Instituto de TelecomunicaçõesDepartamento de InformáticaUniversidade da Beira Interior (mario@di.ubi.pt) Ciência 2010 – Encontro com a Ciência e a Tecnologia em PortugalLisboa, 4-7 de Julho de 2010

2. Overview Overview About Peer-to-Peer Systems Methods for P2P Traffic Classification Deep Packet Inspection Behaviour-based Methods Hybrid Method for P2P Traffic Detection P2P Traffic Detection Using a Behavioural Method Based on Entropy Method for P2P Traffic Detection Using Deep Packet Inspection Main Conclusions

3. Main features Scalability Resiliency Redundancy Overview About Peer-to-Peer Systems • Advantages • Less expensive; • More fault tolerance; • It is possible to put the services in points of the network where they are more needed. • Disadvantages • Security and legal issues.

4. Overview About Peer-to-Peer Systems • Functional classification of P2P application • Management and contents sharing (eg: BitTorrent) • Distributed processing (eg: Seti@Home) • Collaboration and Communication (eg: MSN) • Degree of Decentralization • Purely decentralized systems (eg: eMule, GNUtella) • Partially decentralized systems (DirectConnect) • Hybrid decentralized systems (BitTorrent) • Centralized (Napster) • Structure of the Information System • Unstructured systems (eg: GNutella) • Structured systems (Chord, CAN, Pastry, BitTorrent) • Loosely structured systems(Freenet)

5. Overview About Peer-to-Peer Systems • This work is focused on the corporation perspective of P2P applications; • The traffic generated by P2P file-sharing and P2P TV applications may compromise the performance of critical networked applications or network-based tasks in corporations/institutions.

6. Traditional Traffic Classification: Based on port number obsolete! Methods for P2P Traffic Classification • Current Methods for P2P Traffic Classification • Payload Inspection, Deep Packet Inspection or Signature-based Detection • Based on Flow Traffic Behaviour or Classification in the Dark

7. Traditional trafficclassification: Based on port number obsolete! Methods for P2P Traffic Classification Search for traffic in the ports that are usually used by known peer-to-peer applications. Unable to classify: • new or unknown protocols; • applications that choose a random port number; • applications that disguise the traffic using ports usually used by different protocols (80, 25, 110, …).

8. Deep Packet Inspection Most of the already known P2P protocols may be identified by patterns contained in the payload of an IP packet. Underlying approach: Search for specific signatures (string series) in the payloadof IP packets.

9. Problems: new or unknown protocols; encrypted payloads; legal issues; heavy computation needed to process huge portions of traffic at very high bitrates and/or low latency communications. Deep Packet Inspection • Useful for: • accurate protocol identification; • well know protocols; • non evasive applications; • mechanisms for service charging systems.

10. Underlying approach: identifying patterns, in the traffic behaviour, without looking into the payload contents. Behaviour-based Methods • There are several mechanisms for traffic classification based on traffic behaviour: • Patterns are identified on several traffic characteristics as: • (IP, ports) pairs; • number of connections; • TCP flags; • inter arrival times.

11. Different mechanisms have been investigated to classify traffic using behaviour patterns: Statistical Mechanisms. Statistical methods usually rely on flow and packet level properties of the traffic, such as the flow duration and size, inter-arrival times, IP addresses, TCP and UDP port numbers, TCP flags, packet size, etc; Heuristics Based Methods. Many behavioral mechanisms for traffic classification are based on a predefined set of heuristics. Typical heuristics include the source-destination IP pairs that use both TCP and UDP, the number of distinct addresses and ports a user is connected to, etc. Behaviour-based Methods

12. Different mechanisms have been investigated to classify traffic using behaviour patterns: Machine Learning Techniques. A large part of the studies propose classification mechanisms based on different supervised or unsupervised ML techniques,such as Bayesian estimators or networks, clustering, and decision trees. Behaviour-based Methods

13. Useful for: Unknown protocols; Encrypted traffic; Public networks under data protection laws. Behaviour-based Methods Disadvantages: • Lack of accuracy • Unsuitable for service charging systems.

14. Hybrid Method for P2P Traffic Detection Proposed approaches up to now may fail to identify Peer-to-Peer traffic when: • traffic is encrypted; • payload signatures for a new protocol are unknown; • the aggregation point may such a heavy load that may become infusible to deeply inspect all the packets under high-speed and/or low latency operation. Starting PointHybrid Method for P2P Traffic Detection

15. Hybrid Method for P2P Traffic Detection Hybrid Method for P2P Traffic Detection Combines both strategies: • Flow Traffic Behaviour or Classification in the DarkP2P Traffic Detection Using a Behavioural Method Based on Entropy More details for instance in:“Analysis of Peer-to-Peer Traffic Using a Behavioural Method Based on Entropy”, Proc. IEEE Int. Performance, Computing and Communications Conf. (IPCCC 2008), pp. 201 – 208. • Deep Packet Inspection (Signature-based Detection)

16. Hybrid Method for P2P Traffic Detection • Deep Packet Inspection (Signature-based Detection) More details for instance in: David A. Carvalho, Manuela Pereira and Mário M. Freire"Towards the Detection of Encrypted BitTorrent Traffic Through Deep Packet Inspection“, in Security Technology, Communications in Computer and Information Science, CCIS 58, Springer-Verlag, Berlin Heidelberg, December 2009, ISBN: ISBN: 978-3-642-10846-4, pp. 265–272, 2009 (Invited Paper).Mário M. Freire, David A. Carvalho, and Manuela Pereira"Detection of Encrypted Traffic in eDonkey Network Through Application Signatures"Proceedings of 2009 1st International Conference on Advances in P2P Systems (AP2PS 2009), Sliema, Malta, October 11-16, 2009IEEE Computer SocietyPress, Los Alamitos, CA, ISBN: 978-0-7695-3831-0, pp. 174 - 179.

17. We considered classification in the dark, not as an alternative, but as a complement to DPI techniques. The module for traffic classification in the dark can be used cooperatively with deep packet inspection techniques, concurrently or sequentially. Classification in the dark can be used : only if DPI methods are unable to classify the traffic; or, in every case, in cooperation with DPI methods. Hybrid Method for P2P Traffic Detection

18. One can say that the Entropy reflects the degree of certainty (or uncertainty) of a given variable. From a rough perspective, and for the sake of simplicity, we will just say that it can also disclose the heterogeneity of a pool of sample values, observed for a given period of time. P2P Traffic Detection Using a Behavioural Method Based on Entropy Entropy: Maximum Entropy Value: where n is the size of the values pool Heterogeneity High Low Entropy High Low Source: J. Gomes, P. Inácio, M. Freire, M. Pereira, P. Monteiro, IEEE IPCCC 2008

19. P2P Traffic Detection Using a Behavioural Method Based on Entropy Peer-to-Peer traffic presents a bigger heterogeneity between packet size values, when compared with other traffic classes. A simple HTTP download from the Web Download traffic using eMule file sharing application VoIP call using Skype Source: J. Gomes, P. Inácio, M. Freire, M. Pereira, P. Monteiro, IEEE IPCCC 2008

20. Skype VoIP 1 3.729 Skype VoIP 2 3.698 MSN VoIP 1 3.260 Google Talk VoIP 2.855 eMule download 2 2.498 BitTorrent 2.273 Skype IM 2 2.153 eMule download 1 2.141 MSN IM 1 1.959 Google Talk IM 2 1.917 Skype IM 1 1.886 eMule upload 1 1.843 Google Talk IM 1 1.810 MSN VoIP 2 1.740 MSN IM 2 1.612 HTTP 1.427 eMule upload 2 1.334 Live Streaming 1 1.278 sFTP download 1.004 Streaming download 1 0.772 Live Streaming 2 0.639 sFTP upload 0.552 Download from Web 4 0.352 Streaming download 2 0.282 Download from Web 3 0.175 Download from Web 2 0.073 Mail download 0.050 Download from Web 1 0.014 P2P Traffic Detection Using a Behavioural Method Based on Entropy • Several measures were tested • variance • mean • amplitude • However, Entropy is the measure that better reflects the heterogeneity of the packets size. • For several traces containing traffic from different classes, we calculated the entropy value for a sliding window of 100 packets. • Traces containing Peer-to-Peer traffic were, almost perfectly, organized in the top of a table containing the average of the entropy value. Source: J. Gomes, P. Inácio, M. Freire, M. Pereira, P. Monteiro, IEEE IPCCC 2008

21. P2P Traffic Detection Using a Behavioural Method Based on Entropy The approach was tested for traces containing mixed traffic from several applications. The results were depicted in charts. Source: J. Gomes, P. Inácio, M. Freire, M. Pereira, P. Monteiro, Submitted for publication

22. Method for P2P Traffic Detection Using Deep Packet Inspection • The methodology used for the detection of P2P traffic makes use of an open source and widely used intrusion detection system, called SNORT. • The identification of signatures associated with application packets was made manually through the observation of repetitive patterns in the payload of a sequence of packets generated by a P2P application, even with obfuscation (encryption of the payload). • The signatures in payloads of P2P applications to identify are expressed in terms of SNORT rules.

23. Method for P2P Traffic Detection Using Deep Packet Inspection • Using this methodology, we developed SNORT rules for the detection of P2P traffic generated by the following applications: BitTorrent, Vuze, eMule, aMule, Limewire and GTK-Gnutella. • A complete set of SNORT rules for those P2P applications is available in NMCG Lab at:http://floyd.di.ubi.pt/nmcg/pdf/snortrules.pdf • Particular attention is being paid to encrypted traffic.

24. Method for P2P Traffic Detection Using Deep Packet Inspection Protocol Application BitTorrent BitTorrent Vuze eDonkey eMule aMule Gnutella Limewire Gtk-Gnutella P2P TV Livestation TVUPlayer Goalbit

25. Experimental Testbed for P2P Traffic Detection

26. Experimental Testbed for P2P Traffic Detection Characteristics of hardware and softwareused in the testbed for eMule traffic detection

27. Experimental Testbed for P2P Traffic Detection • In all lab experiences reported here, Snort was forced to analyse other network traffic than P2P, like HTTP, Windows Remote Desktop Connection (RDC), SSH, etc. • In fact, this was quite worthy, since it enabled the testbed to run in similar circumstances of those of deployed P2P classifiers, which also have to deal with network traffic generated by a vast number of applications and then to correctly identify P2P among it.

28. Experimental Testbed for P2P Traffic Detection • Experimental results presented in the next tables for the most triggered rules were obtained through the download of media objects such as the documentary “Inside the Space Shuttle”. • Tables with experimental results show the effectiveness of the proposed Snort rules to detect plain or encrypted traffic generated by eMule. • Example of a SNORT Rule: Snort Rule 1000307:alert udp $EXTERNAL_NET any -> $HOME_NET any (msg:"LocalRule: P2P BitTorrent UDP - Incoming DHT for trackerless comunication request (d1:ad2:id20)"; content:"d1:ad2:id20"; nocase; depth:11; classtype:policy-violation; sid:1000307; rev:3;)

29. Experiments When Obfuscation Is Not Used

30. Experiments Using Obfuscation

31. Main Conclusions • We presented an overview about peer-to-peer networks and approaches for the detection of P2P Traffic. • We proposed a new Hybrid Method for P2P Traffic Detection. • Several lab experiments were carried out to validate the proposed method and to evaluate its accuracy.

32. Acknowledgements Acknowledgements FCT PTDC/EIA/73072/2006 TRAMANET Project: Traffic and Trust Management in Peer-to-Peer Networks Thank youfor your attention! Questions?