Protecting the Internet: Risks and Attacks on BGP Protocol

1. IP Backbone Security > Nicolas FISCHBACH IP Engineering Manager - COLT Telecom nico@securite.org - http://www.securite.org/nico/ > Sébastien LACOSTE-SERIS IP R&D Manager, Security Officer - COLT Telecom kaneda@securite.org - http://www.securite.org/kaneda/ version 1.0

2. Agenda • Introduction • External Gateway Protocol : BGP • Internal Gateway Protocols : OSPF and ISIS • Netflow based DDoS detection • MPLS and IPv6 • Conclusion

3. What « runs » the Internet ? • Key protocols • BGPv4 (Border Gateway Protocol) • DNS (Domain Name System) • A mix of BGP and DNS in all new/recent technologies • DNS to store the information in new/extended type of records • BGP to distribute the information across the network • A few large vendors • Limited range of ASIC powered devices • Some well known software release versions/trains (ie S-train) • Non-technical stuff : people in the NOCs & coffee :-)

4. What are the risks, if any ? (1) • The Internet is considered a critical infrastructure • But the trust model hasn’t really changed since the 70s • Can it survive a “chapter 11” from a really large provider ? • “Slashdot effect” (major links and websites -- even those with distributed content). Could larger deployment of multicast help ? • Best Current Practices make the network more resistant, but are not followed or deployed • Recent publications • IETF/IRTF Working Groups

5. What are the risks, if any ? (2) • Current, new and future type of attacks • Misconfiguration is the most common “attack” • (D)DoS with spoofed source addresses • Kill your network / (IRC) servers • Use of routers as reflectors/amplifiers • No reliable traceback • Short-lived announcements used as source of SPAM/attacks • Advanced routing protocols attacks • Make your (internal/external) routing protocol unstable • Inject new routes/prefixes : MiTM/traffic rerouting attacks • Rootkits and Loadable Kernel Modules • Take control of a device capable of generation thousands of PPS (packets per second) • Control all the routing protocol traffic

6. BGP : Protocol description • BGP (Border Gateway Protocol) • Current version : 4 • Listens on port 179/tcp • Optional authentication : • MD5 : adds an option to TCP (digest based on pseudo-header+header+data+shared password) • Point-to-point over directly connected interfaces or multi-hop between (TTL > 1) non adjacent routers • Routing information is exchanged in BGP Update message :

7. BGP : Protocol description • BGP (Border Gateway Protocol)

8. BGP : Risks • Where are the risks ? • Internet Exchanges (“peering points”) • All providers are usually connected to the same shared infrastructure (a switch for example) • The filtering policy is usually more “relax” for peerings • Some major ones, no real (geo)diversity anymore • Your direct {up,down}stream(s) • Route reflectors • Multi-hop configurations (Man-in-the-middle attack) • Less likely : some backbone router “out there” in the Internet or some hops away • What is never “verified” • Origin-AS/prefix relation, “true” AS_path, source authenticity, etc

9. BGP : Attacks (1) • Information gathering • Find the eBGP peers : • “Forward” and “reverse” traceroute / ICMP Record Route • Public route-servers and looking glasses • Directly adjacent IPs • IPs often used for loopback interfaces (.1+, .254-) • SNMP • Session parameters may be required : • Source/destination ports (ie. which router initiated the connection) • Right TTL

10. BGP : Attacks (2) • Attacks against routers and BGP sessions • SYNflood 179/tcp • Drop the BGP session by RSTing the TCP connection or injecting bogus OPEN/KEEPALIVE/etc messages • Spoofed packet parameters (IPs, ports, SeqNum, TTL) may have to fit • BGP route injection tool : (what is the) challenge ? • Inject the UPDATE • MiTM (or ARP spoofing on IX switches) • Synchronize with/hijack the TCP session (MiTM or spoofed from remote) • Have a previous knowledge of the current configurations of the peers (a MiTM type of attack is more likely to happen) • BGP route injection tools exist (in private circles) • Security bug in the BGP implementation / modified BGPd

11. BGP : Attacks (3) • Attacks against the network • Attacks playing with BGP parameters (local-pref, MEDs, communities) ? • Make your BGP sessions flap : make you or other destinations unreachable • Announce “more specific routes” of large blocks to increase the number of prefixes in the global routing table and eat up memory on all routers • Announce or “remove” some routes/prefixes or change their attributes • Direct all the traffic to a blackhole, direct it to a specific network (DDoS), create loops, etc.

12. BGP : Sequence number prediction • ISN problems on Cisco routers Vulnerable IOS “Less” vulnerable IOS • “Fixed” as of 12.0(15) and 12.1(7) • ISNs are (still) time dependant Source : http://razor.bindview.com/publish/papers/tcpseq.html

13. BGP : Monitoring & protection • What to monitor ? • AS_path and prefixes you receive from upstreams • AS_path and prefixes that other ISPs are getting that contains your ASN/prefixes (route servers/looking glasses) • Are the paths changing (especially the best path) ? • Are some prefixes announced (temporary) in another AS ? • ARP changes (IX public switches) • Your logs for BGP related messages • What is currently missing ? • Origin-AS/prefix mapping : automatically generated filters out of RIRs (Regional Internet Registries) like RIPE NCC’s DB (RPSL) are part of the solution but don’t cover everything : route object that lists the AS in which the network prefix is routed

14. BGP : Filtering policy • Filtering recommendations • Don’t accept to have only /24 • Never accept, announce or redistribute prefixes that are longer than /24 or de-aggregated blocks • Use maximum-prefix (ie “full routing table” is currently ~110K-114K routes) • Only accept the customer’s allocated prefixes • Don’t filter on AS_path only, but also prefixes (customer may announce a more specific route): this the same as in/egress filtering of IP addresses ! • Filter bogon and non-allocated networks • Should you really accept/announce a default route in BGP ? • Have ingress and egress filters to limit the prefixes you receive/send

15. BGP : Ingress/egress filtering (1) • What you should never route/see/allow through • RFC 1918 (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16) • 0.0.0.0/x, 127.0.0.0/8 • 169.254.0.0/16 (auto-configuration when no DHCP) • 192.0.2.0/24 (Netname: TEST-NET, like example.com) • 192.88.99.0/24 (RFC 3048, used by 6to4 routers) • Multicast blocks (D Class) and Martian networks (E+) • “Hijacked” space by some vendors (192.0.0.192 for printers) • (ARIN) Reserved blocks (bogon networks) • Packets to broadcast addresses or where source == destination • What you should route/let through • Your network prefixes (anti-spoofing)

16. BGP : Ingress/egress filtering (2) • Example with ACLs • Filter on network border : CPE/IX/uplinks • Example with route to Null0 (“discard” on Juniper) interface xy access-group in 100 access-group out 100 access-list 100 deny ip host 0.0.0.0 any access-list 100 deny ip 127.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255 access-list 100 deny ip 10.0.0.0 0.255.255.255 255.0.0.0 0.255.255.255 access-list 100 deny ip 172.16.0.0 0.15.255.255 255.240.0.0 0.15.255.255 access-list 100 deny ip 192.168.0.0 0.0.255.255 255.255.0.0 0.0.255.255 access-list 100 deny ip 192.0.2.0 0.0.0.255 255.255.255.0 0.0.0.255 access-list 100 deny ip 192.88.99.0 0.0.0.255 255.255.255.0 0.0.0.255 access-list 100 deny ip 169.254.0.0 0.0.255.255 255.255.0.0 0.0.255.255 access-list 100 deny ip 240.0.0.0 15.255.255.255 any access-list 100 permit ip any any ! Or permit ip <your network prefixes only> ip route 10.0.0.0 255.0.0.0 null0 ip route 172.16.0.0 255.240.0.0 null0 ip route 192.168.0.0 255.255.0.0 null0

17. BGP : Security recommendations (1) • Security measures • Log changes (and think about using IPsec -- CPU and $) • Use MD5, but not the same password with all the peers • Filter and limit the number of prefixes, filter the AS_path • Use a “secondary” or “loopback” IP address for the eBGP session to hide parf of the eBGP session details • Protect routes towards Root Servers (<x>.{root, gtld}-servers.net) : exclude them from the route dampening • Filtering and routing policies are important, but don’t forget: • to harden your router configuration • to use secure in and out-of-band management and monitoring tools/protocols • that physical access to the router gives “full rights” (peering points/data centers)

18. BGP : Security recommendations (2) • Security measures router bgp 65000 no bgp dampening bgp dampening route-map dampening-list bgp log-neighbor-changes network x.x.x.x neighbor y.y.y.y remote-as 65001 neighbor y.y.y.y password <MD5password> neighbor y.y.y.y version 4 neighbor y.y.y.y prefix-list theirnetworks in neighbor y.y.y.y prefix-list ournetworks out neighbor y.y.y.y maximum-prefix 120000 neighbor y.y.y.y route-map ourASpath out ip prefix-list ournetworks seq 5 permit x.x.x.x/y ip prefix-list ournetworks seq 10 deny 0.0.0.0/0 le 32 ip prefix-list theirnetworks seq 5 permit x.x.x.x/y ip prefix-list protected-prefixes permit x.x.x.x/y ip prefix-list <other prefix-list> permit x.x.x.x/y ip as-path access-list 99 permit ^<AS>( <AS>)*$ route-map ourASpath permit 10 match as-path 99 route-map dampening-list deny 10 match ip address prefix-list protected-prefixes route-map dampening-list permit 20 match ip address prefix-list <other prefix-list> set dampening <your dampening parameters>

19. BGP : Future (1) • S-BGP • Covers part of the integrity/AA(A) issues and is based on : • a PKI • to verify ownership (IPs/AS) and Origin-AS/prefix relation • to validate signed UPDATE (with Route Attestations) messages, their content and BGP speakers • to manage certificates and CRLs (Certificate Revocation List), etc • IPsec for the router-to-router session • Implementation and deployment imply that : • RIRs act as a CA (Certificate Authority) and tree root • Servers storing certificates, CRLs, and other information have to be distributed and “protected” from DoS (non-routed ?) • S-BGP supported on large routers with enough memory (volatile and non-volatile) and CPU • ISPs are willing/forced to do correct “paperwork”

20. BGP : Future (2) • Working Groups • IETF • rpsec (Routing Protocol Security Requirements) • ptomaine (Prefix Taxonomy Ongoing Measurement & Inter Network Experiment) • msec (Multicast Security) • RIPE : Routing Working Group • IRTF : Group Security (GSEC, formerly Secure Multicast Group) • Don’t forger router “host” security ! • Forensics (who used to announce which prefix) • Route information stored inside DBs at large peering points or by large providers

21. OSPF : Protocol description • OSPF (Open Shortest Path First) • Protocol type 89 • Multicast traffic : “easy” to inject LSAs (Link State Advertisement) • Active adjacencies between all the routers and the (B)DRs (DR/BDR status is based on Router ID and priority) • SPF (Shortest Path First) recalculation takes time and CPU

22. OSPF : Attacks • Attacks against OSPF • Since the (B)DRs don’t preempt, it’s needed to “kill” the legitimate ones to take the functionality over • A “local” area LSA may be exported to another area (over an ABR) or even to another AS (over an ASBR) • ABRs and ASBRs are key routers together with (B)DRs • OSPF LSAs (even “MD5ed”) can be replayed (sequence number) • inject/withdraw routes • break adjacencies (higher sequence number/hello message) • MAXAGE LSAs can be send to purge the related OSPF DB • More or less impossible to protect the network from an internal attack/threat : routers can “lie” (about their role or modify the information they announce)

23. OSPF : Security (1) • Security measures • Authenticate OSPF exchanges (per interface) with MD5 (not Null nor cleartext). MAC is (OSPF payload+secret) • Turn your network into a NBMA (Non Broadcast Multiple Access - “point-to-point links only”) network • (Positive) side effect of link-state protocols : routers may counter the faked LSAs interface xy !ip ospf authentication-key <key> ip ospf message-digest-key 1 md5 <key> router ospf 1 area 0 authentication [message-digest] interface xy ip ospf network non-broadcast router ospf 1 neighbor x.x.x.x

24. OSPF : Security (2) • Security measures • Don’t put the interfaces that shouldn’t send or receive OSPF LSAs in your network statement or then exclude them with a passive-interface statement • Log changes • You can’t filter what is injected into the local area (the network statement meaning is misleading) only to other Ases • the route information is in the OSPF Database but is not pushed into the RIB (Routing Information Base) router ospf 1 log-adjacency-changes network x.x.x.x passive-interface default no passive-interface xy router ospf 1 distribute-list <ACL> in distribute-list <ACL> out

25. ISIS : Protocol description (1) • IS-IS (Intermediate System to Intermediate System) • Comes from the OSI world (routed OSI procotols) • Doesn’t run on top of IP but directly over the data link • Encodes the packets in TLV (Type-Length-Value) format • Uses hierarchy levels/addressing (L1/L2) and flooding • L1 routing means routing in the same area • L2 routing means between areas

26. ISIS : Protocol description (2) • IS-IS (Intermediate System to Intermediate System) • Floods LSPs (Link State PDUs) • Nothing do to with MPLS’ LSP (Label Switch Path) • Contrary to OSPF DR/BDRs a new IS-IS DIS (Designated IS) with higher priority will take precedence (preempt) and all the routers maintain adjacencies with all the routers in the area (separate L1 and L2 adjacencies on same LAN) • A lot of Service Providers are moving from OSPF to ISIS (usually in relation with MPLS/Traffic Engineering deployment)

27. ISIS : Attacks and security • Attacks • Similar to OSPF attacks but more complex to inject data because of non-IP protocol • Possible to use the “Overload Bit” to have transit traffic not sent over a “overloaded” router and thus try to redirect it • Security measures • Log changes • Use authentication at • the interface level • the area level • the domain level • HMAC-MD5 support (TLV 54) interface xy isis password <password> level-<z> router isis log-adjacency-changes domain-password <password> area-password <password>

28. DDoS detection (1) • Netflow • Accounting data (AS, IP flows, protocols, etc) • Send in clear text over the network (UDP) to a gatherer • Only counts outgoing traffic on the interface • Needs CEF (Cisco Express Forwarding) or dCEF • Requires IOS 12.x and uses ~30MB of memory • With CEF activated Netflow will only do accounting • Without CEF the router will do netflow switching • How to export the data • How to view the data : sh ip cache flow ip flow-export version 5 origin-as ip flow-export destination x.x.x.x interface xy ip route-cache flow

29. DDoS detection (2) • (Un)usual traffic distribution per protocol • TCP : ~90 % (HTTP, FTP, SMTP and P2P tools) • UDP : ~10 % (DNS, SNMP, streaming) • ICMP : <1 % • IGMP : <1 % • Mostly <x> bytes packets • RRDtool and Netflow can be used to graph trends, detect changes and anomalies Source : Flowscan from UW-Madison (http://wwwstats.net.wisc.edu/)

30. DDoS detection (3) • Netflow data on Multi-Layer Switches • Netflow-based MLS flow-mode is “destination-only” no source address is cached) • Enable “full-flow” mode (performance impact on SE1) • Display the entries • Poor man’s netflow : ntop ? ! MLS in hybrid mode set mls flow full ! MLS in native mode mls flow ip full ! MLS in hybrid mode show mls entry ! MLS in native mode show mls ip

31. DDoS prevention (1) • Unicast RPF (Reverse-Path Forwarding) • Mode • Strict : IP packets are checked to ensure that the route back to the source uses the same interface • Loose check: allowed if the prefix exists in the FIB • Only the best path (if no multi-path or equal cost paths) is in the FIB • Asymmetric routes are supported (really :-) • Check the BGP weight if you use strictmode in a multi-homed configuration

32. DDoS prevention (2) • Unicast RPF (Reverse-Path Forwarding) • Strict (you can use an ACL for exceptions or for logs) • “Loose check” ip cef [distributed] interface xy ip verify unicast reverse-path [allow-self-ping] [acl] ip verify unicast source reachable-via any

33. DDoS prevention (3) • ICMP, UDP, TCP SYN rate-limiting • UDP rate-limiting can be a problem if your customer is a streaming company interface xy rate-limit input access-group 100 8000 8000 8000 \ conform-action transmit exceed-action drop rate-limit output access-group 100 8000 8000 8000 \ conform-action transmit exceed-action drop <…> access-list 100 deny tcp any host x.x.x.x established access-list 100 permit tcp any host x.x.x.x access-list 101 permit icmp any any echo access-list 101 permit icmp any any echo-reply

34. DDoS prevention (4) • TCP Intercept • Can do as much good as bad • If enabled : process switching and not “full” CEF anymore • The “destination” host must send a RST (no silent drops) or you’ll DoS yourself • Same is true if you use “blackholed” routes (route to Null0) ip tcp intercept list 100 ip tcp intercept connection-timeout 60ip tcp intercept watch-timeout 10ip tcp intercept one-minute low 1500ip tcp intercept one-minute high 6000 access-list 100 permit tcp any x.x.x.0 0.0.0.255

35. DDoS prevention (5) • Advanced ICMP filtering • Only let the “mission critical” ICMP messages in and out • ICMP filtering is a source of dispute (unreachables, parameter-problem, etc) • ICMP is not just “ping”, you can break a lot of things (Path MTU Discovery for example) • YMMV. interface xy ip access-group 100 in access-list 100 deny icmp any any fragments access-list 100 permit icmp any any echoaccess-list 100 permit icmp any any echo-replyaccess-list 100 permit icmp any any packet-too-bigaccess-list 100 permit icmp any any source-quenchaccess-list 100 permit icmp any any time-exceededaccess-list 100 deny icmp any anyaccess-list 100 permit ip any any

36. DDoS prevention (6) • Advanced technique 1 (1/2) : BGP/Null0 • Pick an IP address from TEST-NET and add a static route to Null0 for it (on all your routers) • Have a “master” BGP router set the next-hop for the source network you want to “drop” to the selected IP • Have BGP redistribute it to the routers in your AS only and uRPF will drop it (at the LC level, not on the RP) • Do not redistribute it to your peers : use a private AS or a “no-export” community router bgp <AS> network <sourceOfDDOS> mask <netmask> route-map ddos-nh route-map ddos-nh set ip next-hop <TEST-NETIPaddr> ip route <TEST-NET> 255.255.255.0 Null0

37. DDoS prevention (7) • Advanced technique 1 (2/2) : BGP/Null0

38. DDoS prevention (8) • Advanced technique 2 (1/2) : BGP/CAR/FIB • Set a special community for the network you want to rate-limit on your “master” BGP router and send this community to your iBGP peers router bgp <AS> network <destOfDDOS> mask <netmask> neighbor x.x.x.x route-map ddos-rl out neighbor x.x.x.x send community access-list 10 permit <destOfDDOS> route-map ddos-rl match ip address 10 set community <AS>:66 no-export ip route <destOfDDOS> 255.255.255.0 Null0

39. DDoS prevention (9) • Advanced technique 2 (2/2) : BGP/CAR/FIB • On the routers change the QoSID entry in the FIB based on this special community • Use the QoSID entry of the FIB to rate-limit router bgp <AS> table-map ddos-rl ip community list 1 permit <AS>:66 route-map ddos-rl match community 1 set ip qos-group 66 interface xy bgp-policy source ip-qos-map rate-limit input qos-group 66 ...

40. Worm detection and protection (1) • How to detect a new worm • New/unusual number of HTTP/SMTP flows and server logs • How to protect with NBAR (Network-Based Application Recognition) • Needs CEF • Available as of 12.1(5)T • Like TCP Intercept - do we need it ? • Side-effect : the TCP handshake is already done but the server never receives the HTTP GET request • Performance impact : ~20% CPU

41. Worm detection and protection (2) • NBAR Restrictions and limitations • Supports up to 24 concurrent URLs, hosts or MIME types matches • Can’t match beyond the first 400 bytes in a URL • Can’t deal with fragmented packets • HTTPS traffic (that’s normal ;-) • Packets originating from/sent to the router (you can’t protect the local HTTP server) • Doesn’t support Unicode (UTF-8/%u) • Tune the scheduler and the timeout ip nbar resources 600 1000 50 scheduler allocate 30000 2000

42. DDoS/worm research/future (1) • ICMP Traceback (itrace) • Each router sends, with a low probability, a message to the destination of a packet it forwarded with information about the previous and next hop and part of the payload • Only “works” for larger (D)DoS attacks • IP Traceback • Traceback information is stored in the ip.id field by the “IPT” routers on the packet’s path • Probability to catch smaller attacks is better than with itrace • MULTOPS (Multi-Level Tree for Online Packet Statistics) • A local data structure on each router stores data about current flows and is analyzed to detect/respond to attacks

43. DDoS/worm research/future (2) • SPIE (Source Path Isolation Engine) • The router stores temporary a hash about each packet it forwards and authorized routers can query this information • Pushback • Router send rate-limit requests for certain networks if they detect attacks (based on traffic characteristics) • IDIP (Intruder Detection and Isolation Protocol) • Protocols and framework used to correlate information, detect and respond to intrusions • HIP (Host Identity Payload/Protocol) • New name space (next to IP/DNS) with IKE like functionalities and public key based authentication for hosts

44. DDoS/worm research/future (3) • CenterTrack • Secondary network used to carry “interesting” packets detected by routers for analysis • Limitations • CPU and memory needs on routers • Fundamental changes (infrastructure, deployment, operations, etc) • IP address spoofing and traceback are the key issues

45. DDoS/worm research/future (4) • Worse to come • A lot of research has been done but nothing has been published/disclosed : “risks are too high” • Most of the worms we’ve seen were quite gentle • Will the next worm affect IIS/Outlook users again ? • What are the effects on the Internet stability ? • What are the trends ? • Routers are used as source • Attacks are more complex and agents are becoming more intelligent • Temporary “use” of non allocated blocks

46. MPLS (1) • MultiProtocol Label Switching • MPLS label added to the IP packet to identify the VPN • Each router (LSR) on the path (LSP) has a local table (LIB) • The label only has a “local” meaning and is/may be changed on each hop

47. MPLS (2) • MultiProtocol Label Switching • Virtual Circuits, not encrypted/authenticated VPNs • “Equivalent” to a layer 2 VPN (ATM/FR) • the security is often provided by hiding the MPLS core structure/cloud from customers by using filtering or non-routed address space (think security by obscurity) • as a customer you have to trust the MPLS core • IPsec can be used to secure the traffic • VPN partitioning done at routing layer • “One routing table per VPN” on each PE router • separate Virtual Routing and Forwarding instance (VRF) • or extended Route Distinguisher (RD) • Current trend in SP networks : deploy MPLS+ISIS w/ Wide Metrics+TE for subsecond convergence and traffic rerouting

48. MPLS (3) • Attacks • Labeled packets injection : • locked by default on all interfaces (Customer Edge Router) • easy if access to the MPLS routers • Inject data in the signaling protocols ((MP-)BGP and IGPs) to modify the VPN topology : IPv4-RRs and VPNv4-RRs (Route Reflectors) • Even a higher risk when the same router is shared for Internet access and a MPLS L2VPN

49. MPLS (4) • Attacks • Use new functionality like FRR (MPLS Fast ReRoute) • RSVP (No Route) Path Error message : allows sniffing by redirecting traffic over a router that is under control and part of the MPLS core • a new LSP is signaled • the adjacency table is updated for the tunnel interface • a LSR receiving a marked packet with label x will accept it on any interface and switch it out

50. MPLS (5) • Security measures • Good configuration of all routers (CE, PE, P, MPLS Core) • ACLs • Static and dynamic routing • VRFs • etc. • The “MPLS network” should start on the PE router, not the CE • Difficult to gather MPLS information from the routers • Use IPsec (without anonymous key exchanges ;-)