Optimal external route selection tips and techniques for isps
1 / 55

Optimal External Route Selection: Tips and Techniques for ISPs - PowerPoint PPT Presentation

  • Uploaded on

Optimal External Route Selection: Tips and Techniques for ISPs . Avi Freedman Net Access. Overview. Brief review of BGP routing concepts Safe routing Determining policy Using MEDs Setting MEDs on internal routes as-path padding to tune external traffic

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Optimal External Route Selection: Tips and Techniques for ISPs' - lotus

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Overview ISPs

  • Brief review of BGP routing concepts

  • Safe routing

  • Determining policy

  • Using MEDs

  • Setting MEDs on internal routes

  • as-path padding to tune external traffic

  • Using local-prefs to tune external traffic

  • Setting MEDs to tune external traffic

Bgp intro
BGP Intro ISPs

  • BGP4 is the protocol used on the Internet to exchange routing information between providers, and to propagate external routing information through networks.

  • Each autonomous network is called an Autonomous System.

  • ASs which inject routing information on their own behalf have ASNs.

Bgp peering
BGP Peering ISPs

  • BGP-speaking routers peer with each other over TCP sessions, and exchange routes through the peering sessions.

  • Providers typically try to peer at multiple places. Either by peering with the same AS multiple times, or because some ASs are multi-homed, a typical network will have many candidate paths to a given prefix.

The bgp route
The BGP Route ISPs

  • The BGP route is, conceptually, a “promise” to carry data to a section of IP space. The route is a “bag” of attributes.

  • The section of IP space is called the “prefix” attribute of the route.

  • As a BGP route travels from AS to AS, the ASN of each AS is stamped on it when it leaves that AS. Called the AS_PATH attribute, or “as-path” in Cisco-speak.

Bgp route attributes
BGP Route Attributes ISPs

  • In addition to the prefix, the as-path, and the next-hop, the BGP route has other attributes, affectionately known as “knobs and twiddles” -

    • weight, rarely used - “sledgehammer”

    • local-pref, sometimes used - “hammer”

    • origin code, rarely used

    • MED (“metric”) - a gentle nudge

Bgp policy
BGP Policy ISPs

  • BGP was designed to allow ASs to express a routing policy. This is done by filtering certain routes, based on prefix, as-path, or other attributes - or by adjusting some of the attributes to influence the best-route selection process.

Bgp best route selection
BGP Best-Route Selection ISPs

  • With all of the paths that a router may accumulate to a given prefix, how does the BGP router choose which is the “best” path?

  • Through an RFC-specified (mostly) route selection algorithm.

Bgp best route selection1
BGP Best-Route Selection ISPs

  • Do not consider IBGP path if not synchronized

  • Do not consider path if no route to next hop

  • Highest weight (local to router)

  • Highest local preference (global within AS)

  • Shortest AS path

  • Lowest origin code IGP < EGP < incomplete

  • Lowest MED

  • Prefer EBGP path over IBGP path

  • Path with shortest next-hop metric wins

  • Lowest router-id

Bgp selection summary
BGP Selection, Summary ISPs

  • So, local-pref is stronger than as-path is stronger than MED.

  • Setting local-pref without careful planning can cause strange things (preferring other paths to get to your own customers)…

Safe routing1
Safe Routing ISPs

  • BGP routes are “promises” to carry traffic to a certain destination. Still, not every provider makes good promises {at all times}.

  • So, it is best to sanity-filter all eBGP sessions.

Safe routing2
Safe Routing ISPs

  • Method 1:

    • The Cisco “maximum-prefix” keyword

      • neighbor <remote-ip> maximum-prefix [percent] [warning]

    • Sets a maximum number of prefixes allowed for a peer.

    • Behavior 1 - Shut down the session and log the fact.

    • Behavior 2 - Leave the session up; just log the warning.

Safe routing filtering
Safe Routing - Filtering ISPs

  • Another method of sanity filtering is to restrict your peers based on routes or as-paths.

  • Usually, it is hard to filter based on routes (except for our friends, the fanatics at ANS).

  • So, from smaller providers it is a good idea to prevent random route redistribution.

Safe routing filtering1
Safe Routing - Filtering ISPs

ip as-path access-list 40 deny _701_

ip as-path access-list 40 deny _1239_

ip as-path access-list 40 deny _3561_

ip as-path access-list 40 deny _1_

ip as-path access-list 40 deny _1673_

ip as-path access-list 40 deny _174_

ip as-path access-list 40 permit .*

  • Apply this access-list inbound for sanity.

I am blackholio
“I am Blackholio” ISPs

  • In sufficiently strange circumstances, this won’t help.

  • If someone (AS 7007, perhaps) strips the as-path information, as-path filters do no good.

Determining policy1
Determining Policy ISPs

  • What do you want to do?

  • The tricky part.

  • Configuring is easy…

  • Do you want to prefer higher-quality connections?

  • Optimize for cost of the links?

Connection quality
Connection Quality ISPs

  • We will assume that you want to optimize for connection quality.

  • This generally means, in the Platonic zero-packet-loss Internet, minimizing latency and avoiding small pipes.

  • We’ll come back to small pipes and backup paths when we talk about local-prefs.

  • We’ll talk about minimizing latency when we explore MEDs.

Connection quality1
Connection Quality ISPs

  • At all times, we must minimize packet loss.

  • In general, this means avoiding public exchanges in favor of private peering and/or transit.

  • Sometimes this might not be economically desirable, but if you don’t tune this way, stay vigilant about inter-connection quality.

  • Best to measure it if you really care...

Measuring Packet Loss with MRTG ISPs

Max Max: 423.0 ms (352.5%) Average Max: 32.0 ms (26.7%) Current Max: 37.0 ms (30.8%)

Max Min: 9.0 ms (7.5%) Average Min: 5.0 ms (4.2%) Current Min: 6.0 ms (5.0%)

Peering points
Peering Points ISPs

  • You want to prefer paths that you hear over uncongested pipes.

  • Assuming you have non-full private interconnects, PIs will be better than public exchanges.

  • Of course, that can depend on which Gigaswitch you’re on; whether you’re at PSK, PACBell, AADS, or the MAEs.

Hot potato
Hot-Potato ISPs

  • In general, traffic is handed off as soon as possible to external providers to minimize backbone utilization and costs.

  • This is not always the best plan if you want to maximize connection quality (assuming your inter-LATA and/or cross-country links are not full).

  • Solution - Listen to and use MEDs.

Asymmetry ISPs

  • For this presentation, we are going to ignore the return path - data coming back into your network.

  • Still, for best tuning you will want to explore this and use as-path padding and possibly controlled de-aggregation (to willing partners)...

Review policy
Review: Policy ISPs

  • Somehow, you want to prefer better-quality links.

  • In the examples that follow, we’ll assume a small but national network, peering at MAE-West, MAE-East, and Pennsauken.

  • Additionally, private interconnects with IDT, PSI, Digex, above.net, and Exodus.

  • Transit through above.net and UUNET.

Goals ISPs

  • Our goals will be to prefer, in this order:

    • Private interconnects

    • Regionality of traffic

    • Pennsauken over MAE-East

    • Public Exchanges

    • Transit pipes, above.net first

Introduction to meds
Introduction to MEDs ISPs

  • The MULTI_EXIT_DISCRIMINATOR, or MED, is a BGP attribute used to:

    • Describe internal network topology.

    • Pass on this topology to external peers.

  • A smaller knob than others, like local-pref or as-path padding.

  • Major problem - no inter-provide consistency on MED semantics.

  • Internally, also called “metrics”.

Setting meds for internal route

Setting MEDs ISPs for Internal Route

Setting meds
Setting MEDs ISPs

  • Use an internally consistent scheme.

  • Usually, people’s MEDs are in the low hundreds or less.

  • Suggestion - use average delay in ms between POPs.

  • Set MEDs in one direction only.

  • To be advanced, MEDs can be set on a per-router basis in a POP, but usually are not.

Network diagram
Network Diagram ISPs





Setting meds1
Setting MEDs ISPs

  • For SF, CHI, PHL, DC:

    SF-DC +60

    SF-CHI +40

    CHI-PHL +30

    CHI-DC +25

    PHL-DC +10

    PHL-PSK +0

    DC-MAE-E +5

    SF-MAE-W +5

Network diagram w meds
Network Diagram w/ MEDs ISPs










Route maps in dc
Route Maps in DC ISPs

route-map from-sf

set metric +60

route-map from-chi

set metric +40

route-map from-phl

set metric +10

neighbor <sf-ip> route-map from-sf in


What this does
What this Does ISPs

  • A route originating in PHL will have:

    • metric 60 or or 70 in SF (unless there are multiple link failures)

    • metric 10 or 60 in SF

    • metric 10 or 35 in DC

  • etc…

  • Thus, a provider honoring MEDs (not doing hot-potato) will send packets destined to that route in PSK, to PHL.

Slight improvement
Slight Improvement? ISPs

  • Or, change things to weight PSK vs. DC over PHL vs. DC.

    PSK +0

    MAE-E +20

  • Thus, a provider honoring MEDs will send a PHL-destined packet to PSK. This is generally a good thing.

As path padding
as-path padding ISPs

  • Some think that modifying as-paths is a nasty business.

  • It is a good beginning way to do preferences.

  • If providers have already padded to de-prefer, preserves that “de-preference”.

  • Simple to do.

As path padding1
as-path padding ISPs

  • First, policy?

    • Private interconnects - pad no times

    • Regionality of traffic - pad four times x-country

    • Pennsauken over MAE-East - pad once; twice

    • Public Exchanges - twice at MAE-West

    • Transit pipes, above.net first - pad three

  • Problem - can’t pad easily going cross-country.

  • But we can do the rest.

    • Problem - lots of route-maps and typing.

  • Why? Can’t prepend our own AS inside network, so must have separate roue-map per session.

Route maps
route-maps ISPs

  • On everyone, at above.net:

    route-map prepend-once permit 10

    set as pre 6461 6461 6461

  • On everyone, at UUNET:

    route-map prepend-once permit 10

    set as pre 701 701 701

  • On PSI, at MAE-East and MAE-West:

    route-map prepend-once permit 10

    set as pre 174 174

  • On PSI, at Pennsauken:

    route-map prepend-once permit 10

    set as pre 174

Local prefs
Local-prefs ISPs

  • Most common method of preferring external routes.

  • Local-pref is a number, by default 100, put on routes and passed to all routers within a network.

  • Never passed to an eBGP peer.

Implementing policy
Implementing Policy ISPs

  • Customers - local-pref 200

  • Private interconnects - local-pref 150

  • Pennsauken over MAE-East - 120 for Pennsauken

  • Public Exchanges - 100 at MAE-East and MAE-West

  • Transit pipes, above.net first - 80 from transit pipes

  • Regionality of traffic - defer to MEDS for equal local-pref. May want to add PACBELL cxn and make it 120.

Route maps1
route-maps ISPs

  • At Pennsauken:

    route-map psk in

    set local-pref 120

    set community 4969:800

    neighbor peer-group external-peer-psk route-map psk in


    neighbor <remoteip> route-map psk in

Problem prefers bad paths
Problem: Prefers Bad Paths ISPs

  • The problem with this approach:

  • Take AS 14000, who has a T1 to Sprintlink and a backup-backup-backup 56k to another local provider, say, 13000.

  • Announces as:

    • 1239 14000 and

    • 701 13000 14000 14000 14000 14000 1400

  • Local-prefs can screw with this.

Listening to meds same peer
Listening to MEDs: Same Peer ISPs

  • Nothing special is required to listen to MEDs.

  • Because MEDs mean different things to different networks, one approach is no only set MEDs inbound for your own routes.

  • When listening to MEDs at multiple locations from a peer, set to internal MEDs if you want to hot-potato.

Route map on dc v2
route-map on DC, v2 ISPs

route-map from-sf permit 10

match community 1

set metric +60

Meds from diff ebgp peers
MEDs from Diff. eBGP Peers ISPs

  • “bgp always-compare-med” keyword allows Ciscos to use MEDs among different providers.

  • Otherwise, will use them to compare iBGP routes, or eBGP routes from the same AS.

Preferring external routes w meds
Preferring External Routes w/ MEDs ISPs

  • Can be done, sometimes while preserving remote MED info, but usually remote MED info is lost.

  • Better in some cases than as-path padding or local-prefs (as-path padding is undesirable when you have to pass routes on to customers; local-prefs might use backup links…).

Preferring external routes w meds1
Preferring External Routes w/ MEDs ISPs

  • Assuming not honoring remote MEDs:

    • Set metric inbound to 0 and set internal-route MEDs on routes, then:

    • Private interconnects - no change

    • Regionality of traffic - no change - add normal MEDs

    • Pennsauken over MAE-East - add 20 for MAE-East

    • Public Exchanges - no change, or add 20

    • Transit pipes, above.net first - add 30 or 40

Overriding bgp
Overriding BGP ISPs

  • Some have started to override BGP when evidence suggests better routing, on a per-prefix basis.

  • ASAP from above.net, ?fastpath?, ?others?

  • Ideally actively and autonomously, determine best path to frequently-used prefixes and inject fixer-routes.

  • Soon, Cisco will have hooks for injection.