Design and implementation of the reannz karen network j rg micheel
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

Design and Implementation of the REANNZ KAREN network Jörg Micheel PowerPoint PPT Presentation


  • 130 Views
  • Uploaded on
  • Presentation posted in: General

Design and Implementation of the REANNZ KAREN network Jörg Micheel. Outline for this talk. Network design goals for KAREN Layer 1+2+3 network architecture Network services and implementation International transit network design Network performance

Download Presentation

Design and Implementation of the REANNZ KAREN network Jörg Micheel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Design and Implementation ofthe REANNZ KAREN networkJörg Micheel


Outline for this talk

  • Network design goals for KAREN

  • Layer 1+2+3 network architecture

  • Network services and implementation

  • International transit network design

  • Network performance

  • Checklist for KAREN connectors (REANNZ members)

  • Summary and references


MoRST/REANNZ/KAREN design goals

  • A high performance network for the NZ R&E community!

  • 10 Gbps capable backbone interconnecting all major cities in NZ

  • Access speeds at 1 Gbps and 10 Gbps

  • Unconstrained end-to-end performance at (multi-)Gigabits/second

  • Tailored on-demand performance for specific applications or experiments (bandwidth, delay, jitter)

  • International connectivity at 155 Mbps to AU, 622 Mbps to US

  • New services: multicast, IPv6, Jumbo frames (9000 Bytes MTU)

  • Virtual Private Network functionality for members

  • Telco-grade implementation and network management

  • Security, redundancy, high availability

  • Range of network measurement facilities (wire tap, NetFlow, SNMP data collection, active measurement) and development environment

  • Most importantly: stick to budget and timelines!!!


L1/L2/L3 Network Design

  • L1 Network core as rings on TCL OPTera DWDM

  • L1 Dark fiber spur to neutral POP and AAP

  • L2 Nationwide network based on Extreme X450a and BD10K

  • L3 using Juniper M320 in Auckland and Wellington

  • Note: 10GigE WANPHY is 9.287 Gbps!

  • ANOPS management network based on TCL PIP service and CISCO 2801


Extreme Networks Black Diamond – metro core switch

  • Black Diamond 10808 (BD10K)

  • 22 rack mount units

  • 1280 Gbps capacity (blocking)

  • Up to 48 10-Gigabit ports

  • Up to 480 10/100/1000 ports

  • Powerful VLAN, Virtual router Layer2 and Layer3 capabilities

  • Proprietary EAPS link-protection protocol provides continuity in case of fiber cut

  • L2/L3 Quality-of-Service

  • L2/L3 hardware filtering and priority

  • Jumbo frames at 9212

  • High availability, hardware redundancy


Extreme Networks Summit X450a – edge switch

  • X450a-24t with 24 ports 10/100/1000 copper, four combined SFP GigE ports

  • X450a-24x with 24 ports 1-GigE SFP, four combined 10/100/1000 ports

  • Optional dual 10-Gigabit Ethernet uplinks

  • 1 RU form factor

  • 160 Gigabits-per-second capacity

  • 65 million packets-per-second forwarding performance

  • Stacking capability with XOS 11.7 (April 2007)

  • Other features similar to Black Diamond series


Juniper Networks M320 Multi-service Edge Router

  • 320 Gbps switching capacity

  • 8 FPC slots with 20 Gbps FD capacity

  • ½ rack size

  • 32 PICs per chassis

  • 10 GigE capable


L2/L3 design

  • Connector joins KAREN via dark fiber

  • Switch access into one or more VLANs

  • BGP peering with core

  • L2 packet switched data nationally

  • L3 routing overseas


KAREN Service Matrix


Internet Exchange model (L2 switching + BGP Route Reflector)

“Switch – don’t route”

“Peer with two – route with many (others)”

 scalable Internet Exchange model


Challenges of a L2 network

  • Redundant links will be pruned (Spanning Tree, etc), creating a star topology

  • Only difference between L2 resilience protocols is speed

  • Issue: capacity not utilised

  • Issue: shortest path

  • Issue: protect all VLANs

  • Solution: VLANs EAST/WEST for public IP services

  • Solution: Extreme EAPS for protection


KAREN Multicast – two options for connectors

  • Bootstrap as part of KAREN multicast cloud (quick start for small sites, no MSDP, but doesn’t scale)

  • Create your own multicast domain (requires MSDP, scales well)


Multi-Protocol BGP and routing tables


KAREN International

  • Separate to domestic KAREN, but co-joined

  • As a static 3-point transit network has to implement all services (IPv4/IPv6 uni- and multicast, jumbo frames)

  • Routing policy ensures traffic flows between NZ and overseas peers (but not between other peers)

  • Pacific Wave landing point in Seattle poised for peering expansion

  • Bulk of 9K routes from Internet2 ITN


REANNZ POP


KAREN performance tests

  • Network commissioning in October and November successfully demonstrated capacity, delay and jitter parameters

  • Bandwidth tests carried out as 1 Gbps VLANs POP-to-POP

  • All L2 components stressed at or near capacity limits (see next slide for example)

  • Delay and jitter tests carried out as RTT measurements using hardware loopbacks


KAREN performance tests (as shown on WAND weathermap)

See recorded animations at http://erg.cs.waikato.ac.nz/weathermap/ for other tests carried out during November and December.


KAREN delay and jitter tests


KAREN delay and jitter results


KAREN Connectors 101 (and shopping list)

  • KAREN is a Tier 1 network – you need to behave like a Tier 2 – control your own routing (policy).

  • Consider the services you want: IPv4 unicast, multicast, IPv6, Jumbo

  • Are you a heavy hitter ? Thinking of 10 Gbps ?

     Router that speaks BGP, holds 20K+ routes and does 1 Gbps

  • If you are a heavy hitter, you need VLAN support and 40K+ routes

  • If you want jumbo frames, you need VLAN support and 40K+ routes

  • If you want multicast, you need PIM-SM, preferably MSDP and M-BGP

  • If you want IPv6, you need M-BGP and space for even more routes


KAREN Connectors to date

  • About a dozen connectors at around 15 sites, wide range of equipment

  • CISCO 6500 series edge routers

  • Allied Telesyn AT-9924Ts

  • Juniper M and J series – J6350

  • Linux PC and Quagga Routing Suite

  • Check Point Firewall on Nokia platform


2007 outlook

  • Connectors, connectors, connectors!

  • Access solutions for schools and other non-BGP speakers

  • International IPv6 peering

  • Peering with US FedNets; China, Japan R&D networks

  • IPv6 DNS

  • Better solution to the 2/4/8 peering problem for complex sites

  • Pushing traffic around EAST/WEST for dual attached sites

  • L2 PIM-SM snooping (on top of IGMP snooping)

  • More security, core hardening

  • Stacking support in Napier and 10 Gbps services to Havelock North

  • Measurement infrastructure (active and passive) – capability build fund


Summary

  • KAREN creates a fast lane for the R&E community, inside New Zealand and with overseas R&D networks.

  • It offers a range of new services previously unavailable or inaccessible in New Zealand, such as multicast, IPv6.

  • It offers a test bed for novel tools and applications.

  • Performance is stunning – go and use it!


Acknowledgements and references

  • REANNZ KAREN http://www.karen.net.nz/

  • WAND weather map http://erg.cs.waikato.ac.nz/weathermap/

  • A cast of dozens of hands at TelstraClear and JazzTech

  • Questions: please contact myself or David Brownlie and Clayton Ejiofor at REANNZ.

    Thank you!


  • Login