Design and implementation of the reannz karen network j rg micheel
This presentation is the property of its rightful owner.
Sponsored Links
1 / 24

Design and Implementation of the REANNZ KAREN network Jörg Micheel PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Design and Implementation of the REANNZ KAREN network Jörg Micheel. Outline for this talk. Network design goals for KAREN Layer 1+2+3 network architecture Network services and implementation International transit network design Network performance

Download Presentation

Design and Implementation of the REANNZ KAREN network Jörg Micheel

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Design and implementation of the reannz karen network j rg micheel

Design and Implementation ofthe REANNZ KAREN networkJörg Micheel

Outline for this talk

Outline for this talk

  • Network design goals for KAREN

  • Layer 1+2+3 network architecture

  • Network services and implementation

  • International transit network design

  • Network performance

  • Checklist for KAREN connectors (REANNZ members)

  • Summary and references

Morst reannz karen design goals

MoRST/REANNZ/KAREN design goals

  • A high performance network for the NZ R&E community!

  • 10 Gbps capable backbone interconnecting all major cities in NZ

  • Access speeds at 1 Gbps and 10 Gbps

  • Unconstrained end-to-end performance at (multi-)Gigabits/second

  • Tailored on-demand performance for specific applications or experiments (bandwidth, delay, jitter)

  • International connectivity at 155 Mbps to AU, 622 Mbps to US

  • New services: multicast, IPv6, Jumbo frames (9000 Bytes MTU)

  • Virtual Private Network functionality for members

  • Telco-grade implementation and network management

  • Security, redundancy, high availability

  • Range of network measurement facilities (wire tap, NetFlow, SNMP data collection, active measurement) and development environment

  • Most importantly: stick to budget and timelines!!!

L1 l2 l3 network design

L1/L2/L3 Network Design

  • L1 Network core as rings on TCL OPTera DWDM

  • L1 Dark fiber spur to neutral POP and AAP

  • L2 Nationwide network based on Extreme X450a and BD10K

  • L3 using Juniper M320 in Auckland and Wellington

  • Note: 10GigE WANPHY is 9.287 Gbps!

  • ANOPS management network based on TCL PIP service and CISCO 2801

Extreme networks black diamond metro core switch

Extreme Networks Black Diamond – metro core switch

  • Black Diamond 10808 (BD10K)

  • 22 rack mount units

  • 1280 Gbps capacity (blocking)

  • Up to 48 10-Gigabit ports

  • Up to 480 10/100/1000 ports

  • Powerful VLAN, Virtual router Layer2 and Layer3 capabilities

  • Proprietary EAPS link-protection protocol provides continuity in case of fiber cut

  • L2/L3 Quality-of-Service

  • L2/L3 hardware filtering and priority

  • Jumbo frames at 9212

  • High availability, hardware redundancy

Extreme networks summit x450a edge switch

Extreme Networks Summit X450a – edge switch

  • X450a-24t with 24 ports 10/100/1000 copper, four combined SFP GigE ports

  • X450a-24x with 24 ports 1-GigE SFP, four combined 10/100/1000 ports

  • Optional dual 10-Gigabit Ethernet uplinks

  • 1 RU form factor

  • 160 Gigabits-per-second capacity

  • 65 million packets-per-second forwarding performance

  • Stacking capability with XOS 11.7 (April 2007)

  • Other features similar to Black Diamond series

Juniper networks m320 multi service edge router

Juniper Networks M320 Multi-service Edge Router

  • 320 Gbps switching capacity

  • 8 FPC slots with 20 Gbps FD capacity

  • ½ rack size

  • 32 PICs per chassis

  • 10 GigE capable

L2 l3 design

L2/L3 design

  • Connector joins KAREN via dark fiber

  • Switch access into one or more VLANs

  • BGP peering with core

  • L2 packet switched data nationally

  • L3 routing overseas

Karen service matrix

KAREN Service Matrix

Internet exchange model l2 switching bgp route reflector

Internet Exchange model (L2 switching + BGP Route Reflector)

“Switch – don’t route”

“Peer with two – route with many (others)”

 scalable Internet Exchange model

Challenges of a l2 network

Challenges of a L2 network

  • Redundant links will be pruned (Spanning Tree, etc), creating a star topology

  • Only difference between L2 resilience protocols is speed

  • Issue: capacity not utilised

  • Issue: shortest path

  • Issue: protect all VLANs

  • Solution: VLANs EAST/WEST for public IP services

  • Solution: Extreme EAPS for protection

Karen multicast two options for connectors

KAREN Multicast – two options for connectors

  • Bootstrap as part of KAREN multicast cloud (quick start for small sites, no MSDP, but doesn’t scale)

  • Create your own multicast domain (requires MSDP, scales well)

Multi protocol bgp and routing tables

Multi-Protocol BGP and routing tables

Karen international

KAREN International

  • Separate to domestic KAREN, but co-joined

  • As a static 3-point transit network has to implement all services (IPv4/IPv6 uni- and multicast, jumbo frames)

  • Routing policy ensures traffic flows between NZ and overseas peers (but not between other peers)

  • Pacific Wave landing point in Seattle poised for peering expansion

  • Bulk of 9K routes from Internet2 ITN

Reannz pop


Karen performance tests

KAREN performance tests

  • Network commissioning in October and November successfully demonstrated capacity, delay and jitter parameters

  • Bandwidth tests carried out as 1 Gbps VLANs POP-to-POP

  • All L2 components stressed at or near capacity limits (see next slide for example)

  • Delay and jitter tests carried out as RTT measurements using hardware loopbacks

Karen performance tests as shown on wand weathermap

KAREN performance tests (as shown on WAND weathermap)

See recorded animations at for other tests carried out during November and December.

Karen delay and jitter tests

KAREN delay and jitter tests

Karen delay and jitter results

KAREN delay and jitter results

Karen connectors 101 and shopping list

KAREN Connectors 101 (and shopping list)

  • KAREN is a Tier 1 network – you need to behave like a Tier 2 – control your own routing (policy).

  • Consider the services you want: IPv4 unicast, multicast, IPv6, Jumbo

  • Are you a heavy hitter ? Thinking of 10 Gbps ?

     Router that speaks BGP, holds 20K+ routes and does 1 Gbps

  • If you are a heavy hitter, you need VLAN support and 40K+ routes

  • If you want jumbo frames, you need VLAN support and 40K+ routes

  • If you want multicast, you need PIM-SM, preferably MSDP and M-BGP

  • If you want IPv6, you need M-BGP and space for even more routes

Karen connectors to date

KAREN Connectors to date

  • About a dozen connectors at around 15 sites, wide range of equipment

  • CISCO 6500 series edge routers

  • Allied Telesyn AT-9924Ts

  • Juniper M and J series – J6350

  • Linux PC and Quagga Routing Suite

  • Check Point Firewall on Nokia platform

2007 outlook

2007 outlook

  • Connectors, connectors, connectors!

  • Access solutions for schools and other non-BGP speakers

  • International IPv6 peering

  • Peering with US FedNets; China, Japan R&D networks

  • IPv6 DNS

  • Better solution to the 2/4/8 peering problem for complex sites

  • Pushing traffic around EAST/WEST for dual attached sites

  • L2 PIM-SM snooping (on top of IGMP snooping)

  • More security, core hardening

  • Stacking support in Napier and 10 Gbps services to Havelock North

  • Measurement infrastructure (active and passive) – capability build fund



  • KAREN creates a fast lane for the R&E community, inside New Zealand and with overseas R&D networks.

  • It offers a range of new services previously unavailable or inaccessible in New Zealand, such as multicast, IPv6.

  • It offers a test bed for novel tools and applications.

  • Performance is stunning – go and use it!

Acknowledgements and references

Acknowledgements and references


  • WAND weather map

  • A cast of dozens of hands at TelstraClear and JazzTech

  • Questions: please contact myself or David Brownlie and Clayton Ejiofor at REANNZ.

    Thank you!

  • Login