Access regulation to hot modules in wormhole nocs
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Access Regulation to Hot-Modules in Wormhole NoCs PowerPoint PPT Presentation

Access Regulation to Hot-Modules in Wormhole NoCs. Or: Hot-Modules, Cool NoCs. Isask’har (Zigi) Walter Supervised by: Israel Cidon, Ran Ginosar and Avinoam Kolodny. Technion – Israel Institute of Technology. Hot-Modules. NoC is designed and dimensioned to meet QoS requirements

Download Presentation

Access Regulation to Hot-Modules in Wormhole NoCs

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Access regulation to hot modules in wormhole nocs

Access Regulation toHot-Modules in Wormhole NoCs

Or: Hot-Modules, Cool NoCs

Isask’har (Zigi) Walter

Supervised by:

Israel Cidon, Ran Ginosar and Avinoam Kolodny

Technion – Israel Institute of Technology


Access regulation to

Hot-Modules

  • NoC is designed and dimensioned to meet QoS requirements

    • Buffer sizing, routing, router arbitration, link capacities, …

  • NoC designers cannot tune everything

    • Modules typically have limited capacity

  • High-demanded, bandwidth limited modulescreate edge bottlenecks

    • In SoC, often known in advance

      • Off-chip DRAM, on-chip special purpose processor

  • System performance is strongly affected

    • Even if the NoC has infinite bandwidth

Hot-Modules in Wormhole NoCs


Access regulation to

Hot Module (HM) in NoC

  • Wormhole, BE NoC

  • At high Hot Moduleutilization, multiple worms “get stuck” in the network

  • Two problems arise:

    • System Performance

    • Source Fairness

IP(HM)

Interface

Hot-Modules in Wormhole NoCs


Access regulation to

Hot Module Affects the System

IP1(HM)

IP2

Problem#1

Interface

Interface

Interface

IP3

  • HM is not a local problem. Traffic not destined at the HM suffers too!

Hot-Modules in Wormhole NoCs


Access regulation to

Source Fairness Problem

Problem#2

HM

Interface

Multiple locally fair decisions

Global fairness

  • The limited, expensive HMresource isn’t fairly shared

Hot-Modules in Wormhole NoCs


Our approach

Our Approach

  • Problem is not caused by the NoC

    • But rather by a congested end-point

  • Solution should address the root cause

    • Not the symptoms

  • Utilize existing NoC infrastructure

  • Solve both problems

    • Simple and efficient

Hot-Modules in Wormhole NoCs


Hot module congestion

Hot Module Congestion

During congested periods, sources should not inject packets towards the HM

Will experience increased delay anyway

Better wait at the source, not in the network

Keep routers unmodified!

Hot-Modules in Wormhole NoCs


Access regulation to

HM Allocation Control Basics

Control

IP3

Interface

IP1

AllocationController

IP2(HM)

Interface

NoC

Interface

Interface

IP4

Hot-Modules in Wormhole NoCs


Access regulation to

HM Allocation Control Basics

IP3

Interface

IP1

Control

AllocationController

IP2(HM)

Interface

NoC

Interface

Interface

IP4

Hot-Modules in Wormhole NoCs


Access regulation to

HM Allocation Control Basics

IP3

Interface

IP1

Control

AllocationController

IP2(HM)

Interface

NoC

Interface

Interface

IP4

Hot-Modules in Wormhole NoCs


Access regulation to

HM Control Packets

Dest.

Source

Req. Credit

Dest.

Source

Credit

Credit request packet

Credit reply packet

  • The HM Controller receives all requests and can employ any scheduling policy

  • Control packets are sent using a high service level

    • Bypassing (blocked) data packets!

Hot-Modules in Wormhole NoCs


Multiple priority router

Multiple Priority Router

Control packets

Hot-Modules in Wormhole NoCs


Access regulation to

Enhanced Request packet

Dest.

Source

Req. Credit

Priority

Expiration

Deadline

  • The request may include additional data as needed

    • payload’s priority, deadline, expiration time, etc.

Optional fields

Credit request packet

Hot-Modules in Wormhole NoCs


Hm allocation controller

HM Allocation Controller

Optional

  • The HM Allocation Controller is customized according to system’s requirements

CreditRequests

CreditReplies

Requests Decoder

Reply Encoder

PendingRequestsTable

LocalArbiter

HM Access Controller

Hot-Modules in Wormhole NoCs


Further enhancements

Further Enhancements

  • Short packets are not negotiated

  • Source’s quota is slowly self-refreshing

  • The mechanism is turned-off when the network is not congested

  • Crediting modules ahead of time hides request-grant latency

    • For light-load periods

Hot-Modules in Wormhole NoCs


Not classic flow control

Not Classic Flow-Control

  • Flow-control protects destination’s buffer

    • A pair-wise protocol

  • HM access regulation protects the system

    • Many-to-one protocol

Hot-Modules in Wormhole NoCs


Results synthetic scenario

Results – Synthetic scenario

R

R

R

R

R

R

R

R

R

R

R

R

HM

Module

Module

Module

R

R

R

R

Module

Module

Module

Module

Module

Module

Module

Module

Module

Module

Module

Module

  • Hotspot traffic

    • All-to-one traffic with all-to-all background traffic

  • High network capacity

  • Limited hot module bandwidth

  • HM controller arbitration: Round-robin

Hot-Modules in Wormhole NoCs


System performance

System Performance

Without regulation

WithRegulation

Average Packet Latency

X30

X10

Hot-Modules in Wormhole NoCs


Hot vs non hot module traffic

Hot vs. non-Hot Module Traffic

Background TrafficWithout regulation

HM Trafficwithout regulation

HM Trafficwith regulation

Background TrafficWith regulation

Average Packet Latency

X40

Using regulation, non-HM traffic latency is drastically reduced

Hot-Modules in Wormhole NoCs


Source fairness

Source Fairness

Source#16no regulation

Source#16with regulation

Source#5with regulation

Source#5no regulation

R

R

R

R

1

2

3

4

R

R

R

R

5

6

7

8

R

R

R

R

9

10

11

12

R

R

R

R

13

14

15

16

Hot-Modules in Wormhole NoCs


Fairness in saturated network

Fairness in Saturated Network

No allocation control

With allocation control

Simulation results for a 4-by-4 system,Data packet length: 200 flitsControl packet length: 2 flits

  • Hot-Module Utilization: 99.99%

  • Regulated Hot-Module Utilization: 98.32%

Hot-Modules in Wormhole NoCs


Mpeg 4 decoder

MPEG-4 Decoder

SRAM2

SDRAM

22% of all traffic

MED CPU

VU

RAST

AU

SRAM2

SDRAM

IDCT

SRAM1

UP SAMP

BAB

RISC

ADSP

25% of all traffic

  • Real SoC

  • Over provisioned NoC

  • Two hot-modules

Hot-Modules in Wormhole NoCs


Results mpeg 4 decoder

Results – MPEG-4 Decoder

All traffic

HM/non-HM traffic breakdown

X8

X2

@80% load: X8 reduction

@80% load: X2 reduction

Hot-Modules in Wormhole NoCs


The hms are better utilized

The HMs are better utilized

Significant differences in BW!

No allocation control

With allocation control

Flows destined at HM1

Flows destined at HM2

Total

1HM1 2HM1 3HM1 4HM1 9HM1 10HM1 11HM1

8HM2 10HM2 11HM2 12HM2

  • Without regulation, the hot-modules are only 60% utilized

    • Traffic to one HM blocks the traffic to the other!

Hot-Modules in Wormhole NoCs


Hot module placement

Hot-Module Placement

Hot-Modules in Wormhole NoCs


Summary

Summary

  • Hot-modules are common in real SoCs

  • Hot-modules ruin system performance and are not fairly shared

    • Even in NoCs with infinite capacity

    • The network intensifies the problem

    • But can also provide tools for resolving it

  • Simple mechanism achieves dramatic improvement

    • Completely eliminating the HM effects

Hot-Modules, Cool NoCs!

Hot-Modules in Wormhole NoCs


Access regulation to

Hot-Modules, Cool NoCs!

QNoC

Research

Group

Thank you!

Questions?

[email protected]

QNoC

Research

Group

Hot-Modules in Wormhole NoCs


Backup slides

Backup Slides

Hot-Modules in Wormhole NoCs


Wormhole routing

Wormhole Routing

  • Suits well on chip interconnect

    • Small number of buffers

    • Low latency

IP2

Interface

Interface

IP1

Hot-Modules in Wormhole NoCs


Router based approach

Router-Based Approach?

Input Buffer

Output Buffer

  • Virtual circuit

  • Fair queuing

  • Dedicated queues

  • Deflective routing

  • Packet combining

  • Packet dropping

  • Backpressure (credit/rate based)

  • and more…

X-Bar

  • Can router detect congested periods?

Hot-Modules in Wormhole NoCs


Router based solutions

Router-Based Solutions?

Input Buffer

Output Buffer

  • Routers must be fast, power and area efficient

    • A few buffers

    • Efficient routing

    • Simple arbitration policy

    • No state/flow memory

  • Problem caused by end-points

    • Address the root-cause

X-Bar

Hot-Modules in Wormhole NoCs


Future work

Future Work

  • Dynamically set hot-modules

  • Other scheduling policies at hot-module controller

  • Single/Multiple control modules for multiple HMs

  • Placement

Hot-Modules in Wormhole NoCs


Access regulation to

References

[1]E. Bolotin, I. Cidon, R. Ginosar, A. Kolodny, “QoS Architecture and Design Process for Cost-Effective Network on Chip”, Journal of Systems Architecture, 2004

[2] G. F. Pfister and V. A. Norton, "Hot Spot contention and combining in multistage interconnection networks," IEEE Trans. Comp., Oct. 1985

[3] D. Bertozzi, A. Jalabert, S. Murali R. Tamhankar, S. Stergiou, L. Benini, and G. De Micheli , "NoC Synthesis Flow for Customized Domain Specific Multiprocessor Systems-on-Chip", IEEE Transactions on Parallel and Distributed Systems, 2005

Hot-Modules in Wormhole NoCs


Is that new problem

Is that new problem?

Seem to draw most attention

  • Hotspots were comprehensively studied in the past

  • Classically, solutions are categorized by the mechanism policy

    • Avoidance-based (frequently impossible)

    • Detection-based (requires threshold tuning)

    • Prevention-based (overhead during light load)

  • And by the mechanism implementation

    • Central arbitration

    • Router-based

    • End-to-end flow-control

Hot-Modules in Wormhole NoCs


Saturation un fairness

Saturation (Un)Fairness

Less than 1% of HM BW!

  • A saturated router divides available BW equally between inputs

R

R

R

R

HM

IP

IP

IP

R

R

R

R

IP

IP

IP

IP

R

R

R

R

IP

IP

IP

IP

Hot-Modules in Wormhole NoCs


  • Login