Review of ece301 computer organization
This presentation is the property of its rightful owner.
Sponsored Links
1 / 172

Review of ECE301: Computer Organization PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on
  • Presentation posted in: General

Review of ECE301: Computer Organization. AMD Barcelona: 4 cores. Abstractions. Abstraction helps us deal with complexity Hide lower-level detail Instruction set architecture (ISA) The hardware/software interface Application binary interface The ISA plus system software interface

Download Presentation

Review of ECE301: Computer Organization

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Review of ece301 computer organization

Review of ECE301: Computer Organization

AMD Barcelona: 4 cores

ECE610 - Fall 2013


Abstractions

Abstractions

  • Abstraction helps us deal with complexity

    • Hide lower-level detail

  • Instruction set architecture (ISA)

    • The hardware/software interface

  • Application binary interface

    • The ISA plus system software interface

  • Implementation

    • The details underlying and interface

E. D. Dijkstra

“… the main challenge of computer science is how not to get lost in the complexities of their own making.”

ECE610 - Fall 2013


Defining performance

Defining Performance

  • Which airplane has the best performance?

ECE610 - Fall 2013


Response time and throughput

Response Time and Throughput

  • Response time

    • How long it takes to do a task

  • Throughput

    • Total work done per unit time

      • e.g., tasks/transactions/… per hour

  • How are response time and throughput affected by

    • Replacing the processor with a faster version?

    • Adding more processors?

  • We’ll focus on response time for now…

ECE610 - Fall 2013


Relative performance

Relative Performance

  • Define Performance = 1/Execution Time

  • “X is n time faster than Y”

  • Example: time taken to run a program

    • 10s on A, 15s on B

    • Execution TimeB / Execution TimeA= 15s / 10s = 1.5

    • So A is 1.5 times faster than B

ECE610 - Fall 2013


Measuring execution time

Measuring Execution Time

  • Elapsed time

    • Total response time, including all aspects

      • Processing, I/O, OS overhead, idle time

    • Determines system performance

  • CPU time

    • Time spent processing a given job

      • Discounts I/O time, other jobs’ shares

    • Comprises user CPU time and system CPU time

    • Different programs are affected differently by CPU and system performance

ECE610 - Fall 2013


Cpu clocking

CPU Clocking

  • Operation of digital hardware governed by a constant-rate clock

Clock period

Clock (cycles)

Data transferand computation

Update state

  • Clock period: duration of a clock cycle

    • e.g., 250ps = 0.25ns = 250×10–12s

  • Clock frequency (rate): cycles per second

    • e.g., 4.0GHz = 4000MHz = 4.0×109Hz

ECE610 - Fall 2013


Cpu time

CPU Time

  • Performance improved by

    • Reducing number of clock cycles

    • Increasing clock rate

    • Hardware designer must often trade off clock rate against cycle count

ECE610 - Fall 2013


Cpu time example

CPU Time Example

  • Computer A: 2GHz clock, 10s CPU time

  • Designing Computer B

    • Aim for 6s CPU time

    • Can do faster clock, but causes 1.2 × clock cycles

  • How fast must Computer B clock be?

ECE610 - Fall 2013


Levels of program code

Levels of Program Code

  • High-level language

    • Level of abstraction closer to problem domain

    • Provides for productivity and portability

  • Assembly language

    • Textual representation of instructions

  • Hardware representation

    • Binary digits (bits)

    • Encoded instructions and data

ECE610 - Fall 2013


Instruction count and cpi

Instruction Count and CPI

  • Instruction Count for a program

    • Determined by program, ISA and compiler

  • Average cycles per instruction

    • Determined by CPU hardware

    • If different instructions have different CPI

      • Average CPI affected by instruction mix

ECE610 - Fall 2013


Cpi example

CPI Example

  • Computer A: Cycle Time = 250ps, CPI = 2.0

  • Computer B: Cycle Time = 500ps, CPI = 1.2

  • Same ISA

  • Which is faster, and by how much?

A is faster…

…by this much

ECE610 - Fall 2013


Cpi in more detail

CPI in More Detail

  • If different instruction classes take different numbers of cycles

  • Weighted average CPI

Relative frequency

ECE610 - Fall 2013


Cpi example1

CPI Example

  • Alternative compiled code sequences using instructions in classes A, B, C

  • Sequence 1: IC = 5

    • Clock Cycles= 2×1 + 1×2 + 2×3= 10

    • Avg. CPI = 10/5 = 2.0

  • Sequence 2: IC = 6

    • Clock Cycles= 4×1 + 1×2 + 1×3= 9

    • Avg. CPI = 9/6 = 1.5

ECE610 - Fall 2013


Performance summary

Performance Summary

The BIG Picture

  • Performance depends on

    • Algorithm: affects IC, possibly CPI

    • Programming language: affects IC, CPI

    • Compiler: affects IC, CPI

    • Instruction set architecture: affects IC, CPI, Tc

ECE610 - Fall 2013


Power trends

Power Trends

  • In CMOS IC technology

(source: intel.com)

×30

5V → 1V

×1000

ECE610 - Fall 2013


Reducing power

Reducing Power

  • Suppose a new CPU has

    • 85% of capacitive load of old CPU

    • 15% voltage and 15% frequency reduction

  • The power wall

    • We can’t reduce voltage further

    • We can’t remove more heat

  • How else can we improve performance?

ECE610 - Fall 2013


Uniprocessor performance

Uniprocessor Performance

Constrained by power, instruction-level parallelism, memory latency

ECE610 - Fall 2013


Multiprocessors

Multiprocessors

  • Multicore microprocessors

    • More than one processor per chip

  • Requires explicitly parallel programming

    • Compare with instruction level parallelism

      • Hardware executes multiple instructions at once

      • Hidden from the programmer

    • Hard to do

      • Programming for performance

      • Load balancing

      • Optimizing communication and synchronization

(source: Intel Inc. via Embedded.com)

ECE610 - Fall 2013


Manufacturing ics

Manufacturing ICs

  • Yield: proportion of working dies per wafer

ECE610 - Fall 2013


Amd opteron x2 wafer

AMD Opteron X2 Wafer

  • X2: 300mm wafer, 117 chips, 90nm technology

  • X4: 45nm technology

ECE610 - Fall 2013


Integrated circuit cost

Integrated Circuit Cost

  • Nonlinear relation to area and defect rate

    • Wafer cost and area are fixed

    • Defect rate determined by manufacturing process

    • Die area determined by architecture and circuit design

ECE610 - Fall 2013


Example

Example

ECE610 - Fall 2013


Spec cpu benchmark

SPEC CPU Benchmark

  • Programs used to measure performance

    • Supposedly typical of actual workload

  • Standard Performance Evaluation Corp (SPEC)

    • Develops benchmarks for CPU, I/O, Web, …

  • SPEC CPU2006

    • Elapsed time to execute a selection of programs

      • Negligible I/O, so focuses on CPU performance

    • Normalize relative to reference machine

    • Summarize as geometric mean of performance ratios

      • CINT2006 (integer) and CFP2006 (floating-point)

ECE610 - Fall 2013


Cint2006 for opteron x4 2356

CINT2006 for Opteron X4 2356

High cache miss rates

ECE610 - Fall 2013


Processor design

Processor design

ECE610 - Fall 2013


Instruction execution

Instruction Execution

  • PC  instruction memory, fetch instruction

  • Register numbers register file, read registers

  • Depending on instruction class

    • Use ALU to calculate

      • Arithmetic result

      • Memory address for load/store

      • Branch target address

    • Access data memory for load/store

    • PC  target address or PC + 4

ECE610 - Fall 2013


Mips instruction set

MIPS Instruction Set

Microprocessor without Interlocked Pipeline Stages

ECE610 - Fall 2013


Introduction

Introduction

  • CPU performance factors

    • Instruction count

      • Determined by ISA and compiler

    • CPI and Cycle time

      • Determined by CPU hardware

  • We will examine two MIPS implementations

    • A simplified version

    • A more realistic pipelined version

  • Simple subset, shows most aspects

    • Memory reference: lw, sw

    • Arithmetic/logical: add, sub, and, or, slt

    • Control transfer: beq

ECE610 - Fall 2013


Three instruction classes

Three Instruction Classes

ECE610 - Fall 2013


Cpu overview

CPU Overview

ECE610 - Fall 2013


Multiplexers

Multiplexers

  • Can’t just join wires together

    • Use multiplexers

ECE610 - Fall 2013


Control

Control

ECE610 - Fall 2013


Full datapath

Full Datapath

ECE610 - Fall 2013


Datapath with control

Datapath With Control

ECE610 - Fall 2013


R type instruction

R-Type Instruction

ECE610 - Fall 2013


Load instruction

Load Instruction

ECE610 - Fall 2013


Branch on equal insn

Branch-on-Equal Insn.

ECE610 - Fall 2013


Performance issues

Performance Issues

  • Longest delay determines clock period

    • Critical path: load instruction

    • Instruction memory  register file  ALU  data memory  register file

  • Not feasible to vary period for different instructions

  • Violates design principle

    • Making the common case fast

  • We will improve performance by pipelining

ECE610 - Fall 2013


Pipeline performance

Pipeline Performance

  • Assume time for stages is

    • 100ps for register read or write

    • 200ps for other stages

  • Compare pipelined datapath with single-cycle datapath

ECE610 - Fall 2013


Pipeline performance1

Pipeline Performance

Single-cycle (Tc= 800ps)

Pipelined (Tc= 200ps)

ECE610 - Fall 2013


Mips pipeline

MIPS Pipeline

  • Five stages, one step per stage

    • IF: Instruction fetch from memory

    • ID: Instruction decode & register read

    • EX: Execute operation or calculate address

    • MEM: Access memory operand

    • WB: Write result back to register

ECE610 - Fall 2013


Pipeline speedup

Pipeline Speedup

  • If all stages are balanced

    • i.e., all take the same time

    • Time between instructionspipelined= Time between instructionsnonpipelined Number of stages

  • If not balanced, speedup is less

  • Speedup due to increased throughput

    • Latency (time for each instruction) does not decrease

ECE610 - Fall 2013


Hazards

Hazards

  • Situations that prevent starting the next instruction in the next cycle

  • Structure hazards

    • A required resource is busy

  • Data hazard

    • Need to wait for previous instruction to complete its data read/write

  • Control hazard

    • Deciding on control action depends on previous instruction

ECE610 - Fall 2013


Data hazards

Data Hazards

  • An instruction depends on completion of data access by a previous instruction

    • add$s0, $t0, $t1sub$t2, $s0, $t3

ECE610 - Fall 2013


Forwarding aka bypassing

Forwarding (aka Bypassing)

  • Use result when it is computed

    • Don’t wait for it to be stored in a register

    • Requires extra connections in the datapath

ECE610 - Fall 2013


Load use data hazard

Load-Use Data Hazard

  • Can’t always avoid stalls by forwarding

    • If value not computed when needed

    • Can’t forward backward in time!

ECE610 - Fall 2013


Code scheduling to avoid stalls

Code Scheduling to Avoid Stalls

  • Reorder code to avoid use of load result in the next instruction

  • C code for A = B + E; C = B + F;

lw$t1, 0($t0)

lw$t2, 4($t0)

add$t3, $t1, $t2

sw$t3, 12($t0)

lw$t4, 8($t0)

add$t5, $t1, $t4

sw$t5, 16($t0)

lw$t1, 0($t0)

lw$t2, 4($t0)

lw$t4, 8($t0)

add$t3, $t1, $t2

sw$t3, 12($t0)

add$t5, $t1, $t4

sw$t5, 16($t0)

stall

stall

13 cycles

11 cycles

ECE610 - Fall 2013


Control hazards

Control Hazards

  • Branch determines flow of control

    • Fetching next instruction depends on branch outcome

    • Pipeline can’t always fetch correct instruction

      • Still working on ID stage of branch

  • In MIPS pipeline

    • Need to compare registers and compute target early in the pipeline

    • Add hardware to do it in ID stage

ECE610 - Fall 2013


Stall on branch

Stall on Branch

  • Wait until branch outcome determined before fetching next instruction

ECE610 - Fall 2013


Branch prediction

Branch Prediction

  • Longer pipelines can’t readily determine branch outcome early

    • Stall penalty becomes unacceptable

  • Predict outcome of branch

    • Only stall if prediction is wrong

  • In MIPS pipeline

    • Can predict branches not taken

    • Fetch instruction after branch, with no delay

ECE610 - Fall 2013


Mips with predict not taken

MIPS with Predict Not Taken

Prediction correct

Prediction incorrect

ECE610 - Fall 2013


More realistic branch prediction

More-Realistic Branch Prediction

  • Static branch prediction

    • Based on typical branch behavior

    • Example: loop and if-statement branches

      • Predict backward branches taken

      • Predict forward branches not taken

  • Dynamic branch prediction

    • Hardware measures actual branch behavior

      • e.g., record recent history of each branch

    • Assume future behavior will continue the trend

      • When wrong, stall while re-fetching, and update history

ECE610 - Fall 2013


Mips pipelined datapath

MIPS Pipelined Datapath

MEM

Right-to-left flow leads to hazards

WB

ECE610 - Fall 2013


Pipeline registers

Pipeline registers

  • Need registers between stages

    • To hold information produced in previous cycle

ECE610 - Fall 2013


If for load store

IF for Load, Store, …

ECE610 - Fall 2013


Id for load store

ID for Load, Store, …

ECE610 - Fall 2013


Ex for load

EX for Load

ECE610 - Fall 2013


Mem for load

MEM for Load

ECE610 - Fall 2013


Wb for load

WB for Load

Wrongregisternumber

ECE610 - Fall 2013


Corrected datapath for load

Corrected Datapath for Load

ECE610 - Fall 2013


Multi cycle pipeline diagram

Multi-Cycle Pipeline Diagram

  • Form showing resource usage

ECE610 - Fall 2013


Multi cycle pipeline diagram1

Multi-Cycle Pipeline Diagram

  • Traditional form

ECE610 - Fall 2013


Single cycle pipeline diagram

Single-Cycle Pipeline Diagram

  • State of pipeline in a given cycle

ECE610 - Fall 2013


Pipelined control

Pipelined Control

  • Control signals derived from instruction

    • As in single-cycle implementation

ECE610 - Fall 2013


Pipelined control1

Pipelined Control

ECE610 - Fall 2013


Dynamic branch prediction

Dynamic Branch Prediction

  • In deeper and superscalar pipelines, branch penalty is more significant

  • Use dynamic prediction

    • Branch prediction buffer (aka branch history table)

    • Indexed by recent branch instruction addresses

    • Stores outcome (taken/not taken)

    • To execute a branch

      • Check table, expect the same outcome

      • Start fetching from fall-through or target

      • If wrong, flush pipeline and flip prediction

ECE610 - Fall 2013


1 bit predictor shortcoming

1-Bit Predictor: Shortcoming

  • Inner loop branches mispredicted twice!

outer: … …inner: …

beq …, …, inner …beq …, …, outer

  • Mispredict as taken on last iteration of inner loop

  • Then mispredict as not taken on first iteration of inner loop next time around

ECE610 - Fall 2013


2 bit predictor

2-Bit Predictor

  • Only change prediction on two successive mispredictions

ECE610 - Fall 2013


Calculating the branch target

Calculating the Branch Target

  • Even with predictor, still need to calculate the target address

    • 1-cycle penalty for a taken branch

  • Branch target buffer

    • Cache of target addresses

    • Indexed by PC when instruction fetched

      • If hit and instruction is branch predicted taken, can fetch target immediately

ECE610 - Fall 2013


Exceptions and interrupts

Exceptions and Interrupts

  • “Unexpected” events requiring changein flow of control

    • Different ISAs use the terms differently

  • Exception

    • Arises within the CPU

      • e.g., undefined opcode, overflow, syscall, …

  • Interrupt

    • From an external I/O controller

  • Dealing with them without sacrificing performance is hard

ECE610 - Fall 2013


Handling exceptions

Handling Exceptions

  • In MIPS, exceptions managed by a System Control Coprocessor (CP0)

  • Save PC of offending (or interrupted) instruction

    • In MIPS: Exception Program Counter (EPC)

  • Save indication of the problem

    • In MIPS: Cause register

    • We’ll assume 1-bit

      • 0 for undefined opcode, 1 for overflow

  • Jump to handler at 8000 00180

ECE610 - Fall 2013


An alternate mechanism

An Alternate Mechanism

  • Vectored Interrupts

    • Handler address determined by the cause

  • Example:

    • Undefined opcode:C000 0000

    • Overflow:C000 0020

    • …:C000 0040

  • Instructions either

    • Deal with the interrupt, or

    • Jump to real handler

ECE610 - Fall 2013


Handler actions s w

Handler Actions (S/W)

  • Read cause, and transfer to relevant handler

  • Determine action required

  • If restartable

    • Take corrective action

    • use EPC to return to program

  • Otherwise

    • Terminate program

    • Report error using EPC, cause, …

ECE610 - Fall 2013


Exceptions in a pipeline

Exceptions in a Pipeline

  • Another form of control hazard

  • Consider overflow on add in EX stage

    add $1, $2, $1

    • Prevent $1 from being clobbered

    • Complete previous instructions

    • Flush “add”and subsequent instructions

    • Set Cause and EPC register values

    • Transfer control to handler

  • Similar to mispredicted branch

    • Use much of the same hardware

ECE610 - Fall 2013


Pipeline with exceptions

Pipeline with Exceptions

ECE610 - Fall 2013


Exception properties

Exception Properties

  • Restartable exceptions

    • Pipeline can flush the instruction

    • Handler executes, then returns to the instruction

      • Refetched and executed from scratch

  • PC saved in EPC register

    • Identifies causing instruction

    • Actually PC + 4 is saved

      • Handler must adjust

ECE610 - Fall 2013


Exception example

Exception Example

  • Exception on add in

    40sub $11, $2, $444and $12, $2, $548or $13, $2, $64Cadd $1, $2, $150slt $15, $6, $754lw $16, 50($7)…

  • Handler

    80000180sw $25, 1000($0)80000184sw $26, 1004($0)…

ECE610 - Fall 2013


Exception example1

Exception Example

ECE610 - Fall 2013


Exception example2

Exception Example

ECE610 - Fall 2013


Memory hierarchy

Memory Hierarchy

ECE610 - Fall 2013


Memory technology

Memory Technology

  • Static RAM (SRAM)

    • 0.5ns – 2.5ns, $2000 – $5000 per GB

  • Dynamic RAM (DRAM)

    • 50ns – 70ns, $20 – $75 per GB

  • Magnetic disk

    • 5ms – 20ms, $0.20 – $2 per GB

  • Ideal memory

    • Access time of SRAM

    • Capacity and cost/GB of disk

Flash memory

ECE610 - Fall 2013


Principle of locality

Principle of Locality

  • Programs access a small proportion of their address space at any time

  • Temporal locality

    • Items accessed recently are likely to be accessed again soon

    • e.g., instructions in a loop, induction variables

  • Spatial locality

    • Items near those accessed recently are likely to be accessed soon

    • E.g., sequential instruction access, array data

ECE610 - Fall 2013


Taking advantage of locality

Taking Advantage of Locality

  • Memory hierarchy

  • Store everything on disk

  • Copy recently accessed (and nearby) items from disk to smaller DRAM memory

    • Main memory

  • Copy more recently accessed (and nearby) items from DRAM to smaller SRAM memory

    • Cache memory attached to CPU

ECE610 - Fall 2013


Memory hierarchy levels

Memory Hierarchy Levels

  • Block (aka line): unit of copying

    • May be multiple words

  • If accessed data is present in upper level

    • Hit: access satisfied by upper level

      • Hit ratio: hits/accesses

  • If accessed data is absent

    • Miss: block copied from lower level

      • Time taken: miss penalty

      • Miss ratio: misses/accesses= 1 – hit ratio

    • Then accessed data supplied from upper level

ECE610 - Fall 2013


Address subdivision

Address Subdivision

ECE610 - Fall 2013


Cache misses

Cache Misses

  • On cache hit, CPU proceeds normally

  • On cache miss

    • Stall the CPU pipeline

    • Fetch block from next level of hierarchy

    • Instruction cache miss

      • Restart instruction fetch

    • Data cache miss

      • Complete data access

ECE610 - Fall 2013


Example larger block size

31

10

9

4

3

0

Tag

Index

Offset

22 bits

6 bits

4 bits

Example: Larger Block Size

  • 64 blocks, 16 bytes/block

    • To what block number does address 1200 map?

  • Block address = 1200/16 = 75

  • Block number = 75 modulo 64 = 11

ECE610 - Fall 2013


Block size considerations

Block Size Considerations

  • Larger blocks should reduce miss rate

    • Due to spatial locality

  • But in a fixed-sized cache

    • Larger blocks  fewer of them

      • More competition  increased miss rate

    • Larger blocks  pollution

  • Larger miss penalty

    • Can override benefit of reduced miss rate

    • Early restart and critical-word-first can help

ECE610 - Fall 2013


Measuring cache performance

Measuring Cache Performance

  • Components of CPU time

    • Program execution cycles

      • Includes cache hit time

    • Memory stall cycles

      • Mainly from cache misses

  • With simplifying assumptions:

ECE610 - Fall 2013


Cache performance example

Cache Performance Example

  • Given

    • I-cache miss rate = 2%

    • D-cache miss rate = 4%

    • Miss penalty = 100 cycles

    • Base CPI (ideal cache) = 2

    • Load & stores are 36% of instructions

  • Miss cycles per instruction

    • I-cache: 0.02 × 100 = 2

    • D-cache: 0.36 × 0.04 × 100 = 1.44

  • Actual CPI = 2 + 2 + 1.44 = 5.44

    • Ideal CPU is 5.44/2 =2.72 times faster

ECE610 - Fall 2013


Average access time

Average Access Time

  • Hit time is also important for performance

  • Average memory access time (AMAT)

    • AMAT = Hit time + Miss rate × Miss penalty

  • Example

    • CPU with 1ns clock, hit time = 1 cycle, miss penalty = 20 cycles, I-cache miss rate = 5%

    • AMAT = 1 + 0.05 × 20 = 2ns

      • 2 cycles per instruction

ECE610 - Fall 2013


Set associative cache organization

Set Associative Cache Organization

ECE610 - Fall 2013


Replacement policy

Replacement Policy

  • Direct mapped: no choice

  • Set associative

    • Prefer non-valid entry, if there is one

    • Otherwise, choose among entries in the set

  • Least-recently used (LRU)

    • Choose the one unused for the longest time

      • Simple for 2-way, manageable for 4-way, too hard beyond that

  • Random

    • Gives approximately the same performance as LRU for high associativity

ECE610 - Fall 2013


Virtual memory

Virtual Memory

  • Use main memory as a “cache” for secondary (disk) storage

    • Managed jointly by CPU hardware and the operating system (OS)

  • Programs share main memory

    • Each gets a private virtual address space holding its frequently used code and data

    • Protected from other programs

  • CPU and OS translate virtual addresses to physical addresses

    • VM “block” is called a page

    • VM translation “miss” is called a page fault

ECE610 - Fall 2013


Address translation

Address Translation

  • Fixed-size pages (e.g., 4K)

ECE610 - Fall 2013


Translation using a page table

Translation Using a Page Table

ECE610 - Fall 2013


Page tables

Page Tables

  • Stores placement information

    • Array of page table entries, indexed by virtual page number

    • Page table register in CPU points to page table in physical memory

  • If page is present in memory

    • PTE stores the physical page number

    • Plus other status bits (referenced, dirty, …)

  • If page is not present

    • PTE can refer to location in swap space on disk

ECE610 - Fall 2013


Replacement and writes

Replacement and Writes

  • To reduce page fault rate, prefer least-recently used (LRU) replacement

    • Reference bit (aka use bit) in PTE set to 1 on access to page

    • Periodically cleared to 0 by OS

    • A page with reference bit = 0 has not been used recently

  • Disk writes take millions of cycles

    • Block at once, not individual locations

    • Write through is impractical

    • Use write-back

    • Dirty bit in PTE set when page is written

ECE610 - Fall 2013


Fast translation using a tlb

Fast Translation Using a TLB

  • Address translation would appear to require extra memory references

    • One to access the PTE

    • Then the actual memory access

  • But access to page tables has good locality

    • So use a fast cache of PTEs within the CPU

    • Called a Translation Look-aside Buffer (TLB)

    • Typical: 16–512 PTEs, 0.5–1 cycle for hit, 10–100 cycles for miss, 0.01%–1% miss rate

    • Misses could be handled by hardware or software

ECE610 - Fall 2013


Fast translation using a tlb1

Fast Translation Using a TLB

ECE610 - Fall 2013


Tlb misses

TLB Misses

  • If page is in memory

    • Load the PTE from memory and retry

    • Could be handled in hardware

      • Can get complex for more complicated page table structures

    • Or in software

      • Raise a special exception, with optimized handler

  • If page is not in memory (page fault)

    • OS handles fetching the page and updating the page table

    • Then restart the faulting instruction

ECE610 - Fall 2013


Tlb miss handler

TLB Miss Handler

  • TLB miss indicates

    • Page present, but PTE not in TLB

    • Page not preset

  • Must recognize TLB miss before destination register overwritten

    • Raise exception

  • Handler copies PTE from memory to TLB

    • Then restarts instruction

    • If page not present, page fault will occur

ECE610 - Fall 2013


Page fault handler

Page Fault Handler

  • Use faulting virtual address to find PTE

  • Locate page on disk

  • Choose page to replace

    • If dirty, write to disk first

  • Read page into memory and update page table

  • Make process runnable again

    • Restart from faulting instruction

ECE610 - Fall 2013


Tlb and cache interaction

TLB and Cache Interaction

  • If cache tag uses physical address

    • Need to translate before cache lookup

  • Alternative: use virtual address tag

    • Complications due to aliasing

      • Different virtual addresses for shared physical address

ECE610 - Fall 2013


Sources of misses the three c s

Sources of misses: The Three C’s

  • Compulsory miss

    • Also called cold-start miss

  • Capacitymiss

  • Conflictmiss

    • Also called collision miss

ECE610 - Fall 2013


Sources of cache misses

Sources of Cache Misses

ECE610 - Fall 2013


Cache design trade offs

Cache Design Trade-offs

  • Decrease capacity misses

  • May increase access time

  • Decrease conflict misses

  • May increase access time

  • Decrease compulsory misses

  • Increases miss penalty. For very large block size, may increase miss rate due to pollution.

ECE610 - Fall 2013


Cache coherence problem

Cache Coherence Problem

  • Suppose two CPU cores share a physical address space

    • Write-through caches

ECE610 - Fall 2013


Coherence defined

Coherence Defined

  • Informally: Reads return most recently written value

  • Formally:

    • P writes X; P reads X (no intervening writes) read returns written value

    • P1 writes X; P2 reads X (sufficiently later) read returns written value

      • c.f. CPU B reading X after step 3 in example

    • P1 writes X, P2 writes X all processors see writes in the same order

      • End up with the same final value for X

ECE610 - Fall 2013


Cache coherence protocols

Cache Coherence Protocols

  • Operations performed by caches in multiprocessors to ensure coherence

    • Migration of data to local caches

      • Reduces bandwidth for shared memory

    • Replication of read-shared data

      • Reduces contention for access

  • Snooping protocols

    • Each cache monitors bus reads/writes

  • Directory-based protocols

    • Caches and memory record sharing status of blocks in a directory

ECE610 - Fall 2013


Invalidating snooping protocols

Invalidating Snooping Protocols

  • Cache gets exclusive access to a block when it is to be written

    • Broadcasts an invalidate message on the bus

    • Subsequent read in another cache misses

      • Owning cache supplies updated value

ECE610 - Fall 2013


Multiprocessor

Multiprocessor

ECE610 - Fall 2013


Hardware and software

Hardware and Software

  • Hardware

    • Serial: e.g., Pentium 4

    • Parallel: e.g., quad-core Xeon e5345

  • Software

    • Sequential: e.g., matrix multiplication

    • Concurrent: e.g., operating system

  • Sequential/concurrent software can run on serial/parallel hardware

    • Challenge: making effective use of parallel hardware

ECE610 - Fall 2013


Amdahl s law

Amdahl’s Law

  • Sequential part can limit speedup

  • Example: 100 processors, 90× speedup?

    • Tnew = Tparallelizable/100 + Tsequential

    • Solving: Fparallelizable = 0.999

  • Need sequential part to be 0.1% of original time

Amdahl, G. “Validity of the Single Processor Approach to Achieving Large-Scale Computing Capabilities” (1967)

ECE610 - Fall 2013


Amdahl s law1

Amdahl’s Law

ECE610 - Fall 2013


Difficulty of parallel programming

Difficulty of Parallel Programming

  • Parallel software is the problem

  • Need to get significant performance improvement

    • Otherwise, just use a faster uniprocessor, since it’s easier!

  • Difficulties

    • Partitioning

    • Coordination

    • Communications overhead

ECE610 - Fall 2013


Strong vs weak scaling

Strong vs Weak Scaling

  • Strong scaling: problem size fixed

    • Amdahl’s law

  • Weak scaling: problem size proportional to number of processors

    • 10 processors, 10 × 10 matrix

      • Time = 20 × tadd

    • 100 processors, 32 × 32 matrix

      • Time = 10 × tadd + 1000/100 × tadd = 20 × tadd

    • Constant performance in this example

    • Gustafson’s law

      • Computation time is the same; how much bigger problem (=data set)?

ECE610 - Fall 2013


Review of ece301 computer organization

ECE610 - Fall 2013


Shared memory

Shared Memory

  • SMP: shared memory multiprocessor

    • Hardware provides single physicaladdress space for all processors

    • Synchronize shared variables using locks

    • Memory access time

      • UMA (uniform) vs. NUMA (nonuniform)

ECE610 - Fall 2013


Message passing

Message Passing

  • Each processor has private physical address space

  • Hardware sends/receives messages between processors

ECE610 - Fall 2013


Example sum reduction

Example: Sum Reduction

  • Sum 100,000 numbers on 100 processor UMA

    • Each processor has ID: 0 ≤ Pn ≤ 99

    • Partition 1000 numbers per processor

    • Initial summation on each processorsum[Pn] = 0;for (i = 1000*Pn; i < 1000*(Pn+1); i++) sum[Pn] = sum[Pn] + A[i];

  • Now need to add these partial sums

    • Reduction: divide and conquer

    • Half the processors add pairs, then quarter, …

    • Need to synchronize between reduction steps

ECE610 - Fall 2013


Example sum reduction1

Example: Sum Reduction

half = 100;

repeat

synch();

if (half%2 != 0 && Pn == 0)

sum[0] = sum[0] + sum[half-1];

/* Conditional sum needed when half is odd;

Processor0 gets missing element */

half = half/2; /* dividing line on who sums */

if (Pn < half) sum[Pn] = sum[Pn] + sum[Pn+half];

until (half == 1);

ECE610 - Fall 2013


Sum reduction again

Sum Reduction (Again)

  • Sum 100,000 on 100 processors

  • First distribute 100 numbers to each

    • The do partial sums

      sum = 0;for (i = 0; i<1000; i = i + 1) sum = sum + AN[i];

  • Reduction

    • Half the processors send, other half receive and add

    • The quarter send, quarter receive and add, …

ECE610 - Fall 2013


Sum reduction again1

Sum Reduction (Again)

  • Given send() and receive() operations

    limit = 100; half = 100;/* 100 processors */repeat half = (half+1)/2; /* send vs. receive dividing line */ if (Pn >= half && Pn < limit)send(Pn - half, sum); if (Pn < (limit/2)) sum = sum + receive(); limit = half; /* upper limit of senders */until (half == 1); /* exit with final sum */

    • Send/receive also provide synchronization

    • Assumes send/receive take similar time to addition

ECE610 - Fall 2013


Loosely coupled clusters

Loosely Coupled Clusters

  • Network of independent computers

    • Each has private memory and OS

    • Connected using I/O system

      • E.g., Ethernet/switch, Internet

  • Suitable for applications with independent tasks

    • Web servers, databases, simulations, …

  • High availability, scalable, affordable

  • Problems

    • Administration cost (prefer virtual machines)

    • Low interconnect bandwidth

      • c.f. processor/memory bandwidth on an SMP

ECE610 - Fall 2013


Multithreading

Multithreading

  • Performing multiple threads of execution in parallel

    • Replicate registers, PC, etc.

    • Fast switching between threads

  • Fine-grain multithreading

    • Switch threads after each cycle

    • Interleave instruction execution

    • If one thread stalls, others are executed

  • Coarse-grain multithreading

    • Only switch on long stall (e.g., L2-cache miss)

    • Simplifies hardware, but doesn’t hide short stalls (eg, data hazards)

ECE610 - Fall 2013


Simultaneous multithreading

Simultaneous Multithreading

  • In multiple-issue dynamically scheduled processor

    • Schedule instructions from multiple threads

    • Instructions from independent threads execute when function units are available

    • Within threads, dependencies handled by scheduling and register renaming

  • Example: Intel Pentium-4 HT

    • Two threads: duplicated registers, shared function units and caches

ECE610 - Fall 2013


Multithreading example

Multithreading Example

ECE610 - Fall 2013


Instruction and data streams

Instruction and Data Streams

  • An alternate classification

  • SPMD: Single Program Multiple Data

    • A parallel program on a MIMD computer

    • Conditional code for different processors

ECE610 - Fall 2013


Review of ece301 computer organization

SIMD

  • Operate elementwise on vectors of data

    • E.g., MMX and SSE instructions in x86

      • Multiple data elements in 128-bit wide registers

  • All processors execute the same instruction at the same time

    • Each with different data address, etc.

  • Simplifies synchronization

  • Reduced instruction control hardware

  • Works best for highly data-parallel applications

ECE610 - Fall 2013


Vector processors

Vector Processors

  • Highly pipelined function units

  • Stream data from/to vector registers to units

    • Data collected from memory into registers

    • Results stored from registers to memory

  • Example: Vector extension to MIPS

    • 32 × 64-element registers (64-bit elements)

    • Vector instructions

      • lv, sv: load/store vector

      • addv.d: add vectors of double

      • addvs.d: add scalar to each element of vector of double

  • Significantly reduces instruction-fetch bandwidth

ECE610 - Fall 2013


Example daxpy y a x y

Example: DAXPY (Y = a × X + Y)

  • Conventional MIPS code

    l.d $f0,a($sp) ;load scalar a addiu r4,$s0,#512 ;upper bound of what to loadloop: l.d $f2,0($s0) ;load x(i) mul.d $f2,$f2,$f0 ;a × x(i) l.d $f4,0($s1) ;load y(i) add.d $f4,$f4,$f2 ;a × x(i) + y(i) s.d $f4,0($s1) ;store into y(i) addiu $s0,$s0,#8 ;increment index to x addiu $s1,$s1,#8 ;increment index to y subu $t0,r4,$s0 ;compute bound bne $t0,$zero,loop ;check if done

  • Vector MIPS code

    l.d $f0,a($sp) ;load scalar a lv $v1,0($s0) ;load vector x mulvs.d $v2,$v1,$f0 ;vector-scalar multiply lv $v3,0($s1) ;load vector y addv.d $v4,$v2,$v3 ;add y to product sv $v4,0($s1) ;store the result

ECE610 - Fall 2013


Vector vs scalar

Vector vs. Scalar

  • Vector architectures and compilers

    • Simplify data-parallel programming

    • Explicit statement of absence of loop-carried dependences

      • Reduced checking in hardware

    • Regular access patterns benefit from interleaved and burst memory

    • Avoid control hazards by avoiding loops

  • More general than ad-hoc media extensions (such as MMX, SSE)

    • Better match with compiler technology

ECE610 - Fall 2013


Gpu architectures

GPU Architectures

  • Processing is highly data-parallel

    • GPUs are highly multithreaded

    • Use thread switching to hide memory latency

      • Less reliance on multi-level caches

    • Graphics memory is wide and high-bandwidth

  • Trend toward general purpose GPUs

    • Heterogeneous CPU/GPU systems

    • CPU for sequential code, GPU for parallel code

  • Programming languages/APIs

    • DirectX, OpenGL

    • C for Graphics (Cg), High Level Shader Language (HLSL)

    • Compute Unified Device Architecture (CUDA)

NvidiaTegra 4

ECE610 - Fall 2013


Example nvidia tesla

Example: NVIDIA Tesla

Streaming multiprocessor

8 × Streamingprocessors

ECE610 - Fall 2013


Example nvidia tesla1

Example: NVIDIA Tesla

  • Streaming Processors

    • Single-precision FP and integer units

    • Each SP is fine-grained multithreaded

  • Warp: group of 32 threads

    • Executed in parallel,SIMD style

      • 8 SPs× 4 clock cycles

    • Hardware contextsfor 24 warps

      • Registers, PCs, …

ECE610 - Fall 2013


Classifying gpus

Classifying GPUs

  • Don’t fit nicely into SIMD/MIMD model

    • Conditional execution in a thread allows an illusion of MIMD

      • But with performance degradation

      • Need to write general purpose code with care

ECE610 - Fall 2013


Instruction level parallelism

Instruction Level Parallelism

ECE610 - Fall 2013


Instruction level parallelism ilp

Instruction-Level Parallelism (ILP)

  • Pipelining: executing multiple instructions in parallel

  • To increase ILP

    • Deeper pipeline

      • Less work per stage  shorter clock cycle

    • Multiple issue

      • Replicate pipeline stages  multiple pipelines

      • Start multiple instructions per clock cycle

      • CPI < 1, so use Instructions Per Cycle (IPC)

      • E.g., 4GHz 4-way multiple-issue

        • 16 BIPS, peak CPI = 0.25, peak IPC = 4

      • But dependencies reduce this in practice

ECE610 - Fall 2013


Multiple issue

Multiple Issue

  • Static multiple issue

    • Compiler groups instructions to be issued together

    • Packages them into “issue slots”

    • Compiler detects and avoids hazards

  • Dynamic multiple issue

    • CPU examines instruction stream and chooses instructions to issue each cycle

    • Compiler can help by reordering instructions

    • CPU resolves hazards using advanced techniques at runtime

ECE610 - Fall 2013


Speculation

Speculation

  • “Guess” what to do with an instruction

    • Start operation as soon as possible

    • Check whether guess was right

      • If so, complete the operation

      • If not, roll-back and do the right thing

  • Common to static and dynamic multiple issue

  • Examples

    • Speculate on branch outcome

      • Roll back if path taken is different

    • Speculate on load

      • Roll back if location is updated

ECE610 - Fall 2013


Compiler hardware speculation

Compiler/Hardware Speculation

  • Compiler can reorder instructions

    • e.g., move load before branch

    • Can include “fix-up” instructions to recover from incorrect guess

  • Hardware can look ahead for instructions to execute

    • Buffer results until it determines they are actually needed

    • Flush buffers on incorrect speculation

ECE610 - Fall 2013


Speculation and exceptions

Speculation and Exceptions

  • What if exception occurs on a speculatively executed instruction?

    • e.g., speculative load before null-pointer check

  • Static speculation

    • Can add ISA support for deferring exceptions

  • Dynamic speculation

    • Can buffer exceptions until instruction completion (which may not occur)

ECE610 - Fall 2013


Static multiple issue

Static Multiple Issue

  • Compiler groups instructions into “issue packets”

    • Group of instructions that can be issued on a single cycle

    • Determined by pipeline resources required

  • Think of an issue packet as a very long instruction

    • Specifies multiple concurrent operations

    •  Very Long Instruction Word (VLIW)

ECE610 - Fall 2013


Scheduling static multiple issue

Scheduling Static Multiple Issue

  • Compiler must remove some/all hazards

    • Reorder instructions into issue packets

    • No dependencies with a packet

    • Possibly some dependencies between packets

      • Varies between ISAs; compiler must know!

    • Pad with nop if necessary

ECE610 - Fall 2013


Mips with static dual issue

MIPS with Static Dual Issue

  • Two-issue packets

    • One ALU/branch instruction

    • One load/store instruction

    • 64-bit aligned

      • ALU/branch, then load/store

      • Pad an unused instruction with nop

ECE610 - Fall 2013


Mips with static dual issue1

MIPS with Static Dual Issue

ECE610 - Fall 2013


Hazards in the dual issue mips

Hazards in the Dual-Issue MIPS

  • More instructions executing in parallel

  • EX data hazard

    • Forwarding avoided stalls with single-issue

    • Now can’t use ALU result in load/store in same packet

      • add $t0, $s0, $s1load $s2, 0($t0)

      • Split into two packets, effectively a stall

  • Load-use hazard

    • Still one cycle use latency, but now two instructions

  • More aggressive scheduling required

ECE610 - Fall 2013


Scheduling example

Scheduling Example

  • Schedule this for dual-issue MIPS

Loop: lw $t0, 0($s1) # $t0=array element addu $t0, $t0, $s2 # add scalar in $s2 sw $t0, 0($s1) # store result addi $s1, $s1,–4 # decrement pointer bne $s1, $zero, Loop # branch $s1!=0

  • IPC = 5/4 = 1.25 (c.f. peak IPC = 2)

ECE610 - Fall 2013


Loop unrolling

Loop Unrolling

  • Replicate loop body to expose more parallelism

    • Reduces loop-control overhead

  • Use different registers per replication

    • Called “register renaming”

    • Avoid loop-carried “anti-dependencies”

      • Store followed by a load of the same register

      • Aka “name dependence”

        • Reuse of a register name

ECE610 - Fall 2013


Loop unrolling example

Loop Unrolling Example

  • IPC = 14/8 = 1.75

    • Closer to 2, but at cost of registers and code size

ECE610 - Fall 2013


Dynamic multiple issue

Dynamic Multiple Issue

  • “Superscalar” processors

  • CPU decides whether to issue 0, 1, 2, … each cycle

    • Avoiding structural and data hazards

  • Avoids the need for compiler scheduling

    • Though it may still help

    • Code semantics ensured by the CPU

ECE610 - Fall 2013


Dynamic pipeline scheduling

Dynamic Pipeline Scheduling

  • Allow the CPU to execute instructions out of order to avoid stalls

    • But commit result to registers in order

  • Example

    lw $t0, 20($s2)addu $t1, $t0, $t2sub $s4, $s4, $t3slti $t5, $s4, 20

    • Can start sub while addu is waiting for lw

ECE610 - Fall 2013


Dynamically scheduled cpu

Dynamically Scheduled CPU

Preserves dependencies

Hold pending operands

Results also sent to any waiting reservation stations

Reorders buffer for register writes

Can supply operands for issued instructions

ECE610 - Fall 2013


Register renaming

Register Renaming

  • Reservation stations and reorder buffer effectively provide register renaming

  • On instruction issue to reservation station

    • If operand is available in register file or reorder buffer

      • Copied to reservation station

      • No longer required in the register; can be overwritten

    • If operand is not yet available

      • It will be provided to the reservation station by a function unit

      • Register update may not be required

ECE610 - Fall 2013


Speculation1

Speculation

  • Predict branch and continue issuing

    • Don’t commit until branch outcome determined

  • Load speculation

    • Avoid load and cache miss delay

      • Predict the effective address

      • Predict loaded value

      • Load before completing outstanding stores

      • Bypass stored values to load unit

    • Don’t commit load until speculation cleared

ECE610 - Fall 2013


Why do dynamic scheduling

Why Do Dynamic Scheduling?

  • Why not just let the compiler schedule code?

  • Not all stalls are predictable

    • e.g., cache misses

  • Can’t always schedule around branches

    • Branch outcome is dynamically determined

  • Different implementations of an ISA have different latencies and hazards

ECE610 - Fall 2013


Does multiple issue work

Does Multiple Issue Work?

The BIG Picture

  • Yes, but not as much as we’d like

  • Programs have real dependencies that limit ILP

  • Some dependencies are hard to eliminate

    • e.g., pointer aliasing

  • Some parallelism is hard to expose

    • Limited window size during instruction issue

  • Memory delays and limited bandwidth

    • Hard to keep pipelines full

  • Speculation can help if done well

ECE610 - Fall 2013


Power efficiency

Power Efficiency

  • Complexity of dynamic scheduling and speculations requires power

  • Multiple simpler cores may be better

ECE610 - Fall 2013


The opteron x4 microarchitecture

The Opteron X4 Microarchitecture

72 physical registers

ECE610 - Fall 2013


The opteron x4 pipeline flow

The Opteron X4 Pipeline Flow

  • For integer operations

  • FP is 5 stages longer

  • Up to 106 RISC-ops in progress

  • Bottlenecks

    • Complex instructions with long dependencies

    • Branch mispredictions

    • Memory access delays

  • ECE610 - Fall 2013


    System architecture

    System Architecture

    ECE610 - Fall 2013


    Sun fire x4150 1u server

    Sun Fire x4150 1U server

    4 cores each

    16 x 4GB = 64GB DRAM

    ECE610 - Fall 2013


    Typical x86 pc i o system

    Typical x86 PC I/O System

    ECE610 - Fall 2013


    I o commands

    I/O Commands

    • I/O devices are managed by I/O controller hardware

      • Transfers data to/from device

      • Synchronizes operations with software

    • Command registers

      • Cause device to do something

    • Status registers

      • Indicate what the device is doing and occurrence of errors

    • Data registers

      • Write: transfer data to a device

      • Read: transfer data from a device

    done

    error

    0x0f000010

    Status

    0x0f000014

    Data

    ECE610 - Fall 2013


    I o register mapping

    I/O Register Mapping

    • Memory mapped I/O

      • Registers are addressed in same space as memory

      • Address decoder distinguishes between them

      • OS uses address translation mechanism to make them only accessible to kernel

    ECE610 - Fall 2013


    Polling

    Polling

    • Periodically check I/O status register

      • If device ready, do operation

      • If error, take action

    • Common in small or low-performance real-time embedded systems

      • Predictable timing

      • Low hardware cost

    • In other systems, wastes CPU time

    ECE610 - Fall 2013


    Interrupts

    Interrupts

    • When a device is ready or error occurs

      • Controller interrupts CPU

    • Interrupt is like an exception

      • But not synchronized to instruction execution

      • Can invoke handler between instructions

      • Cause information often identifies the interrupting device

    • Priority interrupts

      • Devices needing more urgent attention get higher priority

      • Can interrupt handler for a lower priority interrupt

    ECE610 - Fall 2013


    I o data transfer

    I/O Data Transfer

    • Polling and interrupt-driven I/O

      • CPU transfers data between memory and I/O data registers

      • Time consuming for high-speed devices

    • Direct memory access (DMA)

      • OS provides starting address in memory

      • I/O controller transfers to/from memory autonomously

      • Controller interrupts on completion or error

    ECE610 - Fall 2013


    Dma vm interaction

    DMA/VM Interaction

    • OS uses virtual addresses for memory

      • DMA blocks may not be contiguous in physical memory

    • Should DMA use virtual addresses?

      • Would require controller to do translation

    • If DMA uses physical addresses

      • May need to break transfers into page-sized chunks

      • Or chain multiple transfers

      • Or allocate contiguous physical pages for DMA

    ECE610 - Fall 2013


    Dma cache interaction

    DMA/Cache Interaction

    • If DMA writes to a memory block that is cached

      • Cached copy becomes stale

    • If write-back cache has dirty block, and DMA reads memory block

      • Reads stale data

    • Need to ensure cache coherence

      • Flush blocks from cache if they will be used for DMA

        • Cache flushing by OS (invalidate some blocks)

        • Hardware invalidation (typical in multiprocessors)

      • Or use non-cacheable memory locations for I/O

    ECE610 - Fall 2013


  • Login