CS1104: Computer Organisation comp.nus.sg/~cs1104

1. CS1104: Computer Organisation http://www.comp.nus.edu.sg/~cs1104 School of Computing National University of Singapore

2. PII Lecture 8: Memory Hierarchy • Memory Technologies • Memory Hierarchy Analogy • Illusion of Large, Fast, Cheap Memory • Principle of Locality • Terminology Memory Hierarchy

3. PII Lecture 8: Memory Hierarchy • Reading: • Chapter 8 of textbook, which is Chapter 5 in “Computer Organization” by Hamacher, Vranesic and Zaky. Memory Hierarchy

4. Memory Technologies • RAM (Random Access Memory): access time is the same for all locations (in nanoseconds). • DRAM: Dynamic RAM • High density, low power, cheap, slow (access time: 60-120 ns). • Dynamic: needs to be “refreshed” regularly. • SRAM: Static RAM • Low density, high power, expensive, fast (access time: 5-25 ns). • Static: non-volatile – content lasts “forever” (until power is removed). Memory Hierarchy

5. Memory Technologies (2) • ROM (Read-Only Memory): non-volatile, allows for read only. • PROM: Programmable ROM • EPROM: Erasable PROM • EEPROM: Electrically EPROM • Flash Memory • Similar to EEPROM • Difference: read a single cell, but write a block Memory Hierarchy

6. 21-bit addresses 19-bit internal chip address A 0 A 1 A 19 A 20 2-bit decoder 512k X 8 memory chip 512K x 8 memory chip D D D D 31-24 23-16 15-8 7-0 19-bit address 8-bit data input/output Chip select Recap: Memory Organisation Another example: Organization of a 2M  32 memory module using 512K  8 static memory chips. Memory Hierarchy

7. 1000:1! 2:1! Technology Trends Capacity Speed (latency) Processor: -- 4x in 3 yrs DRAM: 4x in 3 yrs 2x in 10 yrs Disk: 4x in 3 yrs 2x in 10 yrs DRAM Year Size Cycle Time 1980 64 Kb 250 ns 1983 256 Kb 220 ns 1986 1 Mb 190 ns 1989 4 Mb 165 ns 1993 16 Mb 145 ns 1997 64 Mb 120 ns Memory Hierarchy

8. µProc 60%/yr. (2X/1.5yr) 1000 CPU 100 Processor-Memory Performance Gap:(grows 50% / year) 10 DRAM 9%/yr. (2X/10 yrs) DRAM 1 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Time Technology Trends (2) Memory Hierarchy

9. Analogy: Term Paper in Library • Working on a paper at a desk in library. • Option 1: Every time a book is needed… • Leave desk to go to shelves (or stacks) • Find the book • Bring one book back to desk • Read section interested in • When done, leave desk and go to shelves carrying book • Put book back on shelf • Return to desk to work Memory Hierarchy

10. Analogy: Term Paper in Library (2) • Option 2: Every time a book is needed… • Leave some books on desk after fetching them • Only go to shelves when a book not on desk is needed • At the shelves, fetch related books in case you need them; sometimes you will need to return books not used recently to make space for new books on desk (replacement algorithm) • Return to desk to work • Illusion: whole library on your desk Memory Hierarchy

11. Illusion of Large, Fast, Cheap Memory • Registers in processor: Very fast, in order of nanoseconds to picoseconds. • Disks: Very slow, in order of milliseconds. • Large memories (disks) are slow; fast memories (registers) are small. • How do we create a memory that is large, cheap and fast (most of the time)? • Memory hierarchy. Memory Hierarchy

12. Central Processor Unit (CPU) “Upper” Increasing distance from CPU,decreasing cost/bit Level 1 Levels in memory hierarchy Level 2 Level 3 . . . “Lower” Size of memory at each level Memory Hierarchy Pyramid Level n (Data cannot be in level i unless it is also in i+1.) Memory Hierarchy

13. Probability of reference 0 2n - 1 Address Space Principle of Locality • Principle of locality (or locality of reference): • Program accesses a relatively small portion of the address space at any instant of time. • Temporal locality and spatial locality. Memory Hierarchy

14. Principle of Locality (2) • Temporal locality (locality in time): •  Keep most recently accessed data items closer to the processor. • Library analogy: Recently read books are kept on desk. • Spatial locality (locality in space): •  Move blocks consisting of contiguous words to ‘upper’ levels. • Block is a unit of transfer. • Library analogy: Bring back nearby books on shelves when fetching a book, hoping that you might need them soon. Memory Hierarchy

15. Principle of Locality (3) • What programming constructs lead to principle of locality? Memory Hierarchy

16. Memory Hierarchy: Idea • Temporal locality: keep recently accessed data items closer to processor. • Spatial locality: move contiguous words in memory to upper levels of hierarchy. • Use smaller and faster memory technologies closer to the processor • If hit rate is high enough, hierarchy has access time close to the highest (fastest) level and size equal to the lowest (largest) level. Memory Hierarchy

17. Memory Hierarchy: Terminology • Hit: data appears in some block in the upper level. • Hit rate: the fraction of memory access found in the upper level. • Analogy: fraction of time you find the book on desk. • Miss: data is not at upper level and needs to be retrieved from a block in the lower level. • Miss rate: 1 – Hit rate • Analogy: fraction of time you need to go to shelves for the book. Memory Hierarchy

18. Memory Hierarchy: Terminology (2) • (Average) Hit time: time to access the upper level which consists of • Time to determine hit/miss + memory access time. • Analogy: time to find and pick up book from desk. • (Average) Miss penalty: time to replace a block in the upper level + time to deliver the block to the processor. • Analogy: time to go to shelves, find needed book, and return to your desk. • Hit time << Miss penalty. Memory Hierarchy

19. Processor Register Increasing size Increasing speed Increasing cost per bit Primary cache L1 Secondary cache L2 Main memory Magnetic disk secondary memory Current Memory Hierarchy Memory Hierarchy

20. Processor Second- ary Memory Control Main Memory L2 Cache Datapath L1 $ Regs Current Memory Hierarchy (2) Speed(ns): 0.5ns 2ns 6ns 100ns 10,000,000ns Size (MB): 0.0005 0.05 1-4 100-1000 100,000 Cost ($/MB): -- $100 $30 $1 $0.05 Technology: Regs SRAM SRAM DRAM Disk Memory Hierarchy

21. Management of the Hierarchy • Registers  Memory • By compiler (or assembly programmer). • Cache  Main memory • By the hardware. • Main memory  Disks • By the hardware and operating system. • By the programmer (through files). Memory Hierarchy

22. Virtual Memory • Main memory smaller than address space • Example: 32-bit address allows an address space of 4G bytes, but main memory may only be a few hundred megabytes. • Parts of program not in main memory are stored on secondary storage devices, such as disks. • Operating system moves programs and data automatically between the physical main memory and secondary storage (virtual memory). Memory Hierarchy

23. Things To Remember • Principle of locality and hierarchy of memories of different speed and cost; exploit locality to improve cost-performance. • Terms: Hit, miss, hit rate, miss rate, hit time, miss penalty, block, upper-level memory, lower-level memory. Memory Hierarchy

24. End of file