Virtual Memory

1. Virtual Memory

2. Last Week Memory Management • Increase degree of multiprogramming • Entire process needs to fit into memory • Dynamic Linking and Loading • Swapping • Contiguous Memory Allocation • Dynamic storage memory allocation problem • First-fit, best-fit, worst-fit • Fragmentation - external and internal • Paging • Structure of the Page Table • Segmentation

3. Goals for Today • Virtual memory • How does it work? • Page faults • Resuming after page faults • When to fetch? • What to replace? • Page replacement algorithms • FIFO, OPT, LRU (Clock) • Page Buffering • Allocating Pages to processes

4. disk page 0 page 1 page table page 2 page 3 page 4 page N Physical memory Virtual memory What is virtual memory? • Each process has illusion of large address space • 232 for 32-bit addressing • However, physical memory is much smaller • How do we give this illusion to multiple processes? • Virtual Memory: some addresses reside in disk

5. Virtual memory • Separates users logical memory from physical memory. • Only part of the program needs to be in memory for execution • Logical address space can therefore be much larger than physical address space • Allows address spaces to be shared by several processes • Allows for more efficient process creation

6. Virtual Memory • Load entire process in memory (swapping), run it, exit • Is slow (for big processes) • Wasteful (might not require everything) • Solutions: partial residency • Paging: only bring in pages, not all pages of process • Demand paging: bring only pages that are required • Where to fetch page from? • Have a contiguous space in disk: swap file (pagefile.sys)

7. Disk How does VM work? • Modify Page Tables with another bit (“is present”) • If page in memory, is_present = 1, else is_present = 0 • If page is in memory, translation works as before • If page is not in memory, translation causes a page fault 0 1 2 3 32 :P=1 4183 :P=0 177 :P=1 5721 :P=0 Mem Page Table

8. Page Faults • On a page fault: • OS finds a free frame, or evicts one from memory (which one?) • Want knowledge of the future? • Issues disk request to fetch data for page (what to fetch?) • Just the requested page, or more? • Block current process, context switch to new process (how?) • Process might be executing an instruction • When disk completes, set present bit to 1, and current process in ready queue

9. Steps in Handling a Page Fault

10. Resuming after a page fault • Should be able to restart the instruction • For RISC processors this is simple: • Instructions are idempotent until references are done • More complicated for CISC: • E.g. move 256 bytes from one location to another • Possible Solutions: • Ensure pages are in memory before the instruction executes

11. When to fetch? • Just before the page is used! • Need to know the future • Demand paging: • Fetch a page when it faults • Prepaging: • Get the page on fault + some of its neighbors, or • Get all pages in use last time process was swapped

12. Performance of Demand Paging • Page Fault Rate 0  p  1.0 • if p = 0 no page faults • if p = 1, every reference is a fault • Effective Access Time (EAT) EAT = (1 – p) x memory access + p (page fault overhead + swap page out + swap page in + restart overhead )

13. Demand Paging Example • Memory access time = 200 nanoseconds • Average page-fault service time = 8 milliseconds • EAT = (1 – p) x 200 + p (8 milliseconds) = (1 – p x 200 + p x 8,000,000 = 200 + p x 7,999,800 • If one access out of 1,000 causes a page fault EAT = 8.2 microseconds. This is a slowdown by a factor of 40!!

14. What to replace? • What happens if there is no free frame? • find some page in memory, but not really in use, swap it out • Page Replacement • When process has used up all frames it is allowed to use • OS must select a page to eject from memory to allow new page • The page to eject is selected using the Page Replacement Algo • Goal: Select page that minimizes future page faults

15. Page Replacement • Prevent over-allocation of memory by modifying page-fault service routine to include page replacement • Use modify (dirty) bitto reduce overhead of page transfers – only modified pages are written to disk • Page replacement completes separation between logical memory and physical memory – large virtual memory can be provided on a smaller physical memory

16. Page Replacement

17. Page Replacement Algorithms • Random: Pick any page to eject at random • Used mainly for comparison • FIFO: The page brought in earliest is evicted • Ignores usage • Suffers from “Belady’s Anomaly” • Fault rate could increase on increasing number of pages • E.g. 0 1 2 3 0 1 4 0 1 2 3 4 with frame sizes 3 and 4 • OPT: Belady’s algorithm • Select page not used for longest time • LRU: Evict page that hasn’t been used the longest • Past could be a good predictor of the future

18. toss C toss A Example: FIFO, OPT Reference stream is A B C A B D A D B C OPTIMAL ABC A B D A D B C B 5 Faults toss A or D A B C D A B C FIFO ABC A B DA D BC B 7 Faults toss ?

19. First-In-First-Out (FIFO) Algorithm • Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • 3 frames (3 pages can be in memory at a time per process): 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • 4 frames: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • Belady’s Anomaly: more frames  more page faults 1 1 4 5 2 9 page faults 2 1 3 3 3 2 4 1 1 5 4 2 10 page faults 2 1 5 3 3 2 4 4 3

20. FIFO Illustrating Belady’s Anomaly

21. Optimal Algorithm • Replace page that will not be used for longest period of time • 4 frames example 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • How do you know this? • Used for measuring how well your algorithm performs 1 4 6 page faults 2 3 4 5

22. Least Recently Used (LRU) Algorithm • Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5 • Counter implementation • Every page entry has a counter; every time page is referenced through this entry, copy the clock into the counter • When a page needs to be changed, look at the counters to determine which are to change 1 1 1 5 1 2 2 2 2 2 5 4 3 4 5 3 3 4 3 4

23. 0xffdcd: add r1,r2,r3 0xffdd0: ld r1, 0(sp) Implementing Perfect LRU • On reference: Time stamp each page • On eviction: Scan for oldest frame • Problems: • Large page lists • Timestamps are costly • Approximate LRU • LRU is already an approximation! 13 t=4 t=14 t=14 t=5 14 14

24. LRU: Clock Algorithm • Each page has a reference bit • Set on use, reset periodically by the OS • Algorithm: • FIFO + reference bit (keep pages in circular list) • Scan: if ref bit is 1, set to 0, and proceed. If ref bit is 0, stop and evict. • Problem: • Low accuracy for large memory R=1 R=1 R=0 R=0 R=1 R=0 R=1 R=1 R=1 R=0 R=0

25. R=1 R=1 R=0 R=0 R=1 R=0 R=1 R=1 R=1 R=0 R=0 LRU with large memory • Solution: Add another hand • Leading edge clears ref bits • Trailing edge evicts pages with ref bit 0 • What if angle small? • What if angle big?

26. Clock Algorithm: Discussion • Sensitive to sweeping interval • Fast: lose usage information • Slow: all pages look used • Clock: add reference bits • Could use (ref bit, modified bit) as ordered pair • Might have to scan all pages • LFU: Remove page with lowest count • No track of when the page was referenced • Use multiple bits. Shift right by 1 at regular intervals. • MFU: remove the most frequently used page • LFU and MFU do not approximate OPT well

28. evict add Page Buffering • Cute simple trick: (XP, 2K, Mach, VMS) • Keep a list of free pages • Track which page the free page corresponds to • Periodically write modified pages, and reset modified bit used free unmodified free list modified list (batch writes = speed)

29. Allocating Pages to Processes • Global replacement • Single memory pool for entire system • On page fault, evict oldest page in the system • Problem: protection • Local (per-process) replacement • Have a separate pool of pages for each process • Page fault in one process can only replace pages from its own process • Problem: might have idle resources

30. Allocation of Frames • Each process needs minimum number of pages • Example: IBM 370 – 6 pages to handle SS MOVE instruction: • instruction is 6 bytes, might span 2 pages • 2 pages to handle from • 2 pages to handle to • Two major allocation schemes • fixed allocation • priority allocation

31. Virtual Memory II: ThrashingWorking Set AlgorithmDynamic Memory Management

32. Last Time • We’ve focused on demand paging • Each process is allocated  pages of memory • As a process executes it references pages • On a miss a page fault to the O/S occurs • The O/S pages in the missing page and pages out some target page if room is needed • The CPU maintains a cache of PTEs: the TLB. • The O/S must flush the TLB before looking at page reference bits, or before context switching (because this changes the page table)

33. While “filling” memory we are likely to get a page fault on almost every reference. Usually we don’t include these events when computing the hit ratio Example: LRU, =3 Hit ratio: 9/19 = 47% Miss ratio: 10/19 = 53% R(t): Page referenced at time t. S(t): Memory state when finished doing the paging at time t. In(t): Page brought in, if any. Out(t): Page sent out. : None.

34. Goals for today • Review demand paging • What is thrashing? • Solutions to thrashing • Approach 1: Working Set • Approach 2: Page fault frequency • Dynamic Memory Management • Memory allocation and deallocation and goals • Memory allocator - impossibility result • Best fit • Simple scheme - chunking, binning, and free • Buddy-block scheme • Other issues

35. Thrashing • Def: Excessive rate of paging that occurs because processes in system require more memory • Keep throwing out page that will be referenced soon • So, they keep accessing memory that is not there • Why does it occur? • Poor locality, past != future • There is reuse, but process does not fit • Too many processes in the system

36. Approach 1: Working Set • Peter Denning, 1968 • He uses this term to denote memory locality of a program Def: pages referenced by process in last  time-units comprise its working set For our examples, we usually discuss WS in terms of , a “window” in the page reference string. But while this is easier on paper it makes less sense in practice! In real systems, the window should probably be a period of time, perhaps a second or two.

37. Working Sets • The working set size is num pages in the working set • the number of pages touched in the interval [t-Δ+1..t]. • The working set size changes with program locality. • during periods of poor locality, you reference more pages. • Within that period of time, you will have a larger working set size. • Goal: keep WS for each process in memory. • E.g. If  WSi for all i runnable processes > physical memory, then suspend a process

38. Working Set Approximation • Approximate with interval timer + a reference bit • Example:  = 10,000 • Timer interrupts after every 5000 time units • Keep in memory 2 bits for each page • Whenever a timer interrupts copy and sets the values of all reference bits to 0 • If one of the bits in memory = 1  page in working set • Why is this not completely accurate? • Cannot tell (within interval of 5000) where reference occured • Improvement = 10 bits and interrupt every 1000 time units

39. Using the Working Set • Used mainly for prepaging • Pages in working set are a good approximation • In Windows processes have a max and min WS size • At least min pages of the process are in memory • If > max pages in memory, on page fault a page is replaced • Else if memory is available, then WS is increased on page fault • The max WS can be specified by the application

40. Theoretical aside • Denning defined a policy called WSOpt • In this, the working set is computed over the next references, not the last: R(t)..R(t+-1) • He compared WS with WSOpt. • WSOpt has knowledge of the future… • …yet even though WS is a practical algorithm with no ability to see the future, we can prove that the Hit and Miss ratio is identical for the two algorithms!

41. Key insight in proof • Basic idea is to look at the paging decision made in WS at time t+-1 and compare with the decision made by WSOpt at time t • Both look at the same references… hence make same decision • Namely, WSOpt tosses out page R(t-1) if it isn’t referenced “again” in time t..t+-1 • WS running at time t+-1 tosses out page R(t-1) if it wasn’t referenced in times t...t+-1

42. How do WSOpt and WS differ? • WS maintains more pages in memory, because it needs  time “delay” to make a paging decision • In effect, it makes the same decisions, but it makes them after a time lag • Hence these pages hang around a bit longer

43. How do WS and LRU compare? • Suppose we use the same value of  • WS removes pages if they aren’t referenced and hence keeps less pages in memory • When it does page things out, it is using an LRU policy! • LRU will keep all  pages in memory, referenced or not • Thus LRU often has a lower miss rate, but needs more memory than WS

44. Approach 2: Page Fault Frequency • Thrashing viewed as poor ratio of fetch to work • PFF = page faults / instructions executed • if PFF rises above threshold, process needs more memory • not enough memory on the system? Swap out. • if PFF sinks below threshold, memory can be taken away

45. transition stable Working Sets and Page Fault Rates Working set Page fault rate

46. Dynamic Memory Management • Notice that the O/S kernel can manage memory in a fairly trivial way: • All memory allocations are in units of “pages” • And pages can be anywhere in memory… so a simple free list is the only data structure needed • But for variable-sized objects, we need a heap: • Used for all dynamic memory allocations • malloc/free in C, new/delete in C++, new/garbage collection in Java • Also, managing kernel memory • Is a very large array allocated by OS, managed by program

47. Allocation and deallocation • What happens when you call: • int *p = (int *)malloc(2500*sizeof(int)); • Allocator slices a chunk of the heap and gives it to the program • free(p); • Deallocator will put back the allocated space to a free list • Simplest implementation: • Allocation: increment pointer on every allocation • Deallocation: no-op • Problems: lots of fragmentation heap (free memory) allocation current free position

48. Memory allocation goals • Minimize space • Should not waste space, minimize fragmentation • Minimize time • As fast as possible, minimize system calls • Maximizing locality • Minimize page faults cache misses • And many more • Proven: impossible to construct “always good” memory allocator

49. 20 20 10 20 10 Memory Allocator • What allocator has to do: • Maintain free list, and grant memory to requests • Ideal: no fragmentation and no wasted time • What allocator cannot do: • Control order of memory requests and frees • A bad placement cannot be revoked • Main challenge: avoid fragmentation a b malloc(20)?

50. Impossibility Results • Optimal memory allocation is NP-complete for general computation • Given any allocation algorithm,  streams of allocation and deallocation requests that defeat the allocator and cause extreme fragmentation