CMSC 412 Operating Systems Fall 2002

1. CMSC 412Operating SystemsFall 2002 Liviu Iftode iftode@cs.umd.edu

2. Course overview Goals • Understand how an operating system works as a mediator between the computer architecture and user programs • Learn how OS concepts are implemented in a real operating system • Introduce to systems programming • Learn about performance evaluation • Learn about current trends in OS research

3. OS Learning project OS Implementation OS Concepts (lectures, textbooks) (source code, project doc) recitations homeworks Real OS OS Programming (man pages) (Unix/Linux textbooks)

4. Course Timeline Concept B Concept C Concept A Lecture Tu Th Tu Th Tu Th Apply B Apply A Apply C Lab W W W HW A Due HW BC Due Homework W W Project AB Due Project W

5. Suggested Approach • Read the assigned chapter from the textbook before the lecture to understand the basic idea and become familiar with the terminology • Attend the recitation • Start homeworks and project right away, systems programming is not easy ! • Ask questions during lecture, recitation. • Use the mailing list/newsgroup for discussions, do not be afraid to answer a question posted by your colleague even if you are not sure. This is a way to validate your understanding of the material. Do not forget, questions and discussions are not graded !

6. Course Outline • Processes and process management • Threads and thread programming • Synchronization and Deadlock • Memory management and Virtual Memory • CPU Scheduling • File systems and I/O Management • Networking and Distributed Systems • Security

7. Course requirements Prerequisites • computer architecture • good programming skills (C!!, C++, Java) Expected work • substantial readings (textbooks and papers) • challenging project extended over the entire semester • homeworks (require programming) • midterm and final exams

8. Work Evaluation • midterm exam 25% • homework 25% • project 25% • final exam 25%

9. Homeworks Goals • Deepen the understanding of OS concepts • Develop systems programming skills: virtual memory, threads, synchronization, sockets • Learn to design, implement, debug and evaluate the performance of an OS-bound program Structure • 4-5 homeworks • Both theoretical and C-programming problems

10. Project Goals • learn to design, implement and evaluate basic OS mechanisms and policies Structure • individual project • multiple phases • project report for each phase

11. Textbooks • Stallings Operating Systems. Internals and Design Principles, 4th Edition, Prentice-Hall, 2001. • Silberschatz, Galvin and Gagne, Operating System Concepts, 6th Edition, John Wiley & Sons, 2001. • Papers will be made available on the course homepage

12. Logistics • TAs: Chunyuan Liao Iulian Neamtiu (to be confirmed) • Course homepage: http://www.cs.umd.edu/~iftode/cs412/cs412syllabus.htm • Preliminary schedule and course notes are already available for the entire semester but they may be updated before each class. • Homeworks every other week. • A project phase every other week. • A mailing list/newsgroup will be announced shortly.

13. What is an operating system application (user) • a software layer between the hardware and the application programs/users which provides a virtual machine interface: easy and safe • a resource manager that allows programs/users to share the hardware resources: fair and efficient operating system hardware

14. How does an OS work application (user) system calls upcalls • receives requests from the application: system calls • satisfies the requests: may issue commands to hardware • handles hardware interrupts: may upcall the application • OS complexity: synchronous calls + asynchronous events hardware independent OS commands interrupts hardware dependent hardware

15. Files A file is a storage abstraction • traditional approach: OS does disk block allocation and caching (buffer cache) , disk operation scheduling and replacement in the buffer cache • new approaches: application-controlled cache replacement, log-based allocation (makes writes fast) application/user: copy file1 file2 naming, protection, operations on files operating system: files, directories operations on disk blocks... hardware: disk

16. Traditional file system application: read/write files translate file to disk blocks OS: ... ...buffer cache maintains controls disk accesses: read/write blocks hardware:

17. Mechanism vs Policy application (user) • mechanism: data structures and operations that implement the abstraction (e.g. the buffer cache) • policy: the procedure that guides the selection of a certain course of action from among alternatives (e.g. the replacement policy for the buffer cache) • traditional OS is rigid: mechanism together with policy operating system: mechanism+policy hardware

18. Mechanism-policy split • traditional OS cannot provide the best policy in all cases • new OS approaches separate mechanisms from policies: • OS provides the mechanism + some policy • applications contribute to the policy • flexibility+efficiency require new OS structures and/or new OS interfaces

19. Application-controlled caching application: read/write files replacement policy translate file to disk blocks OS: ... ...buffer cache maintains controls disk accesses: read/write blocks hardware:

20. Processes A process is a processor abstraction • traditional approach: OS switches processes on the processor (process scheduling), provides inter-process communication and handles exceptions • new approaches: application-controlled scheduling, reservation-based scheduling, agile applications user: run application create, kill processes, inter-process comm operating system: processes context switch hardware: processor

21. Traditional approach • OS mediates inter-process communication (IPC) • OS schedules processes on the processor • application provides hints: priorities active processes IPC OS processor

22. Hierarchical scheduling • OS schedules schedulers which schedule dependent processes schedulers processes OS processor

23. Virtual memory Virtual memory is a memory abstraction • traditional approach: OS provides a sufficiently large virtual address space for each running application, does memory allocation and replacement and may ensure protection • new approaches: external memory management, huge (64-bit) address space, global memory application: address space virtual addresses operating system: virtual memory physical addresses hardware: physical memory

24. VM: mechanism and policy virtual address spaces • processes can run being partially loaded in memory • illusion of more memory than physically available: swapping • processes can share physical memory if permitted • replacement policy can be exposed to the application p1 p2 processes: v-to-p memory mappings physical memory:

25. Communication Message passing is a communication abstraction • traditional approach: OS provides naming schemes, reliable transport of messages, packet routing to destination • new approaches: zero-copy protocols, active messages, memory-mapped communication application: sockets naming, messages operating system: TCP/IP protocols network packets hardware: network interface

26. Traditional OS structure • monolithic/layered systems • one/N layers all executed in “kernel-mode” • good performance but rigid user process user system calls memory system file system OS kernel hardware

27. Micro-kernel OS memory server client process file server user mode IPC micro-kernel hardware • client-server model, IPC between clients and servers • the micro-kernel provides protected communication • OS functions implemented as user-level servers • flexible but efficiency is the problem • easy to extend for distributed systems

28. Extensible OS kernel process process user mode my memory service default memory service extensible kernel hardware • user processes can load customized OS services into the kernel • good performance but protection and scalability become problems

29. Virtual Machines • old concept which is heavily revived today • the real hardware in “cloned” in several identical virtual machines • OS functionality built on top of the virtual machine user allocate resource OS on virtual machine exokernel hardware

30. Computer System • Processor: performs data processing • Main memory: stores both data and programs, typically volatile • Disks: secondary memory devices which provide persistent storage • Network interfaces: inter-machine communication • Buses: intra-machine communication • memory bus (processor-memory) • I/O bus (disks, network interfaces, other I/O devices, memory-bus)

31. Basic computer structure Memory CPU memory bus I/O bus disk Net interface

32. Memory caches • motivated by the mismatch between processor and memory speed • closer to the processor than the main memory • smaller and faster than the main memory • act as “attraction memory”: contains the value of main memory locations which were recently accessed (temporal locality) • transfer between caches and main memory is performed in units called cache blocks/lines • caches contain also the value of memory locations which are close to locations which were recently accessed (spatial locality)

33. Cache design issues • cache size and cache block size • mapping: physical/virtual caches, associativity • replacement algorithm: LRU • write policy: write through/write back cpu word transfer cache block transfer main memory

34. Memory Hierarchy • decrease cost per bit • decrease frequency of access • increase capacity • increase access time • increase size of transfer unit cpu word transfer cache block transfer main memory page transfer disks

35. Data transfer on the bus CPU • cache-memory: cache misses, write-through/write-back • memory-disk: swapping, paging, file accesses • memory-Network Interface : packet send/receive • I/O devices to the processor: interrupts Memory cache memory bus I/O bus disk Net interface

36. Direct Memory Access (DMA) • bulk data transfer between memory and an I/O device (disk, network interface) initiated by the processor • address of the I/O device • starting location in memory • number of bytes • direction of transfer (read/write from/to memory) • processor interrupted when the operation completes • bus arbitration between cache-memory and DMA transfers • memory cache must be consistent with DMA

37. Multiprocessors CPU CPU • simple scheme: more than one processor on the same bus • memory is shared among processors-- cache consistency • bus contention increases -- does not scale • alternative (non-bus) system interconnect -- expensive • single-image operating systems Memory cache cache memory bus I/O bus disk Net interface

38. Multicomputers CPU CPU • network of computers: “share-nothing” -- cheap • communication through message-passing: difficult to program • challenge: build efficient shared memory abstraction in software • each system runs its own operating system Memory Memory cache cache memory bus memory bus I/O bus I/O bus network disk Net interface disk Net interface