Operating Systems (part of CSc213/4)

1. Operating Systems (part of CSc213/4) Geoff Coulson Distributed Multimedia Research Group, Department of Computing, Lancaster University geoff@comp.lancs.ac.uk

2. Unit 1: Introduction to operating systems • aims • to define/ characterise operating systems • to classify operating systems into types • to present a brief history of operating systems • to outline the rest of the course

3. What an operating system is and isn’t • the OS is • the software that manages system resources • the software that provides a convenient virtual machine for user programs • the OS is not the window system • the OS is not programs like ls, dir, cd, copy, ... • the OS is not the prompt you run programs from • the OS is not languages and compilers etc.

4. Virtual machine provision • the OS is the software that provides a convenient virtual machine for user programs • virtual machine defined by the syntax and semantics of the system call interface (later...) • processor + interrupts -> threads • memory + backing store -> virtual memory • theads + virtual memory -> processes • devices -> virtual devices • backing store -> file system

5. System resource management • the OS is the software that manages system resources • e.g., CPU, memory, IO channels, discs, devices, networks, files, sockets, buffers... • allocate resources to processes • avoid problems due to contention • maximise use of resources

6. Types of operating system 1 • batch systems • jobs submitted to input queue (e.g. via JCL script) • one job selected at a time and processed to completion • no user interaction • single-user interactive systems • user issues commands • operating system obeys commands and returns control to user • ignores supervisor protection (see later) - user has direct access to full machine • e.g. CP/M, MS-DOS

7. Types of operating system 2 • single-user interactive multiprogramming systems • like above but user can run multiple progs simultaneously • rely on co-operation between multiple programs (processes) • Macintosh finder, Windows 95 etc. • multiprogramming/ timesharing systems • terminal access provided to many users • many commands in execution at the same time • resources shared between activities • stronger security/ protection than single user systems • now tending to be used as single user systems • VME, UNIX, Windows NT etc.

8. Types of operating system 3 • distributed operating systems • offer unified interface to co-ordinate resources of a loosely coupled network of computers • not treated in this course - see 3rd year option • other types of operating system (also not this course) • real time systems • embedded systems • LISP machines • ...

9. Fundamental hardware concepts 1 • processor(s) • user mode for user programs • supervisor mode with privileged instructions for OS • processor enters supervisor mode as a result of the execution of a trap or an interrupt • both trap and interrupt cause execution to jump to a specific memory location that (presumably!) holds OS code • there is also an instruction to make the processor revert from supervisor mode to user mode

10. Fundamental hardware concepts 2 • memory • supervisor memory for OS and user memory for user programs and data • user memory is divided up between processes • hardware support is required • virtual to physical memory translation (translation lookaside buffer) • page and segmentation tables • IO devices • usually only accessible in supervisor mode

11. A brief history of operating systems: late 40s • bare machines programmed in binary by expert users • programs entered using toggles on console • user could stop machine at any time and see the current register contents in lamps on the console • user could dump the contents of memory to a printer to aid debugging

12. A brief history of operating systems: around 1950 • professional computer operators introduced • economic motivation : the operating costs of a computer were orders of magnitude higher than the salary of an operator and the operator could make more productive use of the machine • standard libraries of code for IO device access were beginning to be used to allow programmers to access IO devices • assembler languages introduced

13. A brief history of operating systems: early 50s • first monitor programs - automated program loading and resident device drivers • automated loading helped operator to optimally order jobs - e.g. load an assembler and assemble serveral user programs before loading some other job • resident device drivers meant it was not necessary to load the device driver code with each new program

14. A brief history of operating systems: late 50s • invention of interrupts to allow asynchronous IO • arose from recognition that expensive CPUs were standing idle when data was being loaded and results printed • allowed the CPU to be performing useful computation (in the same program) while waiting for IO • high level languages • FORTRAN and LISP were designed around this time

15. A brief history of operating systems: early 60s • fence registers to protect monitor code • register containing the address of the boundary between user and monitor memories • user and supervisor modes • allowed the execution of privileged instructions by the monitor • direct memory access (DMA) • an IO device could semi-autonomously transfer data to/ from memory without each byte having to be copied explicitly by the CPU

16. A brief history of operating systems: mid 60s • batch mode multiprogramming • allowed multiple programs (processes) to be in memory simultaneously • while one process was waiting for IO the monitor (already an OS now actually!) arranged to run another process • virtual memory • each process saw its own linear memory space (0..N); this removed the need for multiple programs to be placed in an area of memory fixed at compile time • the one level store concept enabled the sum of virtual address spaces of all processes to be larger than the size of physical memory

17. A brief history of operating systems: 70s, 80s, 90s ~1970: interactive timesharing systems • interactive terminal as well as batch mode operation • main focus of this course ~1980: networked and distributed systems; ‘single user’ multiprogrammed workstations 1990+: object oriented operating systems 1990+: multimedia capable systems 1990+: extensible operating systems

18. Evaluating the operating system 1 • performance • throughput (system manager) • turnaround (batch user) • response time (interactive user) • fairness • real-time contraints • etc.

19. Evaluating the operating system 2 • reliability • mean time between failures • mean time to repair • maintainability • easy to debug • easy to extend • easy to port

20. Summary • operating systems manage resources and provide virtual machine abstractions • operating systems have evolved over a long time period and are still evolving • multiprogramming systems are ‘real’ OSs • good resource utilisation, protection, cooperation, ... • rapid switching of resources • operating systems can be evaluated according to a number of criteria

21. Overview of course 1 • unit 1: Introduction • unit 2: OS structuring • layered structuring • microkernel structuring • introduction to TUNIX • unit 3: File management • design issues • UNIX case study

22. Overview of course 2 • unit 4: File management (cont.) • buffering techniques • the UNIX buffer pool manager • unit 5: Device drivers • block and character devices • structure of UNIX device drivers • unit 6: Memory management • limitations of physical memory management • simple virtual memory techniques

23. Overview of course 3 • unit 7: Memory management (cont.) • paging and segmentation • policy in paging systems • unit 8: Scheduling and threads • scheduling policies • a thread implementation • unit 9: Process management • process management issues • processes in UNIX

24. Overview of course 4 • unit 10: OSs for multiprocessors • characteristics of MPs • styles of MP OS • unit 11: Extensible OS architecture • Mike Clarke • unit 12: Questions and answers

25. Books • You shouldn’t need to buy one but the following are worth a look: • Switzer, R., "Operating Systems: A Practical Approach", Prentice Hall, 1993, ISBN 0-13-640152-X • Silberschatz, Peterson and Galvin, "Operating System Concepts", Addison Wesley, ISBN 0-201-54873-9 • Tanenbaum, A.S., "Operating Systems, Design and Implementation", Prentice Hall International Editions, ISBN 0-13-637331-3

26. Unit 2: OS structure • aims • to discuss the major components of an operating system and how these are structurally related • to introduce the micro-kernel operating system architecture

27. OS interfaces • two interfaces • upper interface to user software (traps) • hardware interface to machine services (interrupts and writes to hardware registers) • OS must deal with a range of 'asynchronous' events from above and below - perhaps 10,000+ per second

28. The upper interface (user programs) • programmer's interface is through system calls • these are implemented with the CPU’s trap instruction • executing a system call • put system call number in register • put pointer to arguments in another register • issue trap - causes processer to switch to supervisor mode and jump to a system call manager in private OS memory • arguments are copied to OS memory and validated • system call manager decodes system call number and calls appropriate OS routine • on completion, copy results to user memory, switch back to user mode and resume user code • often wrapped in library code (read(), write()...)

29. Executing in the OS • in traditional UNIX, system calls are invoked ‘procedurally’ • processes executing in the OS are non preemptible • they can, of course, yield voluntarily • use sleep()/ wakeup() internally for condition synchronisation • more recent OSs are multithreaded • multiple threads can execute in the OS simultaneously • they synchronise using semaphores • allows more concurrency but more complex to code

30. The lower interface (hardware) • the OS interfaces to hardware as well as user progs • asks for services from hardware • implemented by writes to hardware device registers • "start a direct memory access (DMA) transfer to disc" • "read the keyboard" • receives asynchronous interrupts from hardware • "this DMA operation has completed" • "someone pressed the keyboard or mouse" • "the clock ticked"

31. Interrupt (and trap) handling • interrupts notify OS of asynchronous events generated by IO devices • each IO device has an associated memory location called an interrupt vector which holds the address of an interrupt service routine • when the device interrupts, the hardware... • moves to supervisor mode • saves the PC and processor status word on stack • jumps to the address specified in the interrupt vector • executes the interrupt service routine

32. The interrupt service routine... ... then saves the rest of the processor registers in storage space associated with the current process • makes the stack pointer point at the interrupt handler stack • takes device specific action (e.g. a clock interrupt will modify an internal variable representing the current time) • enters the scheduler to determine the process to run next; resume chosen process in user mode

33. The structure of operating systems • OSes are large, complex beasts! • there are massive software engineering problems in the design and implementation of such systems by teams • IBM's experiences with huge armies of programmers on OS/360 in 1960's • many current UNIX implementations are still poorly structured • the search for modularisation...

34. The layering principle • design OSs as a number of layers of abstraction • each layer adds further functionality to the system • interfaces between layers are via procedure calls

35. Example layering (inner to outer) • machine dependent “nucleus” • implementation of basic inter-process synchronisation • machine dependent context switch • first level interrupt handler • memory management (page tables, paging h/w, ...) • IO (relies on buffers provided by memory system) • file access (relies on IO and buffers) • scheduling and resource allocation (builds concept of process - a schedulable entity that ‘owns’ resources such as memory, CPU cycles, open files, ...) • user programs

36. The micro-kernel approach • favoured by many modern day operating system implementations • also called server based architecture • not UNIX! • although micro-kernel implementations of UNIX are appearing • minimal “micro-kernel” (cf. nucleus) • support the passing of messages between modules • accept and dispatch interrupts (via messages) • do machine dependent context switch • rest of system implemented as independent servers

37. Servers in a micro-kernel architecture • examples • file system, memory manager, process manager, device drivers, comms protocols, ... • sit ‘side by side’ rather than in a strict hierarchical relationship • each has a private address space • each communicates with the micro-kernel (and/ or each other) across well-defined published interfaces via messages - not procedure calls

38. An example message passing micro-kernel-based OS • TUNIX (Switzer book) • modules comprise • file manager (FM) • process menager (PM) • memory manager (MM) • IPC manager memory mgr IPC mgr device driver process mgr file mgr user micro-kernel

39. Advantages of micro-kernels • portability • simply re-implement the micro-kernel to port • flexibility • can support multiple OS ‘personalities’ (even on same machine!) • reliability • hardware dependent parts are small • other modules can be tested in isolation • reduced concurrency problems in modules - interrupts are delivered synchronously with other messages

40. Disadvantage of micro-kernels • performance! • simple micro-kernel will have minimal overheads • but message passing overhead? • personalities on top of microkernel?

41. Summary • OSs have system call and hardware interfaces • early OSs were structured in an ad-hoc way • onion skin layering was an early appoach to structuring • the micro-kernel approach is now popular • minimal micro-kernel for basic hardware abstraction • higher level system services as isolated, machine independent modules • encourages correctness, portability • but many OSes in current use are implemented with no particular structure at all!

42. Unit 3: File systems • aims • to explore the issues involved in designing a file system • to present details of the MS-DOS and UNIX file systems as a case study

43. Context of file management • we start with the file system as it has (almost) the fewest dependencies memory mgr IPC mgr device driver process mgr file mgr user

44. Basic concepts • we abstract the physical disc to a sequence of physical blocks, 0 to N-1, of size B bytes • we consider files to be composed of logical blocks of size B logical physical

45. Basic design issue: internal file structure • should files be internally structured (e.g. as B-trees of records)? • cf. OS/360, VMS • “file operations can be implemented efficiently in OS” • “the programmer is saved much effort” ...or simply as byte sequences? • “we can never provide exactly the right abstraction for all users” • “so we just provide a simple low level implementation and focus exclusively on implementing that efficiently”

46. Basic design issue: logical->physical mapping • choice of • contiguous mapping • file is found in physical blocks n, n+1, ..., n+k-1 • non contiguous mapping • consecutive logical blocks need not be mapped to consecutive physical blocks contiguous non contiguous

47. Contiguous mapping • advantages • simple bookkeeping • only need to remember <start disc address, number of blocks> • to find b’th byte in a file only need look in logical block (int)b/B and then retrieve physical block (int)b/B + b%B • sequential reading of file is simple and efficient • having read block n, the disc head already over block n+1 • actually only an apparent advantage in multiprogrammed OS!! • disadvantages • to create a k block file we must find k contiguous free blocks and to to grow a file we may need to copy the whole file to a contiguous set of blocks elsewhere on disc • disc fragmentation

48. Non contiguous mapping • advantages • to create a file of k blocks we can use anyk physical blocks • a file can always grow as long as there exists a free block • no fragmentation • disadvantages • more complicated bookkeeping • we need some data structure to store the mapping • sequential file reading causes disc head to jump a lot -- but this happens anyway in a multiprogrammed system as we’ve seen • seems the best option - as long as we can design an appropriate data structure for the mapping...

49. Non contiguous mappings • the MS-DOS approach • the UNIX approach

50. The MS-DOS approach • uses a file allocation table (FAT) • an array with as many entries as there are physical blocks on the disc • stored on disc but read into RAM at boot time • each file f = n1, n2,..., nk is identified by n1, the number of the first physical block in the file • FAT[n1] contains block number n2, • FAT[n2] contains block number n3, • ... • FAT[nk]has special EOF marker