UNIX: Background and the Traditional Process and Kernel Abstractions - PowerPoint PPT Presentation

UNIX:Background and the Traditional Process and Kernel Abstractions

Fred Kuhns

fredk@cse.wustl.edu

Applied Research Laboratory,

Department of Computer Science and Engineering,

Washington University in St. Louis

• Initial design: Ken Thompson, Dennis Ritchie and others at AT&T's Bell Telephone Laboratories (BTL) in 1969.
• AT&T made the source available to Universities for research and educational use.
• 1973 UNIX Version 4 is a C rewrite of previous version.
• The C language originally developed for use on the UNIX system by Dennis Ritchie
• C evolved from 'B', developed by Thompson.
• AT&T sold UNIX to Novel; Novel passed the UNIX trademark to X/OPEN and sold source code to Santa Cruz Operation (SCO).
• Plan 9 is AT&T's successor to UNIX
• AT&T unable to market UNIX as a product, source code made available to Universities for use in Research and Education.
• Influential variant: Berkeley Software Distributions (BSD) distributed by the Computer Systems Research Group, University of California at Berkeley
• Berkeley obtained UNIX from AT&T in December of 1974
• UNIX ported to many different architectures
• Microsoft and SCO collaborated to port UNIX to the Intel 8086: XENIX
• AT&T purchased 20% of Sun: result resulting in joint effort to develop SVR4
• In 1982 AT&T broken up and able to market UNIX. Released System III in 1982 and System V the following year.
• System V UNIX introduced virtual memory (different from BSD, called regions), IPC (shared memory, semaphores, message queues), remote file sharing, shared libraries and STREAMS.

CSE522– Advanced Operating Systems

• 2 BSD : text editor vi
• 3 BSD : demand-paged virtual memory
• 4.0BSD : performance improvements
• 4.1BSD : job control, autoconfiguration
• 4.2/4.3BSD : reliable signals, fast filesystem, improved networking (TCP/IP reference implementation), sophisticated IPC primitives
• 4.4 BSD : stackable and extensible vnode interface, network filesystem, log-structured filesystem, other filesystems, POSIX support, and other enhancements.

CSE522– Advanced Operating Systems

• Interactive Systems first commercial (1977)
• Microsoft and SCO release XENIX (1980)
• 1982 Bill Joy left Berkeley and founded Sun Microsystems.
• SunOS originally based on BSD 4.2
• SunOS 5 (Solaris 2.X) was a collaborative effort based on System V, release 4 (SVR4).
• AIX from IBM
• HP/UX from Hewlett Packard Corporation
• ULTRIX from Digitial Equipment Corporation, followed by DEC OSF/1. DEC purchased by Compaq

CSE522– Advanced Operating Systems

• As UNIX grew, the kernel became large and complex - originally small and elegant.
• Mid-1980 CMU researches began development of a new OS:
• microkernel providing a small set of essential services.
• Support UNIX API
• Support uniprocessor and multiprocessors
• supported distributed environments
• collection of servers provide necessary functionality
• New source so not encumbered by AT&T licenses
• OSF1 and NextStep based on MACH

CSE522– Advanced Operating Systems

• Problem: Incompatible programming APIs and core service implementations across the different UNIX variants
• Solution: Standard set of interfaces. Several standards exist that define the functions, syntax and semantics:
• IEEE POSIX (Portable Operating System Interface for computing environments or portable OS for UNIX) specifications.
• System V Interface Definition (SVID) from AT&T
• X/Open Portability Guide (XPG) from the X/Open Consortium

CSE522– Advanced Operating Systems

• IEEE POSIX specifications.
• 1003.1 (1990) system interfaces and headers
• 1003.1b (1993) realtime extensions
• 1003.1c (1995) threads extensions (pthreads)
• 1003.1 (1996) includes 1003.1 (1990), 1003.1b (1993), 1003.1c (1995), 1003.1i (1995)
• 1003.1g protocol independent interfaces
• 1003.2 (1992) Shell and utilities
• 1003.2a (1992) Interactive shell and utilities
• SVID: SVID4 is for SVR4
• defines the System V programming interface. Must be conformant to call an OS SVR5. AT&T published the System V Verification Suite (SVVS)

CSE522– Advanced Operating Systems

• XPG - formed to develop the Common Applications Environment (CAE).
• XPG3 - Superset of POSIX.1 1998 and utilities from SVID3
• XPG4 - Superset of POSIX.1 1990, POSIX.2 (1992), POSIX.2 (1992), and extension from XPG3
• XPG4v2 (SUS or single UNIX specification or spc 1170 or UNIX 95) - superset of XPG4 with widely used BSD interfaces
• XNS4 (X/Open Network Services) - sockets and XTI extensions
• SUSV2 (UNIX98)- superset of SUS, POSIX.1b (1993), POSIX.1c (1996) and ISO/IEC 9899 (C standard). Includes CDE specification
• XNS5 - superset of LP64 - clean derivative of XNS4

CSE522– Advanced Operating Systems

char

block

Device drivers

execution

environment

application

trap

libraries

user

System call interface

kernel

File subsystem

IPC

System

Services

Process

control

subsystem

Buffercache

scheduler

memory

hardware

CSE522– Advanced Operating Systems

• Two privilege levels, or modes: user and system
• System (Interrupt) versus process context
• Kernel space protected
• operates with system level privileges
• requires special instruction sequence to change from user to kernel mode
• reentrant
• Each process has own "protected" address space
• Per process state (u_area) in kernel space
• All process share same kernel space
• Per process kernel stack (since kernel is re-entrant)

CSE522– Advanced Operating Systems

Privileged

user

kernel

mode

context

process

Application

(user code)

System calls

Exceptions

system

space

kernel

X

not allowed

Interrupts

System tasks

UNIX uses only two privilege levels

CSE522– Advanced Operating Systems

• Process
• compete for system resources
• blocks if can not acquire a resource
• Kernel: Trusted software component, interfaces with hardware, implements system abstractions and interfaces
• manages resource - allocate resource to processes "optimally"
• implements policy for resource allocation (sharing)
• provide high-level abstract interface to resources
• centralized management simplifies synchronization and error recovery

CSE522– Advanced Operating Systems

• Fundamental abstraction
• Executes a sequence of instructions
• Competes for resources
• Address space - memory locations accessible to process
• virtual address space: memory, disk, swap or remote devices
• System provides features of a virtual machine
• shared resources (I/O, CPU etc)
• dedicated registers and memory
• Most created by fork or vfork
• well defined hierarchy: one parent and zero or more child processes. The init process is at the root of this tree
• different programs may be run during the life of a process by calling exec
• processes typically terminate by calling exit

CSE522– Advanced Operating Systems

fork

USER

RUNNING

INITIAL

IDLE

system call,

interrupt

return

fork

swtch()

exit

KERNEL

RUNNING

RUNNABLE

ZOMBIE

swtch()

wait

sleep()

continue

stop

wakeup

stop

STOPPED

stop

ASLEEP

continue

wakeup

STOPPED

ASLEEP

CSE522– Advanced Operating Systems

• Address Space
• text, data, stack, shared memory ...
• Control information (u area, proc)
• u area, proc structure, maps
• kernel stack
• Address translation maps
• Credentials
• user and group ids
• Environment variables
• variable=value
• typically stored at bottom of stack
• Hardware context
• program counter
• stack pointer
• processor status word
• memory management registers
• FPU registers
• Machine registers saved in u area's process control block (PCB) during context switch to another process.

CSE522– Advanced Operating Systems

Kernel stack

stack

Process address space

Data

Text (shared)

0x00000000

0xffffffff

Kernel address space

0x7fffffff

CSE522– Advanced Operating Systems

proc struct

Stack

Stack

Stack

kernel stack/u area

kernel stack/u area

kernel stack/u area

Data

Data

Data

Text (shared)

Text (shared)

Text (shared)

kernel memory

CSE522– Advanced Operating Systems

• U area.
• Part of user space (above stack).
• typically mapped to a fixed address.
• contains info needed while running.
• Can be swapped
• Proc
• contains info needed when not running.
• Not swapped out.
• traditionally fixed size table

CSE522– Advanced Operating Systems

PCB - HW context

pointer to proc

real/effective ids

args, return values or errors to current syscall.

Signal info

file descriptor table

controlling terminal vnode

Proc structure

process ID and group

u area pointer

process state

queue pointers - scheduler, sleep, etc.

Priority

memory management info

flags

CSE522– Advanced Operating Systems

• Every user is assigned a unique user id (uid) and group id (gid).
• Superuser (root) has uid == 0 and gid == 0
• every process both a real and effective pair of Ids.
• effective id => file creation and access,
• real id => real owner of process. Used when sending signals.
• Senders real or effective id must equal receivers real id

CSE522– Advanced Operating Systems

• program that runs directly on the hardware
• loaded at boot time and initializes system,
• creates some initial system processes.
• remains in memory and manages the system
• Resource manager/mediator
• Time share (time-slice) the CPU,
• coordinate access to peripherals,
• manage virtual memory.
• Synchronization primitives.
• Well defined entry points:
• syscalls, exceptions or interrupts.
• Performs privileged operations.

CSE522– Advanced Operating Systems

• Synchronous - kernel performs work on behalf of the process:
• System call interface (UNIX API): central component of the UNIX API
• Hardware exceptions - unusual action of process
• Asynchronous - kernel performs tasks that are possibly unrelated to current process.
• Hardware interrupts - devices assert hardware interrupt mechanism to notify kernel of events which require attention (I/O completion, status change, real-time clock etc)
• System processes - scheduled by OS
• swapper and pagedaemon.

CSE522– Advanced Operating Systems

• Hardware switches to kernel mode, using the per/process kernel stack.
• HW saves PC, Status word and possibly other state on kernel stack.
• Assembly routine saves any other information necessary and dispatches event.
• On completion a return from interrupt (RFI) instruction is performed.

CSE522– Advanced Operating Systems

• Asynchronous event such as disk I/O, network packet reception or clock “tick”.
• Must be serviced in system context.
• Must not block.
• This does take CPU time away from the currently running process
• Multiple interrupt priority levels (ipl) traditionally 0-7.
• User processes and kernel operate at the lowest ipl level.
• Higher level Interrupts can preempt lower priority ones.
• The current ipl level may be set while executing critical code.

CSE522– Advanced Operating Systems

• Synchronous to the currently running process for example divide by zero or invalid memory access.
• Must run the the current processes context.
• An Interrupt can occur during the processing of an exception or trap.

CSE522– Advanced Operating Systems

• Interrupts typically have the highest priority in a system.
• Software interrupts are assigned priorities above that of user processes but below that of interrupts.
• Software interrupts are typically implemented in software.
• Examples are callout queue processing and network packet processing.

CSE522– Advanced Operating Systems

• Implemented as an assembly language stub.
• Trap into kernel dispatch routine which may save additional state and invoke the high-level system call.
• User privileges are verified and any data is copied into kernel (copyin()).
• On return, copy out any user data (copyout()), check for an Asynchronous System Trap, or AST (signal, preemption, etc).

CSE522– Advanced Operating Systems

• Kernel is re-entrant.
• Only one process may be active within the kernel at any given time (others are blocked).
• Nonpreemptive.
• Blocking operations
• Masking interrupts

CSE522– Advanced Operating Systems

• When resource is unavailable (possibly locked), process sets flag and callssleep()
• sleep places process on a blocked queue, sets state to asleep and calls swtch()
• when resource released wakeup() is called
• all processes sleeping on this resource are woken: state set to runnable and placed on the runnable queue(s).
• When running, process must verify resource is available.
• Why?

CSE522– Advanced Operating Systems

• Preemptive round-robin scheduling
• fixed time quantum
• priority is adjusted by nice value and usage factor.
• the more cpu time, the lower the priority => prevents starvation
• Processes sleeping in the kernel are assigned a kernel priority which is higher than user priorities.
• priority boost may depend on the reason for sleep
• may give boost to interactive, I/O bound processes
• Kernel 0-49, user 50-127.

CSE522– Advanced Operating Systems

• Asynchronous events and exceptions
• Signal generation using the kill() system call
• Default operation or user specific handlers
• sets a bit in the pending signals mask in the proc structure
• All signals are handled before return to normal processing
• System calls can be restarted

CSE522– Advanced Operating Systems

• Fork
• create a new process
• copy of parents virtual memory
• parent return value is child’s PID
• child return value is 0
• exec
• overwrite with new program

CSE522– Advanced Operating Systems

• exit() is called
• closes open files
• releases other resources
• saves resource usage statistics and exit status in proc structure
• sends SIGCHLD signal to parent
• wakes parent
• any children are inherited by the init processes
• calls swtch()
• Process is in zombie state
• parent collects, via wait(), exit status and usage

CSE522– Advanced Operating Systems