Unix System Programming

1. Unix System Programming Chung-Ta King Department of Computer Science National Tsing Hua University

2. Outline • UNIX system programming • IPC with shared memory • IPC with message passing

3. Layers of the Unix (system calls: entries to kernel; facilities provided by OS)

4. Processes • A process is a program in execution init swapper csh ls ps Disk getty Terminal inetd lpd Printer Ethernet

5. Low-level Process I/O • All communication of a process with outside is done by reading or writing files => a single interface • File descriptor • A non-negative integer for reference to a file • Three descriptors are created at process creation: stdin (0), stdout (1), stderr (2)all are connected to the terminal by default • More descriptors can be created with proper system calls: fd = open(“outfile”, O_WRONLY, 0644); • Descriptor table: there is limit on # of open files (20) • Related system calls: read, write, open, creat, close, unlink, lseek, dup, dup2

6. Low-level Process I/O: Example /* copy f1 to f2 */ int f1,f2,n; if ((f1=open(arg[1],O_RDONLY)) == -1) /* error if non-exist */ error(“can’t open %s”, argv[1]); if ((f2 = creat(argv[2],0644)) == -1) error(“can’t create %s”,argv[2]); while ((n = read(f1,buf,BUFSIZ)) > 0) /* return: 0 -> EOF; -1 -> error; n < BUFSIZ -> OK (read will return upto end of line) */ if (write(f2,buf,n) != n) error(“write error”, (char *) 0);

7. Process Creation • Switch to another program: • execlp("/usr/ucb/rsh","rsh",”cs20","date",0); • replaces current process image with a new image • Split a process: fork() and wait() • fork() produces two identical processes; the child process returns 0 and parent returns child pid • if (fork() == 0) execlp ("sh","sh",”-c",commandline,(char *) 0); • Shell operation: repeat get next command fork a child to run the command (fork() & execlp();) wait for the child to terminate (wait();)

8. Examine Process Status • ps -ajl F UID PID PPID PRI SIZE RSS ... STAT TTY TIME COMMAND 100 0 1 0 0 780 164 S ? 0:20 init [3] 40 0 2 1 0 0 0 SW ? 0:00 (kflushd) 40 0 3 1 -12 0 0 SW< ? 0:00 (kswapd) 40 0 4 1 0 0 0 SW ? 0:00 (nfsiod) 40 0 5 1 0 0 0 SW ? 0:00 (nfsiod) 140 0 12 1 0 756 100 S ? 0:20 /sbin/update 140 0 13 1 0 768 132 S ? 0:00 /sbin/kernel 100 0 67 1 0 772 112 S 2 0:00 (agetty) 100000 1000 28413 28410 0 1992 696 S 1 0:00 /usr/lib/X11 100 1000 28424 28419 0 0 1232 S p0 0:00 -bin/tcsh 100 519 32452 32451 9 0 1188 S p1 0:00 -tcsh 100000 519 32459 32452 12 0 852 R p1 0:00 ps -ajl

9. Processes and Descriptors fd = open("outfile",01002,0644); dup2(fd,1); /* dup fd to 1 => 1 now link to file */ if (fork() == 0) /* the child */ execlp("/usr/ucb/rsh","rsh",”cs20","date",0); else { /* the parent */ fprint(stderr,”child working …\n”); wait(&status); system("ps -ajl"); tty = open(“/dev/tty”,2); write(tty,“done!”,5); }

10. 2 0 2 0 ex1 ex1 1 3 1 open() dup2() 2 0 2 0 ex1 3 1 date 1 fork() ex1 | rsh 2 0 2 0 ex1 3 1 3 1 system() 2 0 2 0 cs20 csh ps 3 1 3 1 fork() outfile cs21

11. Signals • When an external event of concern occurs, a signal is sent to all processes that were started from the same terminal and terminates them by default • signal(): alters the default action on a signal • signal(SIGINT, SIG_IGN); • signal(SIGINT, handle_int); • signal() returns previous value of the signal and resets to default action • setjmp() and longjmp():

12. Signals: Example #include <signal.h> #include <setjmp.h> jmp_buf sjbuf; main() { if (signal(SIGINT,SIG_IGN) != SIG_IGN) signal(SIGINT,onintr); setjmp(sjbuf); /* save current stack position */ /* main loop */ } onintr() { signal(SIGINT, onintr); /* reset for next interrupt */ longjmp(sjbuf, 0); /* jump to saved state */ }

13. Signals: Alarm • alarm(): causes SIGALRM sent to process n sec later /* “timeout prog” run prog and abort it after 3600 sec */ main() { if ((pid=fork()) == 0) execvp(argv[1], &argv[1]); signal(SIGALRM, onalarm); alarm(3600); if (wait(&status) == -1 || status & 0177) != 0) error(“%s killed”,argv[1]); } onalarm() /* kill child when alarm arrives */ { kill(pid, SIGKILL); /* send pid the signal */ }

14. Outline • UNIX system programming • IPC with shared memory • IPC with message passing

15. Accessing Shared Data • Consider two processes, each of which is to add one to a shared data item, x

16. Critical Section • A mechanism for ensuring that only one process accesses a particular resource at a time is to establish sections of code involving the resource as critical sections and arrange that only one such critical section is executed at a time • The first process to reach a critical section for a particular resource enters and executes the section. • The process prevents all other processes from their critical sections for the same resource. • Once the process has finished its critical section, another process is allowed to enter a critical section for the same resource. • This mechanism is known as mutual exclusion.

17. Locks • The simplest mechanism for ensuring mutual exclusion of critical sections. • A lock is a 1-bit variable that is a 1 to indicate that a process has entered the critical section and a 0 to indicate that no process is in the critical section. • The lock operates much like that of a door lock. • A process coming to the “door” of a critical section and finding it open may enter the critical section, locking the door behind it to prevent other processes from entering • Once the process has finished the critical section, it unlocks the door and leaves.

18. Spin Lock while (lock == 1) do_nothing; /* no operation in while loop */ lock = 1; /* enter critical section */ critical section lock = 0; /* leave critical section */

19. Pthread Lock Routines • Locks are implemented in Pthreads with mutually exclusive lock variables, or “mutex” variables • A mutex must be declared as of type pthread_mutex_t and initialized, usually in the “main” thread: pthread_mutex_t mutex1; . . pthread_mutex_init(&mutex1, NULL); • NULL specifies a default attribute for the mutex. • A mutex can be destroyed with pthread_mutex_destroy()

20. Pthread Lock Routines (Cont’d) • A critical section can then be protected using pthread_mutex_lock() and pthread_mutex_unlock(): pthread_mutex_lock(&mutex1); . critical section . pthread_mutex_unlock(&mutex1); • If a thread reaches a mutex lock and finds it locked, it will wait for the lock to open. • If more than one thread is waiting for the lock to open when it opens, the system will select one thread to be allowed to proceed. • Only the thread that locks a mutex can unlock it.

21. Semaphores • A semaphore, s, is a positive integer (including zero) operated upon by two operations named P and V. • P operation, P(s): waits until s is greater than zero and then decrements s by one and allows the process to continue. • V operation, V(s): increments s by one to release one of the waiting processes (if any). • The P and V operations are performed indivisibly. • A mechanism for activating waiting processes is also implicit in the P and V operations. • Though the exact algorithm is not specified, the algorithm is expected to be fair. • Processes delayed by P(s) are kept in abeyance until released by a V(s) on the same semaphore.

22. Semaphore for Critical Sections • Use a binary semaphore, which acts as a lock variable, but P and V operations should include process scheduling • The semaphore is initialized to 1, indicating that no process is in its critical section • Each mutually exclusive critical section is preceded by a P(s) and terminated with a V(s), i.e., Process 1 Process 2 Process 3 Noncritical section Noncritical section Noncritical section . . . P(s) P(s) P(s) Critical section Critical section Critical section V(s) V(s) V(s) . . . Noncritical section Noncritical section Noncritical section

23. Semaphore for Critical Sections (Cont’d) • Any process might reach its P(s) operation first (or more than one process may reach it simultaneously). • The first process to reach its P(s) operation, or to be accepted, will set the semaphore to 0, inhibiting the other processes from proceeding past their P(s) operations • Any process reaching its P(s) operation will be recorded so that one can be selected when the critical section is released • When the process reaches its V(s) operation, it sets the semaphore s to 1 and one of the processes waiting is allowed to proceed into its critical section.

24. General Semaphore • Can take on positive values other than zero and one. • Such semaphores provide, for example, a means of recording the number of “resource units” available or used and can be used to solve producer/consumer problems. • Semaphore routines exist for UNIX processes. They do not exist in Pthreads as such, though they can be written and they do exist in the real-time extension to Pthreads • Semaphores can be used to implement most critical section applications, but they are open to human errors: • Every P must have a corresponding V=> omission of a P or V, or misnaming the semaphore ...

25. Program Examples • To sum the elements of an array, a[1000]: int sum, a[1000]; sum = 0; for (i = 0; i < 1000; i++) sum = sum + a[i];

26. Using Unix Processes • Divide the calculation into two parts: Process 1 Process 2 sum1 = 0; sum2 = 0; for (i = 0; i < 1000; i = i + 2) for (i = 1; i < 1000; i = i + 2) sum1 = sum1 + a[i]; sum2 = sum2 + a[i]; sum = sum + sum1; sum = sum + sum2; • The result location, sum, will need to be shared and access protected by a lock • Use a shared data structure:

27. The Code #include <sys/types.h> #include <sys/ipc.h> #include <sys/shm.h> #include <sys/sem.h> #include <stdio.h> #include <errno.h> #define array_size 1000 /* no of elements in shared memory */ extern char *shmat(); void P(int *s); void V(int *s);

28. int main() { int shmid, s, pid; /* shared memory, semaphore, proc id */ char *shm; /*shared mem. addr returned by shmat()*/ int *a, *addr, *sum; /* shared data variables*/ int partial_sum; /* partial sum of each process */ int i; /* initialize semaphore set */ int init_sem_value = 1; s = semget(IPC_PRIVATE, 1, (0600 | IPC_CREAT)); if (s == -1) { /* if unsuccessful*/ perror("semget"); exit(1); } if (semctl(s, 0, SETVAL, init_sem_value) < 0) { perror("semctl"); exit(1); }

29. /* create segment*/ shmid = shmget(IPC_PRIVATE,(array_size*sizeof(int)+1), (IPC_CREAT|0600)); if (shmid == -1) { perror("shmget"); exit(1); } /* map segment to process data space */ shm = shmat(shmid, NULL, 0); /* returns address as a character*/ if (shm == (char*)-1) { perror("shmat"); exit(1); } addr = (int*)shm; /* starting address */ sum = addr; /* accumulating sum */ addr++; a = addr; /* array of numbers, a[] */

30. *sum = 0; for (i=0; i < array_size; i++) /* load array with numbers */ *(a + i) = i+1; pid = fork(); /* create child process */ if (pid == 0) { /* child does this */ partial_sum = 0; for (i = 0; i < array_size; i = i + 2) partial_sum += *(a + i);} else { /* parent does this */ partial_sum = 0; for (i = 1; i < array_size; i = i + 2) partial_sum += *(a + i); } P(&s); /* for each process, add partial sum */ *sum += partial_sum; V(&s); printf("\nprocess pid=%d, partial sum=%d\n",pid,partial_sum); if (pid == 0) exit(0); else wait(0); /* terminate child proc */ printf("\nThe sum of 1 to %i is %d\n", array_size, *sum);

31. /* remove semaphore */ if (semctl(s, 0, IPC_RMID, 1) == -1) { perror("semctl"); exit(1); } /* remove shared memory */ if (shmctl(shmid, IPC_RMID, NULL) == -1) { perror("shmctl"); exit(1); } } /* end of main */ void P(int *s) /* P(s) routine*/ { struct sembuf sembuffer, *sops; sops = &sembuffer; sops->sem_num = 0; sops->sem_op = -1; sops->sem_flg = 0; if (semop(*s, sops, 1) < 0) { perror("semop"); exit(1); } return; }

32. void V(int *s) /* V(s) routine */ { struct sembuf sembuffer, *sops; sops = &sembuffer; sops->sem_num = 0; sops->sem_op = 1; sops->sem_flg = 0; if (semop(*s, sops, 1) <0) { perror("semop"); exit(1); } return; } SAMPLE OUTPUT process pid = 0, partial sum = 250000 process pid = 26127, partial sum = 250500 The sum of 1 to 1000 is 500500

33. Using Pthreads • n threads are created, each taking numbers from the list to add to their sums. When all numbers have been taken, threads add their partial results to a shared location sum. • The shared location global_index is used by each thread to select the next element of a[]. • After index is read, it is incremented in preparation for the next element to be read. • Result in sum, need to be shared and protected by a lock.

34. The Code #include <stdio.h> #include <pthread.h> #define array_size 1000 #define no_threads 10 /* shared data */ int a[array_size]; /* array of numbers to sum */ int global_index = 0; /* global index */ int sum = 0; /* final result, also used by slaves */ pthread_mutex_t mutex1; /* mutually exclusive lock variable */

35. void *slave(void *ignored) /* Slave threads */ { int local_index, partial_sum = 0; do { pthread_mutex_lock(&mutex1);/* get next index */ local_index = global_index;/* read current index & save locally*/ global_index++; /* increment global index */ pthread_mutex_unlock(&mutex1); if (local_index < array_size) partial_sum += *(a + local_index); } while (local_index < array_size); pthread_mutex_lock(&mutex1); /* add to global sum */ sum += partial_sum; pthread_mutex_unlock(&mutex1); return (); /* Thread exits */ }

36. main () { int i; pthread_t thread[10]; /* threads */ pthread_mutex_init(&mutex1,NULL); /* initialize mutex */ for (i = 0; i < array_size; i++) /* initialize a[] */ a[i] = i+1; for (i = 0; i < no_threads; i++) /* create threads */ if (pthread_create(&thread[i], NULL, slave, NULL) != 0) perror("Pthread_create fails"); for (i = 0; i < no_threads; i++) /* join threads */ if (pthread_join(thread[i], NULL) != 0) perror("Pthread_join fails"); printf("The sum of 1 to %i is %d\n", array_size, sum); } /* end of main */

37. Outline • UNIX system programming • IPC with shared memory • IPC with message passing

38. Pipes for IPC • Unidirectional byte stream communication mechanism e.g., ls | pr -2 | lpr • int sk[2]; /* sk[0]: read-end; sk[1]: write-end */ pipe(sk); /* create a pipe */ if (fork()) { /* the parent */ close(sk[1]); while(read(sk[0],buf,SIZE) > 0) printf("%s",buf); } else { /* the child */ close(sk[0]); fd=popen("ps -l","r"); while((s=read(fd,buf,SIZE)) > 0) write(sk[1],buf,s); }

39. 3 ex2 3 ex2 4 3 (a) ex2 4 3 ex2 csh fork() 0 ex2 4 ps 1 (b) (c)

40. Sockets • Endpoints for communication and for IPC references; treated like files • Socket type: stream (TCP), datagram (UDP), raw • Socket domain: • UNIX: socket name = path name • Internet: socket name = Internet addr + port #e.g., 140.114.77.100 and 1800 • struct sockaddr_in { short sin_family; /* domain name */ u_short sin_port; /* port address */ struct in_addr sin_addr; /* Internet address */ char sin_zero[8]; /* padding bytes */ };

41. Socketpair for IPC • Two-way stream communication under UNIX domain • int sk[2]; socketpair(AF_UNIX,SOCK_STREAM,0,sk); for (i=0; i<nchild; i++) if (fork() == 0) { close(sk[0]); while (read(sk[1],&num,4) > 0) { num = num*num; write(sk[1],&num,4); } exit(0); } close(sk[1]); for (i=0, pt=a; i<n; i++, pt++) write(sk[0],pt,4); for (i=0, pt=a; i<n; i++, pt++) read(sk[0],pt,4);

42. 3 ex3 4 3 (a) ex3 ex3 ex3 ex3 4 4 4 (b)

43. Datagram: Internet Domain • The sender: struct sockaddr_in remote; struct hostent *hp,*gethostbyname(); sk = socket(AF_INET,SOCK_DGRAM,0); remote.sin_family = AF_INET; /* wild-card NW addr */ hp = gethostbyname(”cs20"); /* get NW addr of cs20 */ bcopy(hp->h_addr,&remote.sin_addr.s_addr,hp->h_length); remote.sin_port = port_no; /* got from receiver */ sendto(sk,MSG,strlen(MSG),0,&remote,sizeof(remote)); read(sk,buf,BUFSIZ);

44. Datagram: Internet Domain (cont.) • The receiver: struct sockaddr_in local,remote; sk = socket(AF_INET,SOCK_DGRAM,0); local.sin_family = AF_INET; local.sin_addr.s_addr = INADDR_ANY; local.sin_port = 0; /* let system assign a port */ bind(sk,&local,sizeof(local)); getsockname(sk,&local,&len); /* get the assigned port */ printf("Port number = %d",local.sin_port); /* publish it */ recvfrom(sk,buf,BUFSIZ,0,&remote,&rlen); sendto(sk,MSG,strlen(MSG),0,&remote,sizeof(remote));

45. Virtual Circuit: Internet Domain • The server: sk = socket(AF_INET,SOCK_STREAM,0); local.sin_family = AF_INET; local.sin_addr.s_addr = INADDR_ANY; local.sin_port = 0; bind(sk,&local,sizeof(local)); listen(sk,5); /* will accept 5 connections */ while (1) { rsk = accept(sk,0,0); if (fork() == 0) { /* fork one child for one request */ dup2(rsk,0); execlp("recho","recho",0); } else close(rsk); }

46. Virtual Circuit: Internet Domain (cont.) • The child server: (the “recho” process) while (read(0,buf,BUFSIZ) > 0) printf("%s",buf); • The client: sk = socket(AF_INET,SOCK_STREAM,0); remote.sin_family = AF_INET; hp = gethostbyname(”cs20"); bcopy(hp->h_addr,&remote.sin_addr.s_addr,hp->h_length); remote.sin_port = port_no; /* got from receiver */ connect(sk,&remote,sizeof(remote)); while(read(0,buf,BUFSIZ) > 0) write(sk,buf,strlen(buf);

47. Non-blocking Receive • Receive will not block the process if no data arrived sk = socket(sk,AF_UNIX,SOCK_DGRAM,0); fcntl(sk,F_SETFL,FNDELAY); /* work on descriptor to set (F_SETFL) the status flag to non-blocking */ /* Bind socket “local” */ while(read(sk,buf,BUFSIZ) < 0) /* return immediately */ if(errno == EWOULDBLOCK) sleep(5); /* if no data arrived => sleep 5 seconds */ printf("%s",buf);

48. I/O Multiplexing • Listen to several events and respond sk = socket(AF_INET,SOCK_DGRAM,0); /* Bind sockets “local” and “remote” */ while(1) { mask = 1 << sk | 1; /* poll stdin and sk for input */ select(20,&mask,0,0,0); /* return if any one input */ if ((mask & 1) > 0) { /* stdin has input: send */ c=read(0,buf,BUFSIZ); sendto(sk,buf,c,0,&remote,rlen); } if ((mask & (1 << sk)) > 0) /* sk has input: receive */ c=recv(sk,buf,BUFSIZ,0); }

49. Broadcasting • Use datagram communication to broadcast to all hosts in a particular sub-network • The remote or destination host network address must be INADDR_ANY #define NET "140.114.77" sk = socket(AF_INET,SOCK_DGRAM,0); /* do binding for socket “local” */ remote.sin_family = AF_INET; remote.sin_addr = inet_makeaddr( inet_network(NET),INADDR_ANY); remote.sin_port = portn;