Advanced Programming in the Unix Environment

1. Advanced Programming in the Unix Environment Ch 3. File I/O System Programming

2. Contents • Unix File I/O basic features • Unix File I/O functions • open, creat, close, lseek, read, write • File Sharing and Kernel Data Structures • Atomic operations • Other File I/O functions • dup, fnctl, ioctl System Programming

3. Introduction • Files in Unix • Simple, unstructured sequences of bytes or characters. • Unbuffered I/O • Each read and write invokes a system call in the kernel Ref) Standard I/O library - buffered I/O • File Descriptors • All open files are referred to by file descriptors. • Kernel returns a file descriptors when open or create a file. • Default file descriptors assigned by Unix Shells • 0 : standard input (STDIN_FILENO) • 1 : standard output (STDOUT_FILENO) • 2 : standard error (STDERR_FILENO) System Programming

4. File I/O functions • Open and Close functions • open, creat, close • Position functions • lseek • Read/Write functions • read, write • Duplicating functions • dup, dup2 • Property control • fcntl • Misc • ioctl System Programming

5. Opening and Closing a File int open(const char *pathname,int oflag,.../* mode_t mode */); returns : file descriptor if OK, -1 on error int creat(const char *pathname, mode_t mode); returns : file descriptor opened for write-only if OK, -1 on error int close(int fildes); returns : 0 if OK, -1 on error • open(): • opens a file specified by pathname with oflag and mode (file access permissions) • returns a file descriptor which is the lowest numbered unused descriptor System Programming

6. Opening and Closing a File (cont’d) • Oflag : • required option flags: O_RDONLY, O_WRONLY, O_RDWR • optional flags: O_APPEND, O_CREAT, ….., O_SYNC • O_APPEND : Append to the end of file on each write • O_CREAT : create a file if not exists; needs the third argument mode • If the file exists, no effect (w/o O_EXCL) • O_EXCL: Generate error if used with O_CREAT and the file exists • The test and creation of a file is an atomic operation • O_TRUNC: if the file exists and opened for either write-only or read- write, truncate its length to 0 • O_SYNC: Each write wait for physical I/O to complete • O_NONBLOCK, O_NOCTTY, .. • ex) open(“/home/kim/afile”, (O_RDWR | O_CREAT | O_TRUNC), 0644) System Programming

7. Opening and Closing a File (cont’d) • creat() : • equivalent to open(pathname, O_WRONLY | O_CREAT |O_TRUNC, mode) • file opened only for writing * if we want to creat a file for writing and then reading it, use open(pathname, O_RDWR | O_CREAT |O_TRUNC, mode) • close() : close a file • When a processor terminates, all open files are automatically closed System Programming

8. Positioning a File Off_t lseek(int filedes, off_t offset, int whence); returns : new file offset if OK, -1 on error • lseek sets the file offset. • whence • SEEK_SET : the file offset is set from the beginning of file • SEEK_CUR : the file offset is set to offset plus the current file position • SEEK_END : the file offset is set to offset plus the endof file position. • offset can be positive or negative • current file offset : measures the number of bytes from the beginning of the file (nonnegative number) • read() and write() start at the current file offset and cause the offset to be incremented • The offset is initialized to 0 when a file opened w/o O_APPEND System Programming

9. lseek cont’d • DOES NOT cause any I/O to take place; only record the offset within the kernel • current_position = lseek(fd, 0, SEEK_CUR); • to determine the current offset • to determine if the referenced file is capable of seeking (pipe or FIFO returns -1) • The file offset can be greater than the file’s current size and the next write will extend the file (creating a hole in the file) System Programming

10. Reading and Writing a File ssize_t read(int filedes, void *buff, size_t nbytes); returns : number of bytes read, 0 if end of file, -1 on error ssize_t write(int filedes, const void *buff, size_t nbytes); returns : number of bytes written if OK, -1 on error • read() : reads data up to nbytes from a file and store into the buffer. • read() may not read nbytes because (of) • end of file is reached • reading from a terminal device, normally up to one line is read • reading from a network, buffering within the network • Some record-oriented devices (magnetic tape) return up to a single record at a time System Programming

11. Reading and Writing a File (cont’d) ssize_t read(int filedes, void *buff, size_t nbytes); returns : number of bytes read, 0 if end of file, -1 on error ssize_t write(int filedes, const void *buff, size_t nbytes); returns : number of bytes written if OK, -1 on error • write(): writes nbytes data from the buffer to a file • write may not write nbytes data due to write errors caused by full disk or exceeding the file size limit System Programming

12. File Sharing in UNIX • Kernel data structures for file sharing by processes • A File descriptor table within a process table entry (or per process) • Each file descriptor with the file descriptor flags and a pointer to a filetable entry • A File table for all open files in the kernel • thefile status flags for the file (read, write, append, sync …) • thecurrent file offset • a pointer to the v-node table entry for the file • V-node table for open files • a v-node (i-node) structure for each open file • v-node : type of the file, pointers to file operation functions, i-node • i-node : owner of the file, the size of thefile, the device, pointers to data block,.. etc. System Programming

13. ... Kernel data Structures for open files V-node table v-node information i-node information current file size File table Process table entry file status flag current file offset v-node ptr fd flag ptr fd 0 : fd 1: fd 2: v-node information i-node information current file size file status flag current file offset v-node ptr System Programming

14. Process table entry fd flag ptr fd 0 : fd 1: fd 2: ... ... Processes sharing a open file proc A proc B fda = open(“/etc/passwd”, O_RDONLY); fdb = open(“/etc/passwd”, O_RDONLY); File table V-node table file status flag current file offset v-node ptr v-node information i-node information current file size file status flag current file offset v-node ptr fd flag ptr fd 0 : fd 1: fd 2: System Programming

15. What really happens? • After each write complete • the current file offset incremented by the number of bytes written. • if the current file offset > the current size, the current file size is set to the current file offset • Each write with O_APPEND • O_APPEND flag set in the file status flags • the current file offset is set to the current file size from i-node before every write • this forces every write to be appended to the current end of file • lseek() - only modifies the current file offset (no I/O) • When lseek() positions a file to the end of file • the current file offset is set to the current file size (no I/O) System Programming

16. Atomic Operations • An operation that cannot be separated, that is, indivisible • What can happen if two processes perform the following code on the same file • Appending to a File • Create a File /* A old way without O_APPEND option */ fd = open(“afile”, O_WRONLY); lseek(fd, 0L, SEEK_END); /* position to EOF */ write(fd, buff, 100); /* and write */ /* A old way without O_CREAT and O_EXCL options */ if ( (fd = open(“afile”, O_WRONLY)) < 0) if (errno == ENOENT) /* no such file or dir */ creat(“afile”, 0644); /* create a file even if already exists */ System Programming

17. Atomic Operations (continued) • New ways with options • Appending to a file • Create a file fd = open(“afile”, O_WRONLY|O_APPEND); write(fd, buff, 100); open("afile", O_WRONLY|O_CREAT, 0644); System Programming

18. Duplicating a File Descriptor int dup(int filedes); int dup2(int filedes, int filedes2) Both return : new file descriptor if OK, -1 on error • Duplicate an existing file descriptor • dup() returns the lowest numbered available file descriptor for the new file descriptor • Ex) dup(5) copies file descriptor 5 into the lowest numbered available file descriptor. • dup2 copies filedes to filedes2 • dup2 closes filedes2 if already opened before the duplication • dup2 is an atomic operation System Programming

19. ... File table file status flag current file offset v-node ptr Duplicating a File Descriptor /* connect stdout to a file */ fd=open(“file”,O_WRONLY);close(1); dup(fd); /* or */ fd=open(“file”,O_WRONLY); dup2(fd,1) /* I/O redirection: shell example */ $ a.out > file Process table entry V-node table v-node information i-node information current file size fd flag ptr fd 0 : fd 1: fd 2: System Programming