Chapter 4 File System Implementation Part 1

1. Chapter 4 File System ImplementationPart 1 Operating System Concepts 10th Edition

2. File-System Structure • File-System Operations • Directory Implementation • Allocation Methods • Free-Space Management • Efficiency and Performance • Recovery • Example: WAFL File System

3. Objectives • Describe the details of implementing local file systems and directory structures • Discuss block allocation and free-block algorithms and trade-offs • Explore file system efficiency and performance issues • Look at recovery from file system failures • Describe the WAFL file system as a concrete example

4. 4.1 File-System Structure • File structure • Logical storage unit • Collection of related information • File systemresides on secondary storage (disks) • Provided user interface to storage, mapping logical to physical • Provides efficient and convenient access to disk by allowing data to be stored, located retrieved easily • Disk provides in-place rewrite and random access • I/O transfers performed in blocks of sectors (usually 512 bytes) • File control block (FCB)– storage structure consisting of information about a file • Device drivercontrols the physical device • To improve I/O efficiency, I/O transfers between memory and disk are performed in units of blocks. Each block has one or more sectors. Depending on the disk drive, sector size varies from 32 bytes to 4,096 bytes; the usual size is 512 bytes.

5. File system organized into layers Layered File System

6. File System Layers • Device drivers manage I/O devices at the I/O control layer • Given commands like “read drive1, cylinder 72, track 2, sector 10, into memory location 1060” outputs low-level hardware specific commands to hardware controller • Basic file system given command like “retrieve block 123” translates to device driver • Also manages memory buffers and caches (allocation, freeing, replacement) • Buffers hold data in transit • Caches hold frequently used data • File organization module understands files, logical address, and physical blocks • Translates logical block # to physical block # • Manages free space, disk allocation

7. File System Layers (Cont.) • Logical file system manages metadata information • Translates file name into file number, file handle, location by maintaining file control blocks (inodes in UNIX) • Directory management • Protection • Layering useful for reducing complexity and redundancy, but adds overhead and can decrease performanceTranslates file name into file number, file handle, location by maintaining file control blocks (inodes in UNIX) • Logical layers can be implemented by any coding method according to OS designer

8. File System Layers (Cont.) • Many file systems, sometimes many within an operating system • Each with its own format (CD-ROM is ISO 9660; Unix has UFS, FFS; Windows has FAT, FAT32, NTFS as well as floppy, CD, DVD Blu-ray, Linux has more than 130 types, with extended file system ext3 and ext4 leading; plus distributed file systems, etc.) • New ones still arriving – ZFS, GoogleFS, Oracle ASM, FUSE

9. 4.2 File-System Implementation • On-disk and in-memory structures • the file system may contain information about how to boot an operating system stored there, the total number of blocks, the number and location of free blocks, the directory structure, and individual files • Boot control blockcontains info needed by system to boot OS from that volume • Needed if volume contains OS, usually first block of volume • Volume control block (superblock, master file table)contains volume details • Total # of blocks, # of free blocks, block size, free block pointers or array • Directory structure organizes the files • Names and inode numbers, master file table

10. File-System Implementation (Cont.) • Per-file File Control Block (FCB) contains many details about the file • typically inode number, permissions, size, dates • NFTS stores into in master file table using relational DB structures

11. 4.3 Partitions and Mounting • A disk can be sliced into multiple partitions, or a volume can span multiple partitions on multiple disks. • Partition can be a volume containing a file system, or raw – just a sequence of blocks with no file system. • Boot information can be stored in a separate partition. • The bootstrap program (boot loader) is stored in the “boot blocks” at a fixed location on the disk. • This boot loader in turn knows enough about the file-system structure to be able to find and load the kernel and start it executing. It can contain more than the instructions for how to boot a specific operating system. For instance, many systems can be dual-booted, allowing us to install multiple operating systems on a single system.

12. 4.4 Directory Implementation The selection of directory-allocation and directory-management algorithms significantly affects the efficiency, performance, and reliability of the file system. In this section 4.4.1 Linear List The simplest method of implementing a directory is to use a linear list of file names with pointers to the data blocks. • Simple to program • Time-consuming to execute • Linear search time • Could keep ordered alphabetically via linked list or use B+ tree • To create a new file, we must first search the directory to be sure that no existing file has the same name. Then, we add a new entry at the end of the directory. • To delete a file, we search the directory for the named file, then release the space allocated to it. • The real disadvantage of a linear list of directory entries is that finding a file requires a linear search. • Directory information is used frequently, and users will notice if access to it is slow. In fact, many operating systems implement a software cache to store the most recently used directory information. A cache hit avoids the need to constantly reread the information from disk. • A sorted list allows a binary search and decreases the average search time.

13. 4.4.2 Hash Table • Hash Table – linear list with hash data structure • The hash table takes a value computed from the file name and returns a pointer to the file name in the linear list. Therefore, it can greatly decrease the directory search time. Insertion and deletion are also fairly straightforward. • The major difficulties with a hash table are its generally fixed size and the dependence of the hash function on that size. For example, assume that we make a linear hash table that holds 64 entries. The hash function converts file names into integers from 0 to 63, for instance, by using the remainder of a division by 64. If we later try to create a 65th file, we must enlarge the directory hash table—say, to 128 entries. As a result, we need a new hash function that must map file names to the range 0 to 127, and we must reorganize the existing directory entries to reflect their new hash-function values. • Decreases directory search time • Collisions– situations where two file names hash to the same location • Only good if entries are fixed size, or use chained-overflow method

14. 4.5 Allocation Methods • 4.5.1 Contiguous allocation – each file occupies set of contiguous blocks • Best performance in most cases • Simple – only starting location (block #) and length (number of blocks) are required • Contiguous allocation has some problems, however. • - One difficulty is finding space for a new file. • - Another problem with contiguous allocation is determining how much space is needed for a file. • An allocation method refers to how disk blocks are allocated for files:

15. First fit. Allocate the first hole that is big enough. Searching can start either at the beginning of the set of holes or at the location where the previous first-fit search ended. We can stop searching as soon as we find a free hole that is large enough. • Best fit. Allocate the smallest hole that is big enough. We must search the entire list, unless the list is ordered by size. This strategy produces the smallest leftover hole. • Worst fit. Allocate the largest hole. Again, we must search the entire list, unless it is sorted by size. This strategy produces the largest leftover hole, which may be more useful than the smaller leftover hole from a best-fit approach.

16. Contiguous Allocation Q • Mapping from logical to physical LA/512 R Block to be accessed = Q + starting address Displacement into block = R

17. 4.5.2 Linked Allocation Method • Linked allocation – each file a linked list of blocks • File ends at nil pointer • No external fragmentation • Each block contains pointer to next block • No compaction, external fragmentation • Free space management system called when new block needed • Improve efficiency by clustering blocks into groups but increases internal fragmentation • Reliability can be a problem • Locating a block can take many I/Os and disk seeks

18. Linked Allocation

19. An important variation on linked allocation is the use of a file-allocation table (FAT). • A section of the disk at the beginning of each volume is set aside to contain the table. The table has one entry for each disk block and is indexed by block number. The FAT is used in much the same way as a linked list. The directory entry contains the block number of the first block of the file. The table entry indexed by that block number contains the block number of the next block in the file. This chain continues until the last block, which has a special end-of-file value as the table entry An illustrative example is the FAT structure for a file consisting of disk blocks 217, 618, and 339.

20. End of Part 1