1 / 82

Chapter 11: File System Implementation

Chapter 11: File System Implementation. Chapter 11: File System Implementation. File-System Structure File-System Implementation Directory Implementation Allocation Methods Free-Space Management Efficiency and Performance Recovery Log-Structured File Systems NFS

etan
Download Presentation

Chapter 11: File System Implementation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Chapter 11: File System Implementation

  2. Chapter 11: File System Implementation • File-System Structure • File-System Implementation • Directory Implementation • Allocation Methods • Free-Space Management • Efficiency and Performance • Recovery • Log-Structured File Systems • NFS • Example: WAFL File System

  3. Objectives • To describe the details of implementing local file systems and directory structures • To describe the implementation of remote file systems • To discuss block allocation and free-block algorithms and trade-offs

  4. File-System Structure • File structure • Logical storage unit • Collection of related information • File system resides on secondary storage (disks) • File system organized into layers • File control block – storage structure consisting of information about a file

  5. Disk Organization Boot Sector Volume Directory Blk0 Blk1 … Blkk-1 Track 0, Cylinder 0 … Blkk Blkk+1 Blk2k-1 Track 0, Cylinder 1 … … Blk Blk Blk Track 1, Cylinder 0 … … Blk Blk Blk Track N-1, Cylinder 0 … … Blk Blk Blk Track N-1, Cylinder M-1

  6. Layered File System

  7. A Typical File Control Block

  8. In-Memory File System Structures • The following figure illustrates the necessary file system structures provided by the operating systems. • Figure 12-3(a) refers to opening a file. • Figure 12-3(b) refers to reading a file.

  9. In-Memory File System Structures

  10. Virtual File Systems • Virtual File Systems (VFS) provide an object-oriented way of implementing file systems. • VFS allows the same system call interface (the API) to be used for different types of file systems. • The API is to the VFS interface, rather than any specific type of file system.

  11. Schematic View of Virtual File System

  12. Chapter 11: File System Implementation • File-System Structure • File-System Implementation • Directory Implementation • Allocation Methods • Free-Space Management • Efficiency and Performance • Recovery • Log-Structured File Systems • NFS • Example: WAFL File System

  13. Directory Implementation • Linear list of file names with pointer to the data blocks. • simple to program • time-consuming to execute • Hash Table – linear list with hash data structure. • decreases directory search time • collisions – situations where two file names hash to the same location • fixed size

  14. Chapter 11: File System Implementation • File-System Structure • File-System Implementation • Directory Implementation • Allocation Methods • Free-Space Management • Efficiency and Performance • Recovery • Log-Structured File Systems • NFS • Example: WAFL File System

  15. Allocation Methods • An allocation method refers to how disk blocks are allocated for files: • Contiguous allocation • Linked allocation • Indexed allocation

  16. Contiguous Allocation • Each file occupies a set of contiguous blocks on the disk • Simple – only starting location (block #) and length (number of blocks) are required • Random access

  17. Contiguous Allocation of Disk Space

  18. Contiguous Allocation • Mapping from logical (LA) to physical Q (quotient) LA/512 R (remainder) • Block to be accessed = Q + starting address • Displacement into block = R

  19. Contiguous Allocation • Problems • Wasteful of space • dynamic storage-allocation problem: finding the best hole • tend to externally fragment the physical disk space into small sets of contiguous blocks • these sets are too little to contain most files • does not accommodate dynamic file sizes • if add to end of file and not enough physical adjacent blocks, must rewrite entire file. • can use best fit, first fit, worst fit strategies.

  20. Extent-Based Systems • Many newer file systems (I.e. Veritas File System) use a modified contiguous allocation scheme • Extent-based file systems allocate disk blocks in extents • An extent is a contiguous block of disks • Extents are allocated for file allocation • A file consists of one or more extents. • When run out of space, allocate another extent and link it to the first extent • Problems • Internal fragmentation • Could get external fragmentation if extents are different sizes.

  21. pointer block = First block … Head: 417 ... Length Length Length Byte 0 Byte 0 Byte 0 ... ... ... Byte 4095 Byte 4095 Byte 4095 Block 0 Block 1 Block N-1 Linked Allocation • Each file is a linked list of disk blocks: blocks may be scattered anywhere on the disk.

  22. Linked Allocation (Cont.) • Simple – need only starting address • Free-space management system – no waste of space • No random access • Mapping Q LA/511 R Block to be accessed is the Qth block in the linked chain of blocks representing the file. Displacement into block = R + 1

  23. Linked Allocation

  24. Linked Lists • Disadvantages • random access of bytes in the stream will be slow. • requires list traversal • each block must be read form device so the link field can be obtained and next block referenced • can use doubly linked lists to increase performance

  25. Linked Lists • Disadvantages • overhead • must store pointer to next block and length of the block • if use doubly linked lists, increases storage • Pointers take up space: if pointer is 4 bytes and block is 512 bytes, waste 0.78% of disk. • Solution: • collect blocks into multiples called clusters. • allocate clusters rather than blocks. • pointers use much smaller percentage of disk space • increase in internal fragmentation • clustersimprove disk access time for many other algorithms; used in most OS’s

  26. DOS FAT file system • File-allocation table (FAT) – disk-space allocation used by MS-DOS and OS/2. • Uses linked allocation • MS-DOS floppy disk • uses the file allocation table (FAT) file system • disk divided into • a reserved area (contains the boot program) • the file allocation tables • a root directory • file space

  27. DOS FAT file system • Space allocated for files represented by values in the allocation table • provide a linked list of all the blocks in the file • special values indicate end-of-file, unallocated blocks, bad blocks • Original FAT • no subdirectories • limited to very small disks • hard to recover the disk if the allocation tables were damaged.

  28. DOS FAT file system • Simple FAT variant • one FAT entry corresponding to each block on the disk • a file is a set of disk blocks • the FAT entry corresponding to the first entry designates the logical sector number of the second block • the FAT entry of the second block specifies the logical sector number of the third block • etc. • the FAT entry for the last block contains an end-of-file (EOF) designator • The FAT table maps the logical sector numbers to physical blocks

  29. File-Allocation Table

  30. DOS FAT file system • Simple FAT variant (cont) • the file descriptor gives the address of the first sector i • this is also the index into the FAT • can use the FAT entry to reference the next logical sector in the file. • if j is the contents of the FAT table for entry i • j is the logical sector number for the next block • j is also the index to the next FAT entry • i is the physical location of the current block

  31. DOS FAT file system • To allocate a new block: find the first 0-valued table entry • then replace the previous end-of-file value with the address of the new block • replace the 0 in the new block with the end-of-file value. • FAT can result in many disk head seeks • disk head must move to the start of the partition to read the FAT and find the location of the block in question • then move to the location of the block itself. • worst case: both moves occur for each of the blocks. • solution: cache the FAT • FAT is good for random access • disk had can find location of any block by reading the FAT.

  32. DOS FAT file system • other FAT variants • FAT varies among disk types • change number of entries in the FAT (can be 12, 16, or 32) • change number of actual tables • change size of the logical sector addressed by a FAT entry

  33. DOS FAT file system • other FAT variants: use clusters of sectors • cluster is a group of contiguous sectors • a cluster is treated as a virtual sector within the FAT. • the FAT entry addresses a cluster rather than an individual sectors • disk space is allocated to files on a cluster basis • today: • floppy disks use 12-bit FATs • hard drives use 16-bit or 32-bit FATs

  34. Indexed Allocation • Brings all pointers together into the index block. • Logical view. index table

  35. Example of Indexed Allocation

  36. Indexed Allocation (Cont.) • Need index table • Random access • Dynamic access without external fragmentation, but have overhead of index block. • Mapping from logical to physical in a file of maximum size of 256K words and block size of 512 words. We need only 1 block for index table. Q LA/512 R Q = displacement into index table R = displacement into block

  37. Indexed Allocation – Mapping (Cont.) • Mapping from logical to physical in a file of unbounded length (block size of 512 words). • Linked scheme – Link blocks of index table (no limit on size). Q1 LA / (512 x 511) R1 Q1= block of index table R1is used as follows: Q2 R1 / 512 R2 Q2 = displacement into block of index table R2 displacement into block of file:

  38. Indexed Allocation – Mapping (Cont.) • Two-level index (indexed scheme) (maximum file size is 5123) Q1 LA / (512 x 512) R1 Q1 = displacement into outer-index R1 is used as follows: Q2 R1 / 512 R2 Q2 = displacement into block of index table R2 displacement into block of file:

  39. Indexed Allocation – Mapping (Cont.)  outer-index file index table

  40. UNIX file system • UNIX file structure uses a variant of the indexed allocation scheme • external file descriptor contains pointers to 15 different storage blocks • first 12 blocks of the file are indexed directly by the first 12 pointers • the last 3 pointers used for indirect pointers • each points to an index block

  41. UNIX file system • Usual implementation • file man uses 4K blocks • the 12 direct pointers accommodate files up to 48K • if a file needs more blocks, file man allocates an index block • the 13th pointer points to this index block • the index block contains only direct pointers • if yet more blocks are needed, the 14th pointer is pointed to a double indirect block, • this is an index block, each of whose entries point to another index block • if yet more blocks are needed, the 15th pointer is pointed to a triple indirect block

  42. Combined Scheme: UNIX (4K bytes per block)

  43. UNIX files • How large can UNIX files be? • depends on size of blocks and size of disk addresses used in the system. • 12 direct blocks • blocks 0 - 11 are accessed via the direct pointers in the inode • Assume an indirect block can store 1,000 disk addresses. • Block addresses 12 - 1, 011 accessed indirectly through the single indirect block. • the 14th block pointer is double indirect. • points to a block that points to 1,000 indirect blocks • blocks 1,012 to 1,001,011 accessed via double indirect list • the 15th block pointer is triple indirect • addresses blocks 1,001,012 to 1,001,001,011

  44. UNIX files • How large can UNIX files be? • This is 4000GB. • 32-bit versions of BSD UNIX do not use triple indirect pointer • 32-bit block addresses preclude file sizes > 4GB • 64-bit block addresses need triple indirect

  45. Chapter 11: File System Implementation • File-System Structure • File-System Implementation • Directory Implementation • Allocation Methods • Free-Space Management • Efficiency and Performance • Recovery • Log-Structured File Systems • NFS • Example: WAFL File System

  46. Free-Space Management • How should unallocated blocks be managed? • Need a data structure to keep track of them • Index block • same as for conventional file, but no info in any block • initially very large; makes it impractical: index table too large • Linked list • Very large • Hard to manage spatial locality • want to allocate closely located blocks to a file • this minimizes seek time • hard to do with a linked list; have to traverse the list to find appropriate blocks

  47. Linked Free Space List on Disk

  48. Free-Space Management • Need a data structure to keep track of them • Grouping • Modification of free-list (linked-list) approach • Store the addresses of n free blocks in the first free block • n - 1 of these are actually free; last block contains pointers to the nextn free block. Etc. • Can now find a large number of free blocks quickly. • Counting • Don’t keep the address of every free block. • Free blocks are often consecutive • So keep the address of the first free block and the number of following consecutive blocks that are free. • Each entry in the free-space list consists of a disk address and a count.

  49. Free-Space Management • Bit vector (block status map or “disk map”) (n blocks) 0 1 2 n-1 … 0  block[i] free 1  block[i] occupied  bit[i] = Block number calculation (number of bits per word) * (number of 0-value words) + offset of first 1 bit Most CPUs have special bit-manipulation instructions. Example: Intel and PowerPC have instructions that return the offset in a word of the first bit with the value 1

  50. Free-Space Management (Cont.) • Bit map requires extra space • Example: block size = 212 bytes disk size = 230 bytes (1 gigabyte) n = 230/212 = 218 bits (or 32K bytes) • Easy to get contiguous files • Counting

More Related