file system interface and implementations
Download
Skip this Video
Download Presentation
File System Interface and Implementations

Loading in 2 Seconds...

play fullscreen
1 / 40

File System Interface and Implementations - PowerPoint PPT Presentation


  • 134 Views
  • Uploaded on

File System Interface and Implementations. Fred Kuhns CS523 – Operating Systems. FS Framework in UNIX. Provides persistent storage Facilities for managing data file - abstraction for data container, supports sequential and random access

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'File System Interface and Implementations' - betty_james


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
file system interface and implementations

File System Interface and Implementations

Fred Kuhns

CS523 – Operating Systems

CS523S: Operating Systems

fs framework in unix
FS Framework in UNIX
  • Provides persistent storage
  • Facilities for managing data
    • file - abstraction for data container, supports sequential and random access
    • file system - permits organizing, manipulating and accessing files
  • User interface specifies behavior and semantics of relevant system calls
    • Interface exported abstractions: files, directories, file descriptorsand differentfile systems

CS523S: Operating Systems

kernel files and directories
Kernel, Files and Directories
  • kernel provides control operations to name, organize and control access to files but it does not interpret contents
  • Running programs have an associated current working directory. Permits use of relative pathnames. Otherwise complete pathnames are required.
  • File viewed as a collection of bytes
    • Applications requiring more structure must define and implement themselves

CS523S: Operating Systems

kernel files and directories1
Kernel, Files and Directories
  • files and directories form hierarchical tree structure name space.
    • tree forms a directed acyclic graph
  • Directory entry for a file is known as a hard link.
    • Files may also have symbolic links
  • File may have one or more links
  • POSIX defines library routines {opendir(), readdir(), rewinddir(), closedir()}

struct dirent {

ino_t d_ino;

char d_name[NAME_MAX + 1];

}

CS523S: Operating Systems

file and directory organization
File and Directory Organization

/

(hard) links

bin

etc

dev

usr

vmunix

sh

local

etc

/usr/local/bin/bash

bin

bash

CS523S: Operating Systems

file attributes
File Attributes
  • Type – directory, regular file, FIFO, symbolic link, special.
  • Reference count – number of hard links {link(), unlink()}
  • size in bytes
  • device id – device files resides on
  • inode number - one inode per file, inodes are unique within a disk partition (device id)
  • ownership - user and group id {chown()}
  • access modes - Permissions and modes {chmod()}
    • {read, write execute} for {owner, group or other}
  • timestamps – three different timestamps: last access, last modify, last attributes modified. {utime()}

CS523S: Operating Systems

permissions and modes
Permissions and Modes
  • Three Mode Flags = {suid, sgid and sticky}
    • suid –
      • File: if set and executable then set the user’s effective user id
      • Directory: Not used
    • sgid –
      • File: if set and executable then set the effective group id. If sgid is set but not executable then mandatory file/record locking
      • Directory: if set then new files inherit group of directory otherwise group or creator.
    • sticky –
      • File: if set and executable file then keep copy of program in swap area.
      • Directory: if set and directory writable then remove/rename if EUID = owner of file/directory or if process has write permission for file. Otherwise any process with write permission to directory may remove or rename.

CS523S: Operating Systems

user view of files
User View of Files
  • File Descriptors (open, dup, dup2, fork)
    • All I/O is through file descriptors
    • references the open file object
    • per process object
    • file descriptors may be dup’ed {dup(), dup2()}, copied on fork {fork()} or passed to unrelated process {(see ioctl() or sendmsg(), recvmsg()}permitting multiple descriptors to reference one object.
  • File Object - holds context
    • created by an open() system call
    • stores file offset
    • reference to vnode
  • vnode - abstract representation of a file

CS523S: Operating Systems

how it works
How it works

fd = open(path, oflag, mode);

lseek(), read(), write() affect offset

File Descriptors

{{0, uf_ofile}

{1, uf_ofile}

{2 , uf_ofile}

{3 , uf_ofile}

{4 , uf_ofile}

{5 , uf_ofile}}

Open File Objects

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...}}

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

CS523S: Operating Systems

file systems
File Systems
  • File hierarchy composed of one or more File Systems
  • One File System is designated the Root File System
  • Attached to mount points
  • File can not span multiple File Systems
  • Resides on one logical disk

CS523S: Operating Systems

logical disks
Logical Disks
  • Viewed as linear sequence of fixed sized, randomly accessible blocks.
    • device driver maps FS blocks to underlying storage device.
    • created using newfs or mkfs utilities
  • A file system must reside in a logical disk, however a logical disk need not contain a file system (for example the swap device).
  • Typically logical disk corresponds to partion of a physical disk. However, logical disk may:
    • map to multiple physical disks
    • be mirrored on several physical disks
    • striped across multiple disks or other RAID techniques.

CS523S: Operating Systems

file abstraction
File Abstraction
  • Abstracts different types of I/O objects
    • for example directories, symbolic links, disks, terminals, printers, and pseudodevices (memory, pipes sockets etc).
  • Control interface includes fstat, ioctl, fcntl
  • Symbolic links: file contains a pathname to the linked file/directory. {lstat(), symlink(), readlink()}
  • Pipe and FIFO files:
    • FIFO created using mknod(), lives in the file system name space
    • Pipe created using pipe(), persists as long as opened for reading or writing.

CS523S: Operating Systems

oo style interfaces
OO Style Interfaces

Instance of derived class

Abstract base class

Struct interface_t

{

// Common functions:

open (), close ()

// Common data:

type, count

// Pure virtual functions

*ops (Null pointer)

// Private data

*data (Null pointer)

}

Struct interface_t

{

open (), close ()

type, count

*ops

*data

}

{my_read()

my_write()

my_init()

my_open()

… }

{device_no,

free_list,

lock, …}

CS523S: Operating Systems

overview
Overview

System calls

vnode interface

/proc

tmpfs

UFS

HSFS

PCFS

RFS

NFS

swapfs

cdrom

diskette

disk

Process

address

space

Anonymous

memory

Example from Solaris

CS523S: Operating Systems

vfs vnode framework
Vfs/Vnode Framework
  • Concurrently support multiple file system types
  • transparent interoperation of different file systems within one file hierarchy
    • enable file sharing over network
    • abstract interface allowing easy integration of new file systems by vendors

CS523S: Operating Systems

objectives
Objectives
  • Operation performed on behalf of current process
  • Support serialized access, I.e. locking
  • must be stateless
  • must be reentrant
  • encourage use of global resources (cache, buffer)
  • support client server architectures
  • use dynamic storage allocation

CS523S: Operating Systems

vnode vfs interface
Vnode/vfs interface
  • Define abstract interfaces
  • vfs: Fundamental abstraction representing a file system to the kernel
    • Contains pointerss to file system (vfs) dependent operations such as mount, unmount.
  • vnode: Fundamental abstraction representing a file in the kernel
    • defines interface to the file, pointer to file system specific routines. Reference counted.
    • accessed in two ways:
      • 1) I/O related system calls
      • 2) pathname traversal

CS523S: Operating Systems

vfs overview
vfs Overview

Struct vfsops {

*vfs_mount,

*vfs_root, …}

Struct vfsops {

*vfs_mount,

*vfs_root, …}

rootvfs

private data

private data

Struct vfs {

*vfs_next,

*vfs_vnodecovered,

*vfs_ops,

*vfs_data, …}

Struct vfs {

*vfs_next,

*vfs_vnodecovered,

*vfs_ops,

*vfs_data, …}

Struct vnode {

*v_vfsp,

*v_vfsmountedhere,…}

Struct vnode {

*v_vfsp,

*v_vfsmountedhere,…}

Struct vnode {

*v_vfsp,

*v_vfsmountedhere,…}

/ (root)

/usr

/ (mounted fs)

CS523S: Operating Systems

mounting a fs
Mounting a FS
  • mount(spec, dir, flags, type, dataptr, datalen);
  • SVR5 uses a global virtual file system switch table (vfssw)
  • allocate and initialize private data
  • initialize vfs struct
  • initialize root vnode in memory (VFS_ROOT)

CS523S: Operating Systems

pathname traversal
Pathname traversal
  • Verify vnode is dir or stop
  • invoke VOP_LOOKUP (ufs_lookup())
  • if found, return pointer to vnode (locked)
  • else not found and last component, return success and vnode of parent directory (locked)
  • not found, release directory, repeat loop

CS523S: Operating Systems

local file systems
Local File Systems
  • S5fs - System V file system. Based on the original implementation.
  • FFS/UFS - BSD developed filesystem with optimized disk usage algorithms

CS523S: Operating Systems

s5fs disk layout
S5fs - Disk layout
  • Viewed as a linear array of blocks
  • Typical disk block size 512, 1024, 2048 bytes
  • Physical block number is the block’s index
  • disk uses cylinder, track and sector
  • first few blocks are the boot area, which is followed by theinode list (fixed size)

CS523S: Operating Systems

disk layout
Disk Layout

tract

sector

heads

cylinder

platters

Rotational speed

disk seek time

CS523S: Operating Systems

s5fs disk layout1
S5fs disk layout

bootarea

superblock

inode list

data

Boot area - code to initialize bootstrap the system

Superblock - metadata for filesystem. Size of FS, size

of inode list, number of free blocks/inodes,

free block/inode list

inode list - linear array of 64byte inode structs

CS523S: Operating Systems

s5fs some details
s5fs - some details

inode

name

Di_mode (2)

di_nlinks (2)

di_uid (2)

di_gid (2)

di_size (4)

di_addr (39)

di_gen (1)

di_atime (4)

di_mtime (4)

di_ctime (4)

.

8

..

45

“”

0

myfile

123

2 byte

14byte

On-disk inode

directory

CS523S: Operating Systems

locating file data blocks
Locating file data blocks

Assume 1024 Byte Blocks

0

1

2

3

4

5

6

7

8

9

10 - indirect

11 - double indirect

12 - triple indirect

256 blocks

65,536 blocks

16,777,216 blocks

CS523S: Operating Systems

s5fs kernel implementation
S5fs Kernel Implementation
  • In-Core Inodes - also include vnode, device id, inode number, flags
  • Inode lookup uses a hash queue based on inode number (amy also use device number)
  • kernel locks inode for reading/writing
  • Read/Write use a buffer cache or VM

CS523S: Operating Systems

problems with s5fs
Problems with s5fs
  • Superblock
  • on-disk inodes
  • Disk block allocation
  • file name size

CS523S: Operating Systems

fast file system ffs
Fast File System - FFS
  • Disk partition divided into cylinder groups
  • superblocks restructured and replicated across partition
    • Constant information
    • cylinder group summary info such as free inodes and free block
  • support block fragments
  • Long file names
  • new disk block allocation strategy

CS523S: Operating Systems

ffs allocation strategy
FFS Allocation strategy
  • Goal: Collocate similar data/info.
  • file inodes located in same cyl group as dir.
  • new dirs created in different cyl groups.
  • Place file data blocks/inode in same cyl group - for size < 48K
  • allocate sequential blocks at a rotationally optimal position.
  • Choose cyl group with “best” free count

CS523S: Operating Systems

is ffs ufs better
Is FFS/UFS Better?
  • Measurements have shown substantial performance benefits over s5fs
  • FFS however, is sub-optimal when the disk is nearly full. Thus 10% is always kept free.
  • Modern disks however, no longer match the underlying assumptions of FFS

CS523S: Operating Systems

buffer cache
Buffer Cache

Free

(LRU)

Hash (device,inode)

CS523S: Operating Systems

other limitations of s5fs and ffs
Other Limitations of s5fs and FFS
  • Performance - hardware designs and modern architectures have redefined the computing environment
  • Crash Recovery do you like waiting for fsck()?
  • Security - do we need more than just 7 bits
  • File Size limitations

CS523S: Operating Systems

performance issues
Performance Issues
  • FFS has a target rotational delay
    • read/write entire track
    • many disks have built-in caches
  • Due to FS Caching, most I/O operations are writes.
  • Synchronous writes of metadata
  • Disk head seeks are expensive

CS523S: Operating Systems

sun ffs cluster
Sun-FFS (cluster)
  • Sets rotational delay to 0
  • read clustering
  • write clustering

CS523S: Operating Systems

log structured fs
Log-Structured FS
  • Entire disk dedicated to log
  • writes to tail of log file
  • garbage collection daemon
  • Dir and Inode structures retained
  • Issue is locating inodes
  • writes a segment at a time

CS523S: Operating Systems

log structured fs1
Log-structured FS
  • Requires a large cache for read efficiency
  • Write efficiency is obtained since the system is always writing to the end of the log file. Why does this help?
  • Why does performance compare to Sun-FFS?
  • What about crash recovery?

CS523S: Operating Systems

4 4bsd portal fs
4.4BSD Portal FS

Portal

daemon

User process

fd

<path>

/p/<path>

fd

Sockets

Protal file system

CS523S: Operating Systems

review of vnode vfs
Review of vnode/vfs
  • Provides a general purpose interface
  • allows multiple file systems to be used simultaneously in a system
  • OO Interface -
    • although limited, no inheritance, fixed interfaces
    • How can we improve on this?

CS523S: Operating Systems

stackable filesystems
Stackable Filesystems

application

application

  • For a given mount point, there is now possible many file systems

/mylocal

MyFS

/local

UFS

CS523S: Operating Systems

ad