File system interface and implementations
Download
1 / 40

File System Interface and Implementations - PowerPoint PPT Presentation


  • 134 Views
  • Uploaded on

File System Interface and Implementations. Fred Kuhns CS523 – Operating Systems. FS Framework in UNIX. Provides persistent storage Facilities for managing data file - abstraction for data container, supports sequential and random access

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'File System Interface and Implementations' - betty_james


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
File system interface and implementations

File System Interface and Implementations

Fred Kuhns

CS523 – Operating Systems

CS523S: Operating Systems


Fs framework in unix
FS Framework in UNIX

  • Provides persistent storage

  • Facilities for managing data

    • file - abstraction for data container, supports sequential and random access

    • file system - permits organizing, manipulating and accessing files

  • User interface specifies behavior and semantics of relevant system calls

    • Interface exported abstractions: files, directories, file descriptorsand differentfile systems

CS523S: Operating Systems


Kernel files and directories
Kernel, Files and Directories

  • kernel provides control operations to name, organize and control access to files but it does not interpret contents

  • Running programs have an associated current working directory. Permits use of relative pathnames. Otherwise complete pathnames are required.

  • File viewed as a collection of bytes

    • Applications requiring more structure must define and implement themselves

CS523S: Operating Systems


Kernel files and directories1
Kernel, Files and Directories

  • files and directories form hierarchical tree structure name space.

    • tree forms a directed acyclic graph

  • Directory entry for a file is known as a hard link.

    • Files may also have symbolic links

  • File may have one or more links

  • POSIX defines library routines {opendir(), readdir(), rewinddir(), closedir()}

struct dirent {

ino_t d_ino;

char d_name[NAME_MAX + 1];

}

CS523S: Operating Systems


File and directory organization
File and Directory Organization

/

(hard) links

bin

etc

dev

usr

vmunix

sh

local

etc

/usr/local/bin/bash

bin

bash

CS523S: Operating Systems


File attributes
File Attributes

  • Type – directory, regular file, FIFO, symbolic link, special.

  • Reference count – number of hard links {link(), unlink()}

  • size in bytes

  • device id – device files resides on

  • inode number - one inode per file, inodes are unique within a disk partition (device id)

  • ownership - user and group id {chown()}

  • access modes - Permissions and modes {chmod()}

    • {read, write execute} for {owner, group or other}

  • timestamps – three different timestamps: last access, last modify, last attributes modified. {utime()}

CS523S: Operating Systems


Permissions and modes
Permissions and Modes

  • Three Mode Flags = {suid, sgid and sticky}

    • suid –

      • File: if set and executable then set the user’s effective user id

      • Directory: Not used

    • sgid –

      • File: if set and executable then set the effective group id. If sgid is set but not executable then mandatory file/record locking

      • Directory: if set then new files inherit group of directory otherwise group or creator.

    • sticky –

      • File: if set and executable file then keep copy of program in swap area.

      • Directory: if set and directory writable then remove/rename if EUID = owner of file/directory or if process has write permission for file. Otherwise any process with write permission to directory may remove or rename.

CS523S: Operating Systems


User view of files
User View of Files

  • File Descriptors (open, dup, dup2, fork)

    • All I/O is through file descriptors

    • references the open file object

    • per process object

    • file descriptors may be dup’ed {dup(), dup2()}, copied on fork {fork()} or passed to unrelated process {(see ioctl() or sendmsg(), recvmsg()}permitting multiple descriptors to reference one object.

  • File Object - holds context

    • created by an open() system call

    • stores file offset

    • reference to vnode

  • vnode - abstract representation of a file

CS523S: Operating Systems


How it works
How it works

fd = open(path, oflag, mode);

lseek(), read(), write() affect offset

File Descriptors

{{0, uf_ofile}

{1, uf_ofile}

{2 , uf_ofile}

{3 , uf_ofile}

{4 , uf_ofile}

{5 , uf_ofile}}

Open File Objects

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...},

{*f_vnode,f_offset,f_count,...}}

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

Vnode/vfs

In-memory

representation

of file

CS523S: Operating Systems


File systems
File Systems

  • File hierarchy composed of one or more File Systems

  • One File System is designated the Root File System

  • Attached to mount points

  • File can not span multiple File Systems

  • Resides on one logical disk

CS523S: Operating Systems


Logical disks
Logical Disks

  • Viewed as linear sequence of fixed sized, randomly accessible blocks.

    • device driver maps FS blocks to underlying storage device.

    • created using newfs or mkfs utilities

  • A file system must reside in a logical disk, however a logical disk need not contain a file system (for example the swap device).

  • Typically logical disk corresponds to partion of a physical disk. However, logical disk may:

    • map to multiple physical disks

    • be mirrored on several physical disks

    • striped across multiple disks or other RAID techniques.

CS523S: Operating Systems


File abstraction
File Abstraction

  • Abstracts different types of I/O objects

    • for example directories, symbolic links, disks, terminals, printers, and pseudodevices (memory, pipes sockets etc).

  • Control interface includes fstat, ioctl, fcntl

  • Symbolic links: file contains a pathname to the linked file/directory. {lstat(), symlink(), readlink()}

  • Pipe and FIFO files:

    • FIFO created using mknod(), lives in the file system name space

    • Pipe created using pipe(), persists as long as opened for reading or writing.

CS523S: Operating Systems


Oo style interfaces
OO Style Interfaces

Instance of derived class

Abstract base class

Struct interface_t

{

// Common functions:

open (), close ()

// Common data:

type, count

// Pure virtual functions

*ops (Null pointer)

// Private data

*data (Null pointer)

}

Struct interface_t

{

open (), close ()

type, count

*ops

*data

}

{my_read()

my_write()

my_init()

my_open()

… }

{device_no,

free_list,

lock, …}

CS523S: Operating Systems


Overview
Overview

System calls

vnode interface

/proc

tmpfs

UFS

HSFS

PCFS

RFS

NFS

swapfs

cdrom

diskette

disk

Process

address

space

Anonymous

memory

Example from Solaris

CS523S: Operating Systems


Vfs vnode framework
Vfs/Vnode Framework

  • Concurrently support multiple file system types

  • transparent interoperation of different file systems within one file hierarchy

    • enable file sharing over network

    • abstract interface allowing easy integration of new file systems by vendors

CS523S: Operating Systems


Objectives
Objectives

  • Operation performed on behalf of current process

  • Support serialized access, I.e. locking

  • must be stateless

  • must be reentrant

  • encourage use of global resources (cache, buffer)

  • support client server architectures

  • use dynamic storage allocation

CS523S: Operating Systems


Vnode vfs interface
Vnode/vfs interface

  • Define abstract interfaces

  • vfs: Fundamental abstraction representing a file system to the kernel

    • Contains pointerss to file system (vfs) dependent operations such as mount, unmount.

  • vnode: Fundamental abstraction representing a file in the kernel

    • defines interface to the file, pointer to file system specific routines. Reference counted.

    • accessed in two ways:

      • 1) I/O related system calls

      • 2) pathname traversal

CS523S: Operating Systems


Vfs overview
vfs Overview

Struct vfsops {

*vfs_mount,

*vfs_root, …}

Struct vfsops {

*vfs_mount,

*vfs_root, …}

rootvfs

private data

private data

Struct vfs {

*vfs_next,

*vfs_vnodecovered,

*vfs_ops,

*vfs_data, …}

Struct vfs {

*vfs_next,

*vfs_vnodecovered,

*vfs_ops,

*vfs_data, …}

Struct vnode {

*v_vfsp,

*v_vfsmountedhere,…}

Struct vnode {

*v_vfsp,

*v_vfsmountedhere,…}

Struct vnode {

*v_vfsp,

*v_vfsmountedhere,…}

/ (root)

/usr

/ (mounted fs)

CS523S: Operating Systems


Mounting a fs
Mounting a FS

  • mount(spec, dir, flags, type, dataptr, datalen);

  • SVR5 uses a global virtual file system switch table (vfssw)

  • allocate and initialize private data

  • initialize vfs struct

  • initialize root vnode in memory (VFS_ROOT)

CS523S: Operating Systems


Pathname traversal
Pathname traversal

  • Verify vnode is dir or stop

  • invoke VOP_LOOKUP (ufs_lookup())

  • if found, return pointer to vnode (locked)

  • else not found and last component, return success and vnode of parent directory (locked)

  • not found, release directory, repeat loop

CS523S: Operating Systems


Local file systems
Local File Systems

  • S5fs - System V file system. Based on the original implementation.

  • FFS/UFS - BSD developed filesystem with optimized disk usage algorithms

CS523S: Operating Systems


S5fs disk layout
S5fs - Disk layout

  • Viewed as a linear array of blocks

  • Typical disk block size 512, 1024, 2048 bytes

  • Physical block number is the block’s index

  • disk uses cylinder, track and sector

  • first few blocks are the boot area, which is followed by theinode list (fixed size)

CS523S: Operating Systems


Disk layout
Disk Layout

tract

sector

heads

cylinder

platters

Rotational speed

disk seek time

CS523S: Operating Systems


S5fs disk layout1
S5fs disk layout

bootarea

superblock

inode list

data

Boot area - code to initialize bootstrap the system

Superblock - metadata for filesystem. Size of FS, size

of inode list, number of free blocks/inodes,

free block/inode list

inode list - linear array of 64byte inode structs

CS523S: Operating Systems


S5fs some details
s5fs - some details

inode

name

Di_mode (2)

di_nlinks (2)

di_uid (2)

di_gid (2)

di_size (4)

di_addr (39)

di_gen (1)

di_atime (4)

di_mtime (4)

di_ctime (4)

.

8

..

45

“”

0

myfile

123

2 byte

14byte

On-disk inode

directory

CS523S: Operating Systems


Locating file data blocks
Locating file data blocks

Assume 1024 Byte Blocks

0

1

2

3

4

5

6

7

8

9

10 - indirect

11 - double indirect

12 - triple indirect

256 blocks

65,536 blocks

16,777,216 blocks

CS523S: Operating Systems


S5fs kernel implementation
S5fs Kernel Implementation

  • In-Core Inodes - also include vnode, device id, inode number, flags

  • Inode lookup uses a hash queue based on inode number (amy also use device number)

  • kernel locks inode for reading/writing

  • Read/Write use a buffer cache or VM

CS523S: Operating Systems


Problems with s5fs
Problems with s5fs

  • Superblock

  • on-disk inodes

  • Disk block allocation

  • file name size

CS523S: Operating Systems


Fast file system ffs
Fast File System - FFS

  • Disk partition divided into cylinder groups

  • superblocks restructured and replicated across partition

    • Constant information

    • cylinder group summary info such as free inodes and free block

  • support block fragments

  • Long file names

  • new disk block allocation strategy

CS523S: Operating Systems


Ffs allocation strategy
FFS Allocation strategy

  • Goal: Collocate similar data/info.

  • file inodes located in same cyl group as dir.

  • new dirs created in different cyl groups.

  • Place file data blocks/inode in same cyl group - for size < 48K

  • allocate sequential blocks at a rotationally optimal position.

  • Choose cyl group with “best” free count

CS523S: Operating Systems


Is ffs ufs better
Is FFS/UFS Better?

  • Measurements have shown substantial performance benefits over s5fs

  • FFS however, is sub-optimal when the disk is nearly full. Thus 10% is always kept free.

  • Modern disks however, no longer match the underlying assumptions of FFS

CS523S: Operating Systems


Buffer cache
Buffer Cache

Free

(LRU)

Hash (device,inode)

CS523S: Operating Systems


Other limitations of s5fs and ffs
Other Limitations of s5fs and FFS

  • Performance - hardware designs and modern architectures have redefined the computing environment

  • Crash Recovery do you like waiting for fsck()?

  • Security - do we need more than just 7 bits

  • File Size limitations

CS523S: Operating Systems


Performance issues
Performance Issues

  • FFS has a target rotational delay

    • read/write entire track

    • many disks have built-in caches

  • Due to FS Caching, most I/O operations are writes.

  • Synchronous writes of metadata

  • Disk head seeks are expensive

CS523S: Operating Systems


Sun ffs cluster
Sun-FFS (cluster)

  • Sets rotational delay to 0

  • read clustering

  • write clustering

CS523S: Operating Systems


Log structured fs
Log-Structured FS

  • Entire disk dedicated to log

  • writes to tail of log file

  • garbage collection daemon

  • Dir and Inode structures retained

  • Issue is locating inodes

  • writes a segment at a time

CS523S: Operating Systems


Log structured fs1
Log-structured FS

  • Requires a large cache for read efficiency

  • Write efficiency is obtained since the system is always writing to the end of the log file. Why does this help?

  • Why does performance compare to Sun-FFS?

  • What about crash recovery?

CS523S: Operating Systems


4 4bsd portal fs
4.4BSD Portal FS

Portal

daemon

User process

fd

<path>

/p/<path>

fd

Sockets

Protal file system

CS523S: Operating Systems


Review of vnode vfs
Review of vnode/vfs

  • Provides a general purpose interface

  • allows multiple file systems to be used simultaneously in a system

  • OO Interface -

    • although limited, no inheritance, fixed interfaces

    • How can we improve on this?

CS523S: Operating Systems


Stackable filesystems
Stackable Filesystems

application

application

  • For a given mount point, there is now possible many file systems

/mylocal

MyFS

/local

UFS

CS523S: Operating Systems


ad