slide1
Download
Skip this Video
Download Presentation
YFS: An Introduction to the next / afs ®

Loading in 2 Seconds...

play fullscreen
1 / 74

YFS: An Introduction to the next / afs ® - PowerPoint PPT Presentation


  • 112 Views
  • Uploaded on

YFS: An Introduction to the next / afs ®. Jeffrey Altman, Daria Brashear, Marc Dionne , & Simon Wilkinson Your File System Inc . and Your File System Ltd. 2014 European AFS and Kerberos Conference. Your File System.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' YFS: An Introduction to the next / afs ®' - yeriel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

YFS: An Introduction to the next /afs®

Jeffrey Altman, Daria Brashear, Marc Dionne, & Simon Wilkinson

Your File System Inc. and Your File System Ltd.

2014 European AFS and Kerberos Conference

slide2

Your File System

  • Your File System Inc. (YFSI) is a New York State Corporation with HQ in Manhattan and registered as a business entity in Canada
  • Your File System Ltd is a wholly owned subsidiary of YFSI with HQ in London
  • YFSI is privately owned and operated
  • YFSI is a Red Hat Partner ISV
slide3

The /afs® Vision

  • Location Transparency: one name space
  • User Mobility: access from any device
  • Security: Flexible model for authentication, privacy, data protection and access control
  • Availability: Temporary loss to small groups for short time periods
  • Integrity: No user initiated backups
  • Heterogeneity: Multiplatform
  • Self service: Low Help Desk costs
  • Atomic Publishing: Software, documentation, web sites, ..
  • Real time collaboration: Distributed File Locking
  • Distributed Administration
slide4

AFS® is 30 years old

The vision was decades ahead of its time in 1983

The implementation is decades behind in 2014

slide5

The Price of Inaction

  • Limited network throughput
  • Increased call processing latency
  • Decreased service reliability and availability
  • Elevated risk of distributed deadlocks
  • Inability to use full capability of available hardware
  • Failure to keep up with competing technologies

That /afs is still in use today is a credit to its vision and the strength of its architecture.

slide6

13 Years of Open Source

Major system rewrites are few and far between

“Contractor Model of Support” leads to many small and localized changes

A lack of consistent vision and quality control

Few incentives to invest in the next 30 years

slide8

The YFS Vision for /afs

  • Application Transparency
    • Be a Tier One file system on all major OSes
  • Embrace multi-producer, multi-consumer work flows
  • Extended Integrity: Disaster Recovery
slide9

The YFS Vision for /afs

  • Be performance competitive
    • Lustre, GPFS, Panasas, …
  • Best of breed data security
  • Improved Ease of Use
  • Designed for the long term
slide10

Preserving the old while providing the new

  • Improved performance with existing hardware
  • Cost reductions due to hardware consolidation
  • Zero data loss as part of a transition
  • No flag day required
    • Mixed deployments are encouraged
slide12

Performance: Why Does it Matter?

Performance issues restrict the jobs that sites are willing to run in /afs

Deploying excessive hardware to solve load distribution and fairness problems is expensive

Support for multiple file systems costs money, requires additional staff, can result in data duplication and out of sync issues

slide13

RX Performance

  • Reduced contention in the listener thread
    • 10 gbit network interface saturation
  • 255 packet window size (per call) without degradation
    • Order of magnitude faster on high latency links
  • Dynamic Thread Pools
    • Thread Count limited by OS resources
slide14

Scalability: Name Space Growth

  • 64-bit volume IDs
  • 96-bit (79 octillion) vnode IDs
  • 64-bit,100ns granular timestamps
    • 2038 ready
  • Ubik databases extensible up to 16 exabytes
  • Partitions, volumes and quotas tracked up to 16 zettabytes
slide15

Performance: Message Flow Optimizations

  • Optimized Cache Manager handshakes
  • Volume Status Information
    • Reduces number of GetVolumeStatus RPCs
    • Permits RW / RO data cache sharing
    • Improved caching of RO volume per user permissions
      • Fewer FetchStatus RPCs for RO volumes
slide16

Performance: Fileserver

  • Host and callback package rewritten
    • Significantly faster callback breaks
  • Vnode lock contention dramatically reduced
  • Distributed writing to shared data sets now possible
slide17

Performance: POSIX O_DIRECT

Open mode supported on some OSes

Bypasses VFS cache and AFS cache for both read and write

No file threshold to tune

Data is copied directly to the caller, or directly from the caller to the file server

slide19

Security: Why Does it Matter?

  • Data breaches and exposures are followed by a high cost
    • Public Relations Nightmare
    • Costs of Identity Theft Detection Services (in U.S.)
    • Loss of employment for key staff members
    • Organizational reorganization
    • Disruption of core mission when forced to address security concerns in crisis mode
slide20

What is YFS Security?

  • Multi-layered policies
    • Flexibility for self service end users
    • System administrator controls
  • Network Security
  • Reduced Information Exposure
  • Minimal Privilege Services
slide21

Security: End Users

  • Self Service Group Management
  • Per-Object ACLs
    • Cross directory hard links now permitted
slide22

Security: System Administrators

  • Volume ACLs
    • Limits the permissions that end users can grant
slide23

Security: System Administrators

  • Volume Security Policy
    • Per-Volume minimum acceptable rx connection security properties
  • File Server Security Policy
    • Per-server minimum acceptable rx connection security properties
    • Only volumes with weaker or equivalent security policy can be attached, moved to, or restored to.
slide24

Security: Network

  • YFS RXGK Security Class
    • GSS Kerberos 5 authentication
    • AES-256 wire privacy and integrity protection
  • Cell wide key for DB servers
  • Individual keys for file servers
  • Per-host keys for BOS Overseer Service
slide25

Security: Callbacks

  • YFS protects the callback channel with AES-256 privacy and integrity protection
    • when rxgk is used for the incoming connection
    • Avoids leaking information about volume and file ids accessed by a client
    • Prevents forged messages from invalidate callback state
slide26

Security: Minimal Privileges

  • Server Processes execute under a daemon account
    • Not Root
slide27

Security: Keyed CMs & Machine IDs

  • Cache Managers can be issued
    • a Kerberos keytab
    • a Protection DB Machine ID
  • Keyed Cache Managers can use privacy for all connections
  • Machines IDs are similar to User IDs
    • Can be placed on ACLs and added to Groups
    • But are not included in system:authuser
slide29

File System Extensions

  • Per File ACLs
  • Cross directory hard links

Extensions for Microsoft Windows

  • Mandatory Locks
    • Advisory locks are not enforced by the file server
  • Symlink Updates
    • Reparse Points can be updated without FileID change
  • CreateFile with Lock
    • Avoids races when simulating OpLock semantics
slide30

Command Output Clarity

  • Modifications to human readable and machine readable output
  • vos examine, listvol, rxdebug, xstat_fs, …
    • Consolidate output
    • Introduce consistency across command options
  • Machine readable output –format is not human formatted
    • All fields are now separated by single tabs
    • Easy to import into spreadsheets and databases
slide31

Library Cleanup

  • All libraries are thread safe
  • Built using libtool
  • Intended for use implementing language bindings
slide32

libacquire

  • A library to obtain tokens
    • rxkad
    • yfs-rxgk
  • aklog is a wrapper
  • Can be linked to pam modules
slide33

Automated Windows Domain Token Acquisition

  • Triggered by access denied errors
  • Automatic Token acquisition using Logon Session Kerberos Credentials
  • Works with all applications that use
    • WNet API: Network Providers
    • Shell API: Explorer, Office, anything with an Open dialog
slide35

Why are Deployment and Configuration Important?

Simplify Server Configuration

Provide Extensibility for New Features

BOS command lines are limited in length

Permit the construction of flexible test suites

slide36

Flexible Configuration

  • Greatly improved configuration flexibility and convenience
    • Custom file layouts are possible
    • All settings centralized in a single configuration area, single file or directory
    • A configuration directory can ease distribution of custom options
    • All command line options can be set in configuration
slide37

Test Suites

  • Goal
    • Provide a test for every new feature, library function, RPC
    • Provide a test with every bug fix, if possible
  • Requirements
    • Ability to spin up the various servers and provide a test configuration
    • All tests must be able to run as a regular user
    • Must be able to serve test data not necessarily under /vicep*
slide38

Extensive test suite coverage

A complete test cell can spin up in a few seconds

Many tests spin up a cell and destroy it when done, maintaining test independence

Client testing through libafscp and fuse client

All new features require tests before merging

slide39

Sample systemdyfs-server.service

[Unit]

Description=YFS Server Service

After=syslog.targetnetwork.target

[Service]

EnvironmentFile=-/etc/sysconfig/yfs

ExecStart=/usr/local/sbin/bosserver -config /s/yfs/server/yfs-server.conf -nofork

ExecStop=/usr/local/bin/bos -config /s/yfs/server/yfs-server.conf shutdown hurricane.marcdionne.net -wait -localauth

User=yfs

Group=yfs

[Install]

WantedBy=multi-user.target

slide40

Sample file layout

[[email protected] /s/yfs/server ]$ ls -l

total 60

drwxr-xr-x. 2 yfsyfs 4096 Mar 23 04:00 bos

-rw-r--r--. 1 yfsyfs 526 Jul 15 2013 bos.keytab

-rwxr-xr-x. 1 yfsyfs 26 Jul 15 2013 cacheinfo

drwxrwxr-x. 6 yfsyfs 4096 Jan 11 15:36 data

drwxrwx---. 2 yfsyfs 4096 Oct 25 10:47 db

-rw-r--r--. 1 yfsyfs 4 Jan 6 09:52 KeyFile

-rw-r--r--. 1 yfsyfs 144 Jan 6 09:52 KeyFileExt

drwxrwx---. 2 yfsyfs 4096 Mar 25 10:28 local

drwxrwxrwx. 2 yfsyfs 12288 Mar 25 10:29 logs

-rw-r--r--. 1 yfsyfs 15 Sep 12 2013 ThisCell

-rw-r--r--. 1 yfsyfs 114 Dec 19 16:56 UserList

-rw-r-----. 1 yfsyfs 2000 Aug 5 2013 vl.keytab

drwxrwxr-x. 2 yfsyfs 4096 Mar 26 18:25 yfs-server.conf

[[email protected] /s/yfs/server ]$ ls -l yfs-server.conf/

total 8

-rw-r--r--. 1 yfsyfs 645 Mar 26 18:25 cellservdb.conf

-rw-rw-r--. 1 yfsyfs 792 Mar 4 15:48 yfs-server.conf

slide41

Sample cellservdb.conf

grand.central.org = {

description = "GCO Public CellServDB 23 Apr 2008"

servers = {

penn.central.org = {

addr = 128.2.203.61

}

grand.mit.edu = {

addr = 18.9.48.14

}

andrew.e.kth.se = {

addr = 130.237.48.87

}

}

}

[cells]

example.com = {

description = "Test cell"

servers = {

blizzard.marcdionne.net = {

addr = 192.168.0.113

}

}

}

marcdionne.net = {

description = "Marc\'s cell"

servers = {

hurricane.marcdionne.net = {

addr = 192.168.0.107

}

}

}

slide42

Sample server configuration

[vlserver]

keytab = /s/yfs/server/vl.keytab

auditlog = /s/yfs/server/logs/audVl

[volserver]

d = 125

auditlog = /s/yfs/server/logs/audVol

[bosserver]

auditlog = /s/yfs/server/logs/audBos

[ptserver]

auditlog = /s/yfs/server/logs/audPt

[salvager]

auditlog = /s/yfs/server/logs/audSalv

[salvageserver]

auditlog = /s/yfs/server/logs/audSalvserv

[dirpath]

SERVER_ETC_DIR = /s/yfs/server

SERVER_DB_DIR = /s/yfs/server/db

SERVER_LOGS_DIR = /s/yfs/server/logs

SERVER_BOSCONFIG_DIR = /s/yfs/server/bos

SERVER_LOCAL_DIR = /s/yfs/server/local

SERVER_PART_PREFIX_DIR = /s/yfs/server/data

[fileserver]

d = 125

p = 200

nojumbo =

auditlog = /s/yfs/server/logs/audFile

security = yfs-rxgk:cryptrxkad:clearrxnullrxkad:crypt

slide44

Why Packaging Changes are Important?

Installation is the initial experience an end user has with the product

If the installation process is frustrating, the end user is likely to be unhappy with the product

Lack of digital signatures can block the installation of a package or trigger a frightening dialog

slide45

Installation Packages

  • New installation packages
    • Windows
    • OSX
    • Linux
      • Debian
      • Fedora
      • RHEL6 and RHEL7
slide47

Microsoft Windows®

  • Single installer
    • 64-bit and 32-bit components
    • Heimdal Side by Side Assembly
    • Heimdal Command Line tools
    • Automatic Cache Sizing
  • All components digitally signed
    • Microsoft Cross Signing of Drivers
slide49

OSX

  • Flat package
  • Integral packages for client, server and development
  • Digital signatures on the package, the kext and the binaries using Apple-issued certificate
slide50

Linux

New packaging for Debian, Fedora and RHEL

Integral packages for client, db services, and file service

Digital signatures on installation packages

slide52

Dual Protocol Stacks

  • Allows advanced features while maintaining backwards compatibility with AFS®
  • AFS protocol suite has all of the capabilities and limitations of OpenAFS
  • YFS features only available on YFS protocol suite
    • rxgk, file server, vol server, vl server, pt server enhancements
  • Transparent negotiation of protocol suite
slide53

Mixed Mode Cells

  • Two cell types are defined:
    • AFS cell deploys OpenAFS or IBM AFS vlservers
    • YFS cell deploys the YFS location server
  • OpenAFS and YFS File Servers can be joined to either cell
slide54

YFS Client in AFS Cell

No support for RXGK, AES-256

No support for file server security policies

slide55

YFS Server in AFS Cell

Improved RX Performance for writes

No Rxgk

Volume IDs restricted to AFS limits

Security Policies cannot be enforced

Only AFS compatible capabilities can be registered

IPv6 addresses cannot be registered

slide56

YFS File Server in AFS cell

vos

AFS

vlserver

AFS volume

format

AFS

fileserver

YFS

fileserver

slide57

YFS Server in AFS Cell

Improved RX Performance for writes

No Rxgk

Volume IDs restricted to AFS limits

Security Policies cannot be enforced

Only AFS compatible capabilities can be registered

IPv6 addresses cannot be registered

slide58

AFS Client in YFS Cell

No support for RXGK, AES-256

No support for file server security policies

Volumes with ID above 232-1 inaccessible

Mandatory locks cannot be requested but will be enforced

Volume sizes and quotas >231KB will be faked

Other restrictions as required to enforce security policies

slide59

AFS File Server in YFS cell

YFS

location

server

vos

AFS volume

format

AFS

fileserver

YFS

fileserver

YFS volume

format

59

slide60

AFS and YFS Volserver Compatibility

  • RW volumes on YFS server cannot be replicated to AFS server
  • Volumes containing YFS tags cannot be moved to an AFS server
    • ACL Data
    • Volume Attributes (ACL or Security Policy)
  • Data transfers protected with Rxkad and Fcrypt
  • RX performance improved in YFS to AFS direction
yfs file server
YFS File Server

YFS POSIX attribute backend store

YFS protocol suite

AFS protocol suite

61

yfs protection server
YFS Protection Server

64 bit Ubik database

YFS protocol suite

AFS protocol suite

62

yfs location server
YFS Location Server

64 bit Ubik database

rxgk

keyserver

YFS protocol suite

AFS protocol suite

63

slide65

Documentation

Updated man pages

New Quick Start Guides

Updated Administrator’s Guide

slide67

Export Licenses

The U.S. Government has classified YFS 1.x as a mass market product

Worldwide Export permitted with a few exceptions

No export restrictions on distribution by customers

slide68

YFS 1.0 Binary License

  • A full suite of clients and servers
    • Windows
    • OSX
    • iOS
    • RHEL5, RHEL6, RHEL7
    • Fedora
    • Debian
    • Solaris
    • AIX
slide69

Support

  • Free updates to new releases (one year)
    • Every four month release cadence
  • Free security updates (two years)
  • Unlimited e-mail / web support (one year)
  • Cell performance evaluation (once per year)
  • Remote monitoring service (one year)
slide70

Products

Cell (no replication)

1 Server (DB and File)

Base cell (replication)

4 DB Servers

4 File Servers

1000 User or Machine IDs

Unlimited Client devices

Additional Servers

Additional IDs

Annual purchases continuesupport

Non-redistribution Source code license available

Training (on-site or web)

slide71

Availability

General Availability End of May 2014

First update, September 2014

slide72

2014 Road Map

  • Feature Priorities
    • IPv6 enhancements
    • Rapid Partition Relocation
    • Extended Volume Names
    • New Directory Format
      • Unlimited Directory Sizes
      • Extended Attributes
      • Alternate Data Streams
    • Read/write Replication
    • Extended Callbacks
ad