Pnfs update 2010 09 17
1 / 24

PNFS Update 2010-09-17 - PowerPoint PPT Presentation

  • Uploaded on

pNFS Update 2010-09-17. Sorin Faibish, EMC. Outline. What is pNFS? pNFS Timeline Standards Status Industry Support pNFS EMC Status and Roadmap. Client. pNFS protocol. Storage-access protocol. Metadata Server. Control protocol. Data Servers. pNFS. pNFS protocol

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'PNFS Update 2010-09-17' - mari

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Pnfs update 2010 09 17 l.jpg

pNFS Update 2010-09-17

Sorin Faibish, EMC

Outline l.jpg

  • What is pNFS?

  • pNFS Timeline

  • Standards Status

  • Industry Support

  • pNFS EMC Status and Roadmap

pNFS Update 2010-09-17

Slide3 l.jpg


pNFS protocol







Data Servers


  • pNFSprotocol

    • standardized: NFSv4.1

  • Storage-access protocol

    • files (NFSv4.1)

    • blocks (FC, iSCSI, FCoE, IB)

    • objects (OSD2)

  • Control protocol

    • Outside of the pNFS standard

  • Distributes data across storage cluster

  • Eliminates or reduces load and capacity balancing

  • And yes: can accelerate I/O

pNFS Update 2010-09-17

Slide4 l.jpg

pNFS Multiple Layouts

Client gets a layout from the NFSv4.1 server

The layout maps the file onto storage devices and addresses

The client uses the layout to perform direct I/O to storage

At any time the server can recall the layout

Client commits changes and returns the layout when it’s done

pNFS is optional, the client can always use regular NFSv4.1 I/O`




NFSv4.1 Server

pNFS Update 2010-09-17

Slide5 l.jpg

Client Apps

  • files (NFSv4.1)

  • objects (OSD2)

  • blocks (SCSI)

  • Future back ends …

NFSv4.1 Layoutgrant & recall

pNFS Client



pNFS Server




File System

Control Protocol

Linux pNFS Clients

Transparent to applications

Common client for different storage back ends

Fewer support issues for storage vendors

Normalizes access to clustered file systems

pNFS Update 2010-09-17

Nfsv4 1 open source status l.jpg
NFSv4.1 – Open Source Status

Two OpenSource Implementations

OpenSolaris and Linux (file, osd and block)

OpenSolaris Client and Server

Support only file-based layout

Support for multi-device striping already present (NFSv4.1 + pNFS)‏

“Simple Policy Engine” for policy-driven layouts also in the gate

Linux Client and Server

Support files (NFSv4.1)‏

Support currently also blocks (SCSI), objects (OSD T10)‏

Client consists of generic pNFS client and “plug ins” for “layout drivers

Windows NFSv4.1 Client from CITI - NEW

pNFS Update 2010-09-17


Timeline l.jpg

2004 – CMU, NetApp and Panasas draft pNFS problem and requirement statements

2005 – CITI, EMC, NetApp and Panasas draft pNFS extensions to NFS

2005 – NetApp and Sun demonstrate pNFS at Connectathon

2005 – pNFS added to NFSv4.1 draft

2006 - 2008 – specification baked

Bake-a-thons, Connectathons

26 iterations of NFSv4.1/pNFS spec

2009 – RFC submitted (680 pages)

2010 – RFC 5661 published

2010 – Fedora 13 includes pNFS server/client rpms (did you triy it yet?)

pNFS Update 2010-09-17


Pnfs standards status l.jpg
pNFS Standards Status

NFSv4.1/pNFS were standardized at IETF

NFSv4 working group (WG)

All done including RFC 5661,3,4:

WG last call (DONE)

Area Director review (DONE)

IETF last call (DONE)

IESG approval for publication (DONE)

IANA review (DONE)

RFC publication (2010)

Consists of several documents:

RFC 5661 - NFSv4.1/pNFS/file layout

RFC 5662 - NFSv4.1 protocol description for IDL (rpcgen) compiler

RFC 5663 - blocks layout

RFC 5664 - objects layout

RFC 5665 - netid specification for transport protocol independence (IPv4, IPv6, RDMA)

pNFS Update 2010-09-17


Industry contributors to pnfs standard l.jpg
Industry Contributors to pNFS Standard

pNFS Update 2010-09-17


Traditional pnfs use cases l.jpg


iSCSI / FC Network

Traditional pNFS Use Cases

  • Seismic Data Processing / Geosciences' Applications

  • Broadcast & Video Production

  • High Performance Streaming Video

  • Finite Element Analysis for Modeling & Simulation

  • HPC for Simulation & Modeling

  • Data Intensive Searching for Computational Infrastructures

pNFS Ethernet Solution

Application Server Racks







pNFS Server

pNFS Update 2010-09-17

Questions client technology l.jpg
Questions – Client Technology

When will a valid pNFS client be available?

How does the parallel NFS block layout work?

What’s an object layout, how do objects compare to blocks/files?

How do current cluster file systems work with pNFS client and server?

How is client access to data servers coordinated and controlled?

Attendee Questions Encouraged…

pNFS Update 2010-09-17

Questions files blocks and objects l.jpg
Questions – Files, Blocks and Objects

How many layout types can there be?

How does pNFS make managing a lot of systems easier?

Can my application control how its data is striped?

When can we expect to see real pNFS performance, not vendor claims based on older technology?

How can I use pNFS in an enterprise environment; would it support non-disruptive (NDU) upgrades?

Attendee Questions Encouraged…

pNFS Update 2010-09-17

Questions data management l.jpg
Questions – Data Management

  • Can I retain data management practices I use today based on Snapshots and Volume replication?

  • Can File, Object and Block layouts co-exist in the same storage network?

  • Can a client use volumes accessed via each layout concurrently?

  • I’m deploying a Unified Ethernet Fabric; how do I secure data access – files, blocks & objects?

  • Attendee questions encouraged…

pNFS Update 2010-09-17

Questions what else l.jpg
Questions – What else

  • NFSv4.1 and pNFS what are the expectations compared to 4.0?

  • What additional function do you want to see added to 4.1:

    • FedFS

    • Copy server side

  • What would you like to see in NFSv4.2 additional to what is proposed:

    • Sparse files

    • Access Permissions

pNFS Update 2010-09-17

Industry support implementations l.jpg



Oracle (Open Solaris)

ESX (prototype in 5.0)


Desy (file)

EMC (block)

IBM (file)

Linux (all layouts)

LSI (block)

NetApp (file)

Panasas (object)

Oracle (file/OpenSolaris)

Industry Support - Implementations

Several other implementations have been tested at Bake-a-thons and Connectathons

pNFS Update 2010-09-17

Linux status l.jpg
Linux Status

  • Client

    • Consists of generic pNFS client and “plug ins” for “layout drivers”

    • Supports files, blocks, objects

    • Contributors: CITI, EMC, NetApp, Panasas

  • Server

    • Supports files, blocks, objects

    • Contributors: CITI, EMC, IBM, NetApp, Panasas

  • Finalizing patches for – NFSv4.1 sessions

  • Predicted timeline:

    • Basic NFSv4.1 features 2H2010

    • NFSv4.1 pNFS and layout drivers by 1H2011

    • Linux distributions shipping supported pNFS in 2H2011.

pNFS Update 2010-09-17

Opensolaris status l.jpg
OpenSolaris Status

Developing both pNFS client and server in open

  • [email protected]

    File-based implementation exploiting ZFS capabilities

    Management via Simple Policy Engine (SPE)

    pNFS data path over RDMA (on Infiniband)

pNFS Update 2010-09-17

Emc and pnfs l.jpg

EMC and pNFS

History and Roadmap

Parallel nfs pnfs l.jpg



NFS Server

Parallel NFS - pNFS

  • NFS file naming, management, and administration

  • Parallel high bandwidth file access (via Storage Network)

  • Block Layout leverages existent SAN infrastructures


Data Network - LAN


Pnfs block layout high i o speeds l.jpg

HPC Jobs

Compute Engines




pNFS Block Layout - high I/O speeds

Storage must be Networked

pNFS addresses storage access issues

  • Remove servers layer between CE and shared storage

  • Separates MD traffic from Data Traffic

  • Asymmetric storage architectures increase scalability

  • Leverages SSD to increase I/O speed

    • Automatic tiering

  • Improves utilization to any SAN infrastructure:

    • FCoE, Infiniband, FC, iSCSI

  • Enable access to PB’s of storage at GB’s/sec speeds

    • Demonstrated by existent MPFS deployments

  • Combine multiple MD servers in a unified storage system

  • MD server is any Celerra NAS server supporting:

    • NFSv3, CIFS, MPFS and pNFS

    • Tiered services for increased scalability

HPC Architecture




Pnfs block layout the beginning l.jpg
pNFS Block Layout – The beginning

  • The ancestors of pNFS Block Layout are NAS accelerators - 1998:

    • EMC-MPFS, Quantum-StoreNext and Mercury-Sanergy

  • EMC donate the FMP (MPFS) protocol and IP

    • Open source version of FMP client (iRoad) - 2003

    • IETF pNFS Block Layout = modified open storage FMP protocol - 2004

  • EMC support pNFS Block Layout in Linux kernel by join work with CITI: Peter Honeyman, Fred Isaman, Bruce Fields

    • Current pNFS block layout open source client and NFSv4.1 demonstrated at bake-a-thons

    • Ongoing funding the project, in 5th year = strong EMC commitment

    • Customers can experience the value of pNFS using the EMC FMP open source driver, or by installing current shipping MPFS product.

Pnfs block layout now l.jpg
pNFS Block Layout – Now

  • pNFS will supports any SAN storage (LSI, EMC, other SAN)

    • Working with other SAN vendors to promote pNFS Block Layout

  • EMC supports NFSv4.1 and pNFS server using pNFS clients in Linux kernel

    • Prototype demonstrated at latest Bake-a-thon

    • Demo on Laptop with VM and real clients

  • EMC is working with all the pNFS developers to accelerate adoption by HPC

    • The goal is to combine all flavors of pNFS servers accessed by each Linux client in one single infrastructure

    • Working with Linux Distributions and Linux kernel developers

  • What value brings pNFS block layout

    • Leverage existent SAN storage and connectivity

    • Allow access to SAN storage by NFSv4 network clients

    • Virtualizes multi-vendor storage arrays into a single unified view

Pnfs block layout roadmap l.jpg
pNFS Block Layout – Roadmap

  • EMC supports pNFS block in current Celerra release

    • Q3 2010 – First vendor to announce pNFS server

    • Fedora 13 has support for pNFS block funded by EMC

  • EMC plans to support NFSv4.1 and pNFS file in future

    • Prototype will be presented at Bake-A-Thon at EMC

  • EMC hosts the Fall Bake-a-Thon in at EMC

    • October 4-8, 2010

  • EMC contributes new protocol enhancement for NFSv4.2

    • Draft:

  • EMC is working with all the pNFS vendors to accelerate adoption

    • Sponsor SNIA ESF

    • Support pNFS for MS Windows (we are already testing)

    • Working with Linux Distributions RHEL 6.1 is the plan (Q2 2011)