the consequences of decentralized security in a cooperative storage system l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
The Consequences of Decentralized Security in a Cooperative Storage System PowerPoint Presentation
Download Presentation
The Consequences of Decentralized Security in a Cooperative Storage System

Loading in 2 Seconds...

play fullscreen
1 / 42

The Consequences of Decentralized Security in a Cooperative Storage System - PowerPoint PPT Presentation


  • 249 Views
  • Uploaded on

The Consequences of Decentralized Security in a Cooperative Storage System. Douglas Thain, Chris Moretti, Paul Madrid, Phil Snowberger, and Jeff Hemmes University of Notre Dame http://www.cse.nd.edu/~ccl IEEE Workshop on Security in Storage 2005. Abstract.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'The Consequences of Decentralized Security in a Cooperative Storage System' - Jims


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
the consequences of decentralized security in a cooperative storage system

The Consequences ofDecentralized Security in a Cooperative Storage System

Douglas Thain, Chris Moretti, Paul Madrid,

Phil Snowberger, and Jeff Hemmes

University of Notre Dame

http://www.cse.nd.edu/~ccl

IEEE Workshop on Security in Storage 2005

abstract
Abstract
  • Suppose that security in storage has been deployed at all endpoints.
  • How does this affect the design of distributed storage systems that rely upon these devices?
  • Clients must become much more:
    • Fault tolerant, adaptive, and self reliant.
    • Aware of resource allocation issues.
    • Helpful to the end user!
  • Environment: Storage Pool at Notre Dame
traditional file system security
Traditional File System Security

appl

appl

appl

I am

John Doe!

untrusted network:

Ethernet

Internet

Security Interface

placement,

replication,

reliability

File System Abstraction

trusted network:

PCIRAIDSAN

Myrinet

Ethernet

disk

disk

disk

Owner of

Inode 9842

is UID 56

file

decentralized security
Decentralized Security

appl

appl

appl

placement,

replication,

reliability

Abstr.

Abstr.

Abstr.

untrusted network:

Ethernet

Internet

I am

John Doe!

Security

Security

Security

disk

disk

disk

Owner of

File /foo/bar

is John Doe

file

what is cooperative storage
What is Cooperative Storage?
  • Many devices bound together that can accomplish more than one device alone.
    • Improve capacity, reliability, performance...
    • Could be one person, or many cooperating users.
  • Key property:
    • Each person retains absolute control of their own resources by setting local policies.
    • People share and collaborate with others that they know and trust. No free love! No central control!
    • However, some resources are set up for the common good by an authority. (CS workstations usable by any member of the CS department, says the chair.)
slide7

App

file transfer

App

Adapter

Central

Filesystem

Distributed Filesystem Abstraction

Adapter

Distributed Database Abstraction

file

server

file

server

file

server

file

server

file

server

file

server

file

server

3PT

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

UNIX

Cluster administrator controls

policy on all storage in cluster

Workstations owners control

policy on each machine.

App

Adapter

???

file

system

file

system

file

system

file

system

file

system

file

system

file

system

cfs central file system
CFS: Central File System

appl

appl

appl

adapter

adapter

adapter

CFS

CFS

CFS

file

server

file

file

file

dsfs dist shared file system

access

data

lookup

file

location

DSFS: Dist. Shared File System

appl

appl

adapter

adapter

DSFS

DSFS

file

server

file

server

file

server

file

file

file

file

file

ptr

file

file

file

file

file

ptr

ptr

dsdb dist shared database
DSDB: Dist. Shared Database

appl

appl

adapter

adapter

DSDB

DSDB

insert

query

direct

access

file

server

database

server

file

server

create

file

file

file

file index

file

file

file

file

file

file

file

file

applications
Applications
  • Simple and Secure Remote Access
    • CDF: Remote Dynamic Linking
    • BaBar: Remote Database Access
    • LHC: Semantic Remote Filesystems
  • Distributed File Systems
    • GRAND: Scalable Archive for Online Data
  • Distributed Databases
    • GEMS: Molecular Dynamics Simulation
    • CVRL: Biometric Image Storage/Analysis
challenges of decentralization
Challenges of Decentralization
  • Unbounded Set of Users
    • There is no global /etc/passwd or /etc/group!
  • Multiple Identities per User
    • Kerberos creds from Notre Dame / Wisconsin.
    • GSI creds from ND/UW/DOE/NCSA.
  • New Decision Points
    • Placement decision made, but action fails!
    • Directory op succeeds, but file creation fails!
  • Unexpected Policy Coupling
    • Data placement may affect access control!
outline of paper
Outline of Paper
  • Centralized vs Decentralized Security
  • Architecture of Cooperative Storage
  • Basic Security Mechanism
    • Problem: Complexity Confuses!
    • Detail: Reservation Right
  • Challenges
    • Authorization in Distributed File Systems
    • Logistics of Third Party Transfer
    • Mechanisms for Active Storage
    • Semantics of Distributed Group Management
basic security mechanism
Basic Security Mechanism
  • Negotiate an Authentication Method
    • Client proposes, server agrees/disagrees.
    • Default ordering works for most + manual override.
    • Different servers/clients may support diff subsets.
  • Then, Authenticate via Chosen Method
    • May involve challenges, cert exchange, etc...
  • Yields a Subject Name for the Session:
    • kerberos:dthain@nd.edu
    • globus:/O=NotreDame/CN=DouglasThain
    • hostname:hedwig.cse.nd.edu
    • unix:dthain
authorization mechanism
Authorization Mechanism
  • Unix Access Controls Are Not Sufficient
    • Integer UIDs are not sufficient for principals.
    • Nine owner/group/others bits are restrictive.
    • Mapping from subjects to Unix is a mess.
  • Place Variable Length ACLs on dirs:

globus:/O=NotreDame/CN=DThain RWLAX

kerberos:dthain@nd.edu RWL

hostname:*.cs.nd.edu RL

globus:/O=NotreDame/* RL

problem complexity confuses
Problem: Complexity Confuses!
  • For beginning users:
    • Negotiated authentication makes life easy.
    • Everybody can authenticate in some way.
    • Most users don’t think about it first.
  • For advanced users:
    • Negotiation has unexpected effects.
    • What happens when credentials expire?
    • For long running / large tasks, better to manually specify the authentication mode.
    • AuthN failure is easier to retry than authZ failure!
  • Unexpected authentication is hard to debug.
    • Full detail logging mode reveals auth algorithm.
    • Always prominently display subject name in all tools!
problem shared namespace

test.c

test.dat

a.out

cms.exe

Problem: Shared Namespace

file

server

globus:/O=NotreDame/* RWLAX

solution reservation v right

/O=NotreDame/CN=Monk

/O=NotreDame/CN=Ted

mkdir

mkdir

/O=NotreDame/CN=Monk RWLA

/O=NotreDame/CN=Ted RWLA

test.c

a.out

test.c

a.out

Solution: Reservation (V) Right

file

server

mkdir only!

O=NotreDame/CN=* V(RWLA)

outline of paper25
Outline of Paper
  • Centralized vs Decentralized Security
  • Architecture of Cooperative Storage
  • Basic Security Mechanism
    • Problem: Complexity Confuses!
    • Detail: Reservation Right
  • Challenges
    • Authorization in Distributed File Systems
    • Logistics of Third Party Transfer
    • Mechanisms for Active Storage
    • Semantics of Distributed Group Management
dsfs dist shared file system26

access

data

lookup

file

location

DSFS: Dist. Shared File System

appl

appl

adapter

adapter

DSFS

DSFS

file

server

file

server

file

server

file

file

file

file

file

ptr

file

file

file

file

file

ptr

ptr

dsfs logistics
DSFS Logistics
  • Consider Creating a File:
    • Fetch list of resources:
      • online catalog / static list / user selected
    • Make placement decision:
      • random / fill in order / user selected
    • Create stub file on dir server. (fail?)
    • Create actual file on data server. (fail?)
  • Note that two access controls are in play:
    • One controls access to the namespace.
    • Another controls access to the data storage.
dsfs applications
DSFS Applications
  • Personal Mass Storage
    • Expand your local filesystem to include all the disks available in a cluster / lab / basement.
  • Distributed /tmp for Cluster Computing
    • Harness remote cluster for the duration of a job.
  • Multi-User Scalable Storage
    • Department provides directory, but no space.
      • /O=NotreDame/O=CSE/CN=* RWL
    • Participants provide their own data servers.
      • /O=NotreDame/O=CSE/CN=JohnDoe RWLA
    • Separates provisioning from access!
dealing with failure
Dealing with Failure
  • Failure to place data is very common!
    • Unexpected access controls on device.
    • Device is temporarily unavailable. (reboot?)
    • Device is newly installed or creds expired.
    • Owner changed the sharing policy.
  • Soln: Client Needs to Model the System
    • Track successes and failures on each device.
    • Failed devices are not tried again for a time.
    • Of course, cannot avoid a device forever...
outline of paper30
Outline of Paper
  • Centralized vs Decentralized Security
  • Architecture of Cooperative Storage
  • Basic Security Mechanism
    • Problem: Complexity Confuses!
    • Detail: Reservation Right
  • Challenges
    • Authorization in Distributed File Systems
    • Logistics of Third Party Transfer
    • Mechanisms for Active Storage
    • Semantics of Distributed Group Management
pins processing in storage
PINS: Processing in Storage
  • Observation:
    • Traditional clusters separate CPU and storage into two distinct systems/problems.
    • Distributed computing is always some direct combination of CPU and I/O needs.
  • Idea: PINS
    • Cluster HW is already a tighly integrated complex of CPU and I/O. Make the SW reflect the HW.
    • Key: Always compute in the same place that the data is located. Leave newly created data in place.
compute via passive storage
Compute via Passive Storage

Compute Y=F(X)

where X={A,B,C,D}

F

Y1

Y2

Y3

Y4

file server

file server

file server

file server

(X 200)

A

B

C

D

S2

S3

S4

S1

compute via active storage

Y1

Y2

Y3

Y4

Compute via Active Storage

Compute Y=F(X)

where X={A,B,C,D}

F

F

F

F

file server

file server

file server

file server

(X 200)

A

B

C

D

S2

S3

S4

S1

technique identity boxing

storage owner

client

> open x.nd.edu

> put sim.exe

> put in.dat

> exec sim.exe

> get out.dat

sim.exe

out.dat

Technique: Identity Boxing

file

server

/ directory ACL:

hostname:*.cse.nd.edu RWLX

globus:/O=NotreDame/* RWLX

Identity Box:

/O=NotreDame/CN=Monk

sim.exe

in.dat

unified semantics
Unified Semantics
  • Same Identity for Exec and Data Access
    • Stage in data as user X.
    • Program runs as user X, data is protected.
    • Access results as user X.
  • Same ACLs for Exec and Data Access
    • Need the X right to run a program.
    • RX rights – given user can run fixed F(X).
    • WX rights – given user can stage in any F(X).
outline of paper36
Outline of Paper
  • Centralized vs Decentralized Security
  • Architecture of Cooperative Storage
  • Basic Security Mechanism
    • Problem: Complexity Confuses!
    • Detail: Reservation Right
  • Challenges
    • Authorization in Distributed File Systems
    • Logistics of Third Party Transfer
    • Mechanisms for Active Storage
    • Semantics of Distributed Group Management
fully decentralized user groups
Fully Decentralized User Groups
  • Distributed Orgs Have Complex Needs
    • CMS Collaboration: 10s of institutions, 100s of PIs, 1000s of graduate students staff.
    • There is no centralized database for CMS.
    • Local managers add/remove members locally.
  • Want Storage Systems that Allow Reference to Groups Managed by Others:
    • Allow access to all staff involved in CMS.
    • Allow access to any NSF program manager.
    • Allow access to all CS faculty at ND/Purdue.
fully decentralized acls

read

client

file

server

check

ACL

Access Control List

group:ccl.nd.edu/faculty RWL

group:serv.nsf.gov/managers RL

group:ftp.cern.org/members RL

file

system

file

univ.edu

univ.edu

group lookups

ccl.nd.edu

serv.nsf.gov

ftp.cern.org

members

members

faculty

managers

members

Fully Decentralized ACLs

group

lookups

challenges of acls
Challenges of ACLs
  • Performance / Availability / Consistency
    • Give the group/ACL owner control.
    • Specify maximum time for stale data.
  • Implemented, but continuing experience leads to reflection on the semantics.
  • Example: What to do under failures?
    • Partial answer: servers fail quickly, client retries up to a user-controlled limit.
    • Consider: Group A gives W access, group B gives R.
    • What happens when group A is unavailable?
    • Two very different questions:
      • What rights does user X have?
      • Can user X perform a read?
outline of paper40
Outline of Paper
  • Centralized vs Decentralized Security
  • Architecture of Cooperative Storage
  • Basic Security Mechanism
    • Problem: Complexity Confuses!
    • Detail: Reservation Right
  • Challenges
    • Authorization in Distributed File Systems
    • Logistics of Third Party Transfer
    • Mechanisms for Active Storage
    • Semantics of Distributed Group Management
practical lessons
Practical Lessons
  • In a system with decentralized security...
  • Users need debugging tools!
    • Simple examples: whoami, rwhoami
  • Client software must become “heavier”
    • Must carefully parse a vast array of errors.
    • Must maintain a model of remote devices.
  • High level names must be used deep within the system software stack.
    • Run processes with subject name, not Unix UID.
for more information
For more information...

Cooperative Computing Lab

http://www.cse.nd.edu/~ccl

Cooperative Computing Tools

http://www.cctools.org

Douglas Thain

  • dthain@cse.nd.edu
  • http://www.cse.nd.edu/~dthain