1 / 30

PETAL: DISTRIBUTED VIRTUAL DISKS

PETAL: DISTRIBUTED VIRTUAL DISKS. E. K. Lee C. A. Thekkath DEC SRC. Highlights. Paper presents a distributed storage management system: Petal consists of a collection of network-connected servers that cooperatively manage a pool of physical disks

vivi
Download Presentation

PETAL: DISTRIBUTED VIRTUAL DISKS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PETAL:DISTRIBUTED VIRTUAL DISKS E. K. LeeC. A. Thekkath DEC SRC

  2. Highlights • Paper presents a distributed storage management system: • Petal consists of a collection of network-connected servers that cooperatively manage a pool of physical disks • Client see Petal as a highly available block-level storage partitioned into virtual disks

  3. Introduction • Petal is a distributed storage system that • Tolerates single component failures • Can be geographically distributed to tolerate site failures • Transparently reconfigures to expand in performance or capacity • Uniformly balances load and capacity • Provides fast efficient support for backup and recovery

  4. Petal User Interface • Petal appears to its clients as a collection of virtual disks: • Block-level interface • Lower-level service than a DFS • Makes system easier to model, design, implement and tune • Can support heterogeneous clients and applications

  5. BSD FFS BSD FFS Client view NTFS EXT2 FS NTFS Scalable Network Petal Virtualdisks

  6. BSD FFS BSD FFS Storage Server Storage Server Storage Server Physical view NTFS EXT2 FS NTFS Scalable Network

  7. Petal Server Modules Global StateModule LivelinessModule RecoveryModule Virtual toPhysical Data AccessModule

  8. Overall design (I) • All state information is maintained on servers • Clients maintain only hints • Liveness module ensures that all servers will agree on the system operational status • Uses majority consensus and periodic exchanges of “I’m alive”/”You’re alive?” messages

  9. Overall design (II) • Information describing • current members of storage system and • currently supported virtual disks is replicated across all servers • Global state module keeps this information consistent • Uses Lamport’s Paxos algorithm • Assumes fail-silent failures of servers

  10. Overall design (III) • Data access and recovery modules • Control how client data are distributed and stored • Support • Simple data striping w/o redundancy • Chained declustering • It distributes mirrored data in a way that balances load in the event of a failure

  11. Address translation (I) • Must translate virtual addresses <virtual-disk ID, offset> into physical addresses <server ID, disk ID, offset> • Mechanism should be fastandfault-tolerant

  12. Address translation (II) • Uses three replicated data structures • Virtual disk directory:translates virtual disk ID into a global map ID • Global map:locates the server responsible for translating the given offset (block number) • Physical map:Locates physical disk and computers physicaloffset within that disk

  13. Server 0 Server 2 Server 2 Server 1 VDir VDir VDir VDir GMap GMap GMap GMap PMap0 PMap2 PMap1 PMap2 Virtual to physical mapping vdiskID offset diskID and diskOffset on this server

  14. Address translation (III) • Three step process: • VDir translates virtual disk ID given by client into a GMap ID • Specified GMap finds server that can translate given offset • PMap of server translates GMap ID and offset to a physical disk and a disk offset • Last two steps are almost always performed by same server

  15. Address translation (IV) • There is one GMap per virtual disk • That GMap specifies • Tuple of servers spanned by the virtual disk • Redundancy scheme used to protect data • GMaps are immutable • Cannot be modified • Must create a new GMap

  16. Address translation (V) • PMaps are similar to page tables • Each PMap entry maps 64 KB of physical disk space • Server that performs the translation will usually perform the disk I/O • Keeping GMaps and PMaps separate minimizes amount of global information that must be replicated

  17. Support for backups • Petal supports snapshots of virtual disks • Snapshots are immutable copies of virtual disks • Created using copy-on-write • VDir maps <virtual-disk ID, epoch(?)> into <GMap ID, epoch> • Epoch identifies current version of virtual disks and snapshots of past versions

  18. Incremental reconfiguration (I) • Used to add/remove new servers and new disks • Three simple steps • Create new GMap • Update VDir entries • Redistribute the data • Challenge is to perform the reconfiguration concurrently with normal client requests

  19. Incremental reconfiguration (II) • To solve the problem • Read requests will • Try first new GMap • Switch to old GMap if new GMap has no appropriate translation • Write requests will always use new GMap

  20. Incremental reconfiguration (III) • Observe that new GMap must be created before any data are moved • Too many read requests will have to consult both GMaps • Seriously degrades system performance • Do instead incremental changes over a fenced region of a virtual disk

  21. Server 2 Server 1 Server 0 Server 3 D2 D1 D6 D5 D1 D0 D5 D4 D0 D3 D4 D7 D3 D2 D7 D6 Chained declustering (I) Virtual Disk

  22. Chained declustering (II) • If one server fails, its workload will be almost equally distributed among remaining servers • Petal uses a primary/secondary scheme for managing copies • Read requests can go to either primary or secondary copy • Write requests must go firsttoprimary copy

  23. Petal prototype • Four servers • Each has fourteen 4.3 GB disks • Four clients • Links are 155 Mb/s ATM links • Petal RPC interface has 24 calls

  24. Latency of a virtual disk

  25. Throughput of a virtual disk Throughput is mostly limited by CPU overhead (233 MHZ CPUs!)

  26. File system performance (Modified Andrew Benchmark)

  27. Conclusion • Block-level interface s simpler and more flexible than a FS interface • Use of distributed software solutions allows geographic distribution • Petal performance is acceptable but for write requests • Must wait for primary and secondary copies to be successfully updated

  28. Paxos: the main idea • Proposers propose decision values from an arbitrary input set and try to collect acceptances from a majority of the accepters • Learners observe this ratification process and attempt to detect that ratification has occurred • Agreement is enforced because only one proposal can get the votes of a majority of accepters

  29. Paxos: the assumptions • Algorithm for consensus in a message-passing system • Assumes the existence of Failure Detectors that let processes give up on stalled processes after some amount of time • Processes can act as proposers, accepters, andlearners • A process may combine all three roles

  30. Paxos: the tricky part • The tricky part is to avoid deadlocks when • There are more than two proposals • Some of the processes fail • Paxos lets • Proposers make new proposals • Accepters release their earlier votes for losing proposals

More Related